On-vehicle device and program

Abstract

To improve accuracy by reducing a processing load of processing for detecting a terminal from an image.SOLUTION: An on-vehicle device 10 includes a controller 12 for detecting a terminal by image recognition processing from an image in a cabin of a vehicle photographed by a camera 21 provided in the vehicle. The controller 12 searches for an arm, a hand, or the face among parts of a driver from the image, sets a periphery of the part detected by search as a terminal search area, and searches for a terminal present in the terminal search area.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an in-vehicle device and a program. [Background technology]

[0002] There are driver monitoring systems that monitor the driver's condition by image recognition using camera images taken inside the vehicle. For example, driver monitoring systems may detect the driver's operation of a terminal (e.g., a mobile phone) or conversation while driving in order to prevent accidents caused by the driver's so-called "distracted driving."

[0003] Note that a technology that can be used in a driver monitoring system has been proposed. For example, Patent Document 1 describes dividing video frames into full search frames that search for an object across the entire area, non-search frames that do not search for an object, and neighborhood search frames that search for an object in the neighborhood of a position where an object is detected in the full search frame. The technology described in Patent Document 1 reduces the processing load because it is not necessary to search for an object in all frames. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-171490 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the conventional technology has problems such as a large processing load and low detection accuracy.

[0006] For example, in the technology described in Patent Document 1, in order to determine the search range in the neighborhood search frame, it is necessary to first search for the object across the entire area of ​​all search frames. As such, the technology described in Patent Document 1 cannot avoid processing that has a large processing load and low detection accuracy, such as detecting a small object such as a mobile phone from the entire range of an image.

[0007] The present invention has been made in view of the above, and aims to reduce the processing load of processing to detect a terminal from an image and improve accuracy. [Means for solving the problem]

[0008] The in-vehicle device according to the present invention includes a controller that detects a terminal by image recognition processing from an image of the interior of the vehicle captured by a camera mounted on the vehicle. The controller searches for a driver's body part, such as an arm, a hand, or a face, from the image, sets the area around the detected body part as a terminal search area, and searches for a terminal present in the terminal search area. [Effects of the Invention]

[0009] According to the present invention, by setting a terminal search area based on the driver's body parts before searching for a terminal, it is possible to avoid the process of searching for small objects from a wide area, thereby reducing the processing load of the process of detecting a terminal from an image and improving accuracy. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an in-vehicle device according to an embodiment. [Figure 2] FIG. 2 is a functional block diagram of the controller. [Figure 3] FIG. 3 is a flowchart showing a processing procedure of the in-vehicle device according to the embodiment. [Figure 4] FIG. 4 is a diagram showing an example of an image. [Figure 5] FIG. 5 is a diagram illustrating a terminal search area. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the in-vehicle device and the program disclosed in the present application will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described below.

[0012] The present embodiment is realized by an in-vehicle device provided in a vehicle. The in-vehicle device is, for example, a drive recorder. However, some or all of the functions of the in-vehicle device may be realized by a server or the like connected to the vehicle via a network.

[0013] The configuration of an in-vehicle device 10 will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an in-vehicle device according to an embodiment. As shown in Fig. 1, the in-vehicle device 10 includes an interface 11, a controller 12, and a memory 13.

[0014] The in-vehicle device 10 is also connected to devices provided in the vehicle, such as a camera 21, a speed sensor 22, a speaker 23, and a display 24.

[0015] The camera 21 captures images of the interior of the vehicle. For example, the camera 21 is provided near the rearview mirror or on the dashboard with the lens facing toward the driver's seat.

[0016] The speed sensor 22 measures the speed of the vehicle. The speaker 23 is a device that outputs sound. The display 24 is a device that outputs images. The in-vehicle device 10 can exchange data and signals with each device. The in-vehicle device 10 may also be connected to other ECUs (Electronic Control Units) and the like (not shown).

[0017] The interface 11 inputs and outputs data between the in-vehicle device 10 and other devices. For example, the interface 11 is a port compatible with USB (Universal Serial Bus), Ethernet (registered trademark), or the like.

[0018] The controller 12 reads and executes a program stored in the memory 13. The controller 12 is a central processing unit (CPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a graphics processing unit (GPU), a system on a chip (SoC), or the like.

[0019] The controller 12 may be a single processor, a multiprocessor configuration, or a multicore configuration having multiple cores in a single chip connected via a single socket.

[0020] The controller 12 executes a program to realize each processing unit shown in Fig. 2. Fig. 2 is a functional block diagram of the controller.

[0021] As shown in FIG. 2, the controller 12 includes an acquisition unit 121, a first search unit 122, a setting unit 123, a second search unit 124, and an output control unit 125.

[0022] The acquisition unit 121 acquires data from each device connected to the controller 12. The acquisition unit 121 acquires images captured by the camera 21. The acquisition unit 121 also acquires the speed of the vehicle from the speed sensor 22.

[0023] The first search unit 122 searches for a body part of the driver from the image. The first search unit 122 searches for an arm, a hand, or a face. Note that an arm is a body part from the shoulder to the fingertips. Also, a hand is a body part from the wrist to the fingertips. However, the definition of a body part is not limited to that shown here.

[0024] The first search unit 122 can search for an object by image recognition using a machine learning technique. The first search unit 122 may perform the search using a convolutional neural network (CNN). Furthermore, the first search unit 122 may perform the search using any technique other than the machine learning technique, such as template matching.

[0025] Here, search refers to a process of detecting a specific object through image recognition. Furthermore, detection refers to identifying the position of an object in an image as a result of the search. The first search unit 122 can detect a target part by searching for a part of the driver's body from the image. In other words, when the first search unit 122 detects a part, information indicating the position of the part (for example, coordinates) can be used in subsequent processes.

[0026] Setting unit 123 sets the periphery of the part detected by first search unit 122 as the terminal search area. For example, setting unit 123 sets a rectangular area with predetermined length and horizontal sides, centered (at the intersection of diagonals) at the coordinates of the detected part, as the terminal search area.

[0027] The second search unit 124 searches for a terminal present in the terminal search area. The second search unit 124 searches for an object using the same method as the first search unit 122. In this embodiment, the terminal searched for by the second search unit 124 is a mobile phone that can be held in one hand by the driver. The shape of the mobile phone may be a flat, so-called smartphone shape, or a foldable type.

[0028] In this way, the controller 12 does not immediately search for the terminal in the acquired image, but first searches for the driver's body parts (face, arms, hands) whose appearance is thought to change when driving and when operating the terminal or making a call. Then, the controller 12 searches for the terminal from around the searched body parts.

[0029] As a result, in this embodiment, it is possible to avoid detecting small objects such as mobile phones (an example of a terminal) from the entire range of an image, thereby not only reducing the processing load but also reducing false positives in searches and improving accuracy.

[0030] The output control unit 125 controls the speaker 23 and the display 24. For example, the output control unit 125 causes the speaker 23 to output sound. Also, the output control unit 125 causes the display 24 to output an image.

[0031] 1, the memory 13 is a storage medium such as an embedded multi-media card (eMMC), etc. The memory 13 functions as a read-only memory (ROM) or a random access memory (RAM).

[0032] The processing flow of the in-vehicle device 10 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the processing procedure of the in-vehicle device according to the embodiment.

[0033] 3, first, the acquisition unit 121 acquires the speed of the vehicle (step S101). That is, the acquisition unit 121 acquires the sensor value of the speed sensor 22.

[0034] Next, the first search unit 122 determines whether the vehicle is moving based on the vehicle speed (step S102). For example, the first search unit 122 determines that the vehicle is moving when the vehicle speed exceeds 0 km / h.

[0035] If the vehicle is not running (No at step S103), the controller 12 returns to step S101 and repeats the process.

[0036] In this way, the controller 12 searches for the driver's arms, hands, or face from the image when the vehicle speed is greater than 0. This makes it possible to prevent the search from being performed even in a situation where terminal operation and conversation are permitted (for example, while the vehicle is stopped).

[0037] Here, it is assumed that camera 21 simultaneously captures a low-resolution (e.g., 1280 pixels x 720 pixels) image and a high-resolution (e.g., 1920 pixels x 1080 pixels) image. The low-resolution image is an example of a first image, which is an image of the interior of the vehicle captured by camera 21. The high-resolution image is an example of a second image, which is an image of the interior of the vehicle captured by camera 21 and has a higher resolution than the first image.

[0038] Although the camera 21 is said to capture the first image and the second image simultaneously, it may also capture the second image, which is a high-resolution image, and generate the first image, which is a low-resolution image, based on the second image through image processing.

[0039] If the vehicle is traveling (step S103, Yes), the acquisition unit 121 acquires a low-resolution image of the interior of the vehicle cabin from the camera 21 (step S104). That is, the acquisition unit 121 acquires an image as shown in Fig. 4. Fig. 4 is a diagram showing an example of the image. As shown in Fig. 4, an image 50 shows a driver operating a steering wheel.

[0040] 3, the first search unit 122 searches for the face and arms of the driver from the image acquired by the acquisition unit 121 (step S105). The first search unit 122 may search for the hands together with or instead of the arms.

[0041] Furthermore, the first search unit 122 measures the distance from the arm to the handlebars (step S106). For example, the first search unit 122 searches for the handlebars in the image and calculates the distance on the image between the position of the detected arm and the position of the detected handlebars.

[0042] If the measured distance is not equal to or greater than the threshold (No in step S107), the controller 12 returns to step S101 and repeats the process. That is, if the controller 12 determines that the driver's hands have not left the steering wheel, it assumes that the driver is not operating the terminal or making a call using the terminal, and interrupts the process of searching for the terminal.

[0043] If the measured distance is equal to or greater than the threshold (step S107, Yes), the acquisition unit 121 acquires a high-resolution image of the interior of the vehicle from the camera 21 (step S108). That is, when the controller 12 determines that the driver's hands are off the steering wheel, it assumes that the driver may be operating the terminal and making a call using the terminal, and continues the process of searching for the terminal.

[0044] In this way, the controller 12 searches for a terminal present in the terminal search area when the distance between the arm or hand detected by the search and the steering wheel of the vehicle is equal to or greater than a threshold value.

[0045] When the driver's arms (or hands) are close enough to the steering wheel, the driver is driving the vehicle and the risk of operating a terminal, etc. is considered to be low. On the other hand, when the driver's arms (or hands) are away from the steering wheel, the driver may be neglecting driving the vehicle and operating a terminal, etc., and the risk is considered to be high. By determining when the risk is high, the controller 12 can reduce false detections and perform efficient searches.

[0046] The setting unit 123 sets the area around the face and arms detected by the search as the terminal search area (step S109). Areas 501, 502, and 503 in Fig. 5 are terminal search areas set by the setting unit 123. Fig. 5 is a diagram illustrating the terminal search area.

[0047] The second search unit 124 searches for a terminal within the terminal search area (step S110). The second search unit 124 performs a search using the high-resolution image acquired by the acquisition unit 121. In this way, the second search unit 124 performs a search using the high-resolution image for the first time after the terminal search area is set. This reduces the processing load.

[0048] If there is no terminal in the terminal search area (step S111, No), that is, if no terminal is detected as a result of the search, the controller 12 returns to step S101 and repeats the process.

[0049] If a terminal is present in the terminal search area (step S111, Yes), that is, if a terminal is detected as a result of the search, the output control unit 125 outputs a warning (step S112). For example, the output control unit 125 causes the speaker 23 to output a warning sound or a voice message instructing the user to stop operating the terminal. Also, for example, the output control unit 125 causes the display 24 to output a text message instructing the user to stop operating the terminal.

[0050] In this way, when a terminal present in the terminal search area is detected, the output control unit 125 notifies the driver of a warning via the output device (for example, the speaker 23 and the display 24). As a result, the output control unit 125 can reflect the detection result in the notification to the user with high accuracy, thereby improving safety.

[0051] In step S106, the second search unit 124 may analyze the image in more detail to determine whether the driver is operating the terminal or whether the driver is making a call using the terminal.

[0052] Further advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. [Explanation of symbols]

[0053] 10 Onboard equipment 11 Interface 12 Controllers 13. Memory 21 Camera 22 Speed ​​sensor 23 Speaker 24 displays 121 Acquisition Department 122 First Search Section 123 Settings 124 Second Search Section 125 Output control section 501, 502, 503 areas

Claims

1. a controller that detects a terminal from an image of the interior of the vehicle captured by a camera provided in the vehicle through image recognition processing; Searching for the driver's arm, hand, or face from the image; setting the periphery of the part detected by the search as a terminal search area; Searching for terminals present in the terminal search area; In-vehicle device.

2. The controller searching for an arm, a hand, or a face of the driver from a first image that is an image of the interior of the vehicle captured by the camera; searching for a terminal present in the terminal search area from a second image, which is an image of the interior of the vehicle captured by the camera and has a higher resolution than the first image; The in-vehicle device according to claim 1 .

3. The controller If the distance between the arm or hand detected by the search and the steering wheel of the vehicle is equal to or greater than a threshold, a search is performed for a terminal present in the terminal search area. The in-vehicle device according to claim 1 .

4. The controller When the speed of the vehicle is greater than 0, searching for an arm, a hand, or a face of the driver from the image. The in-vehicle device according to claim 1 .

5. The controller If a terminal present in the terminal search area is detected, a warning is sent to the driver via an output device. The in-vehicle device according to claim 1 .

6. A computer that executes a process of detecting a terminal by image recognition processing from an image of the interior of the vehicle taken by a camera installed in the vehicle, Searching for the driver's arm, hand, or face from the image; setting the periphery of the part detected by the search as a terminal search area; Searching for terminals present in the terminal search area; A program that executes a process.

Images