Vehicle-mounted devices

Abstract

This reduces the burden of monitoring the vehicle interior. [Solution] The in-vehicle device has a communication unit and a control unit that communicates using the communication unit, and is mounted on a vehicle. The control unit sends information to the information processing device for displaying the captured image of a passenger's face in the captured image taken inside the vehicle in a first manner (unspecified manner).

Description

Technical Field

[0007]

[0001] The present disclosure relates to an in-vehicle device.

Background Art

[0002] By photographing a vehicle while it is in motion or parked, technologies have been proposed not only for preventing accidents during driving but also for taking measures against incidents around the vehicle. Further, Patent Document 1 discloses an example of a system for monitoring the interior of a vehicle in order to prevent forgetting or confining children or the like inside the vehicle.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When a monitoring system for a parked vehicle sends the captured image of the interior of the vehicle to a monitoring center and a monitor monitors the captured image, it is required to reduce the load on the monitor.

[0005] Hereinafter, an in-vehicle device or the like that enables reduction of the load of monitoring the interior of the vehicle will be disclosed.

Means for Solving the Problems

[0006] The in-vehicle device in the present disclosure has a communication unit and a control unit that communicates by means of the communication unit, and is an in-vehicle device mounted on a vehicle. The control unit sends information for displaying the captured image on an information processing device by making the image of the face of a passenger in the captured image of the interior of the vehicle into a first mode to the information processing device.

Effects of the Invention

[0007] According to the in-vehicle device or the like in the present disclosure, it becomes possible to reduce the load of monitoring the interior of the vehicle.

Brief Description of the Drawings

[0008] [Figure 1] This is a diagram showing an example of the configuration of an information provision system. [Figure 2] This is a flowchart illustrating an example of the operation procedure for an in-vehicle device. [Figure 3] This figure shows an example of an captured image. [Modes for carrying out the invention]

[0009] The embodiments will be described below with reference to the drawings.

[0010] Figure 1 shows an example of the configuration of an information provision system in one embodiment. The information provision system 1 has one or more server devices 10, terminal devices 12, and an in-vehicle device 13 mounted on a vehicle 15, all of which are connected to each other via a network 11 so as to be able to communicate information. The server device 10 is, for example, a server computer belonging to a cloud computing system or other computing system and functioning as a server that implements various functions. The terminal device 12 is an information processing terminal that exchanges various information with the server device 10, and is, for example, a personal computer, a tablet terminal device, etc. The vehicle 15 is a bus vehicle that transports passengers such as infants and children, and is configured to charge the battery of the battery device 17 while the engine is running, and is an internal combustion engine vehicle, a hybrid vehicle (HEV; Hybrid Electric Vehicle), a plug-in hybrid vehicle (PHEV; Plug-in Hybrid Electric Vehicle), etc. The in-vehicle device 13 is a computer having communication functions and information processing functions, and is connected to the in-vehicle monitoring system 16 and the battery device 17 so as to be able to communicate, and controls the operation of the monitoring system 16 and the battery device 17. The monitoring system 16 includes one or more cameras 14 positioned to capture images of the interior of the vehicle, along with their control circuits, and a speaker 18 for audio output, along with its control circuit, and operates on power supplied from the battery device 17. The battery device 17 includes a battery, such as a lithium-ion battery, capable of supplying power to various mechanisms of the vehicle 15 and the monitoring system 16, along with a control circuit for controlling its charging and discharging. The network 11 includes, for example, a mobile communication network, the Internet, an ad-hoc network, a LAN (Local Area Network), a MAN (Metropolitan Area Network), or other networks, or any combination thereof.

[0011] Information provision system 1 supports the provision of information to supervisors, guardians, etc., to prevent passengers from being left behind or trapped in vehicle 15. When vehicle 15 has finished dropping off or picking up passengers and parked in a designated parking area such as a garage, the monitoring system 16, under the control of the in-vehicle device 13, uses a camera 14 to capture images of the interior of the vehicle. The in-vehicle device 13 acquires the captured images from the camera 14 and sends them to the server device 10 via the network 11. The server device 10 sends the captured images to a terminal device 12 used by a monitor at the monitoring center. The terminal device 12 displays the captured images to the monitor, allowing the monitor to visually grasp the situation inside vehicle 15 and recognize if a passenger has been left behind or trapped. When a monitor recognizes that a passenger has been left behind or trapped, the terminal device 12 used by the monitor can send a warning notification to a terminal device 12 used by the passenger's supervisor or guardian, or send a warning sound output instruction to the in-vehicle device 13 to output a warning sound to the speaker 18 of the monitoring system 16. When a security officer visually inspects the captured images, it is sufficient to confirm the presence or absence of passengers; images of passengers' faces are considered superfluous information and increase the burden on the visual inspection. In this embodiment, the in-vehicle device 13 or server device 10 operates to alleviate this burden.

[0012] The in-vehicle device 13 of this embodiment is installed in a vehicle 15 used for transporting passengers. The in-vehicle device 13 sends information to an information processing device, i.e., a terminal device 12, to display the captured image of a passenger's face in the captured image taken inside the vehicle 15 in a predetermined manner (hereinafter referred to as the "unspecified manner"), causing the terminal device 12 to display the captured image. Here, the unspecified manner is a manner in which the face cannot be identified, and includes manners in which the face portion is masked, blurred, replaced with an arbitrary image, etc. In the following, image processing that makes the face portion into an unspecified manner will be referred to as the unspecified processing. The unspecified processing may be performed in the in-vehicle device 13, or it may be performed in the server device 10 on the captured image sent from the in-vehicle device 13. In this way, the terminal device 12 displays the captured image in which the face portion has been made into an unspecified manner, so the burden on the monitor who is visually inspecting the captured image to judge unnecessary information is reduced. Thus, it becomes possible to reduce the burden of monitoring inside the vehicle.

[0013] An example configuration of the in-vehicle device 13 will be described. The in-vehicle device 13 includes a communication unit 131, a storage unit 132, a control unit 133, a positioning unit 134, an input unit 135, an output unit 136, and a detection unit 137. These may be configured as a single control device, or as two or more control devices, or as a control device and other devices such as a communication device. The control device includes, for example, an ECU (Electronic Control Unit). The communication device includes, for example, a DCM (Data Communication Module). Each unit is connected to each other or to the equipment of the vehicle 15 via an in-vehicle network compliant with standards such as CAN (Controller Area Network) to enable information communication. The in-vehicle device 13 may also include some equipment equivalent to that of the terminal device 12. The in-vehicle device 13 operates by receiving power from a built-in battery or a battery device 17.

[0014] The communication unit 131 includes modules that support mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation), and modules that support in-vehicle LANs such as CAN. The in-vehicle device 13 is connected to the network 11 via the communication unit 131 through a nearby router device or a mobile communication base station, and communicates information with other devices via the network 11, or communicates information with the monitoring system 16 and the battery device 17 via the in-vehicle LAN.

[0015] The storage unit 132 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. Semiconductor memories are, for example, RAM (Random Access Memory) or ROM (Read Only Memory). RAM is, for example, SRAM (Static RAM) or DRAM (Dynamic RAM). ROM is, for example, EEPROM (Electrically Erasable Programmable ROM). The storage unit 132 functions, for example, as a main memory, auxiliary memory, or cache memory. The storage unit 132 stores information used in the operation of the control unit 133 and information obtained through the operation of the control unit 133.

[0016] The control unit 133 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processor is a general-purpose processor such as a CPU (Central Processing Unit), or a dedicated processor such as a GPU (Graphics Processing Unit) specialized for specific processing. The dedicated circuit is, for example, an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. The control unit 133 controls each part of the in-vehicle device 13 and performs information processing related to the operation of the in-vehicle device 13. The functions of the control unit 133 are realized by executing control and processing programs on the processor included in the control unit 133. Alternatively, some or all of the functions of the control unit 133 may be realized by the dedicated circuit included in the control unit 133.

[0017] The positioning unit 134 includes one or more GNSS (Global Navigation Satellite System) receivers. The GNSS includes, for example, at least any one of GPS (Global Positioning System), QZSS (Quasi-Zenith Satellite System), BeiDou, GLONASS (Global Navigation Satellite System), and Galileo. The positioning unit 134 sends the positioning result to the control unit 133, and the control unit 133 obtains the position information of the in-vehicle device 13.

[0018] The input unit 135 includes one or more input interfaces. The input interface is, for example, a microphone that receives voice input, a physical key, a capacitive key, a pointing device, a touch screen provided integrally with a display, etc. The input unit 135 receives an operation for inputting information used for the operation of the control unit 133, and sends the input information to the control unit 133.

[0019] The output unit 136 includes one or more output interfaces. The output interface is, for example, a speaker or a display. The display is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display. The output unit 136 outputs the information obtained by the operation of the control unit 133.

[0020] The detection unit 137 is an interface with one or more sensors that detect the states of each part of the vehicle 15, or has one or more sensors. The sensors include sensors that detect the on / off of the accessory power supply of the vehicle 15, the start / stop of the engine, etc., a vehicle speed sensor, etc. The detection unit 137 sends various information obtained by the sensors to the control unit 133.

[0021] FIG. 2 is a flowchart for explaining the operation procedure of the in-vehicle device 13 in the present embodiment. Each step in FIG. 2 is a step of information processing executed by the control unit 133. The procedure in FIG. 2 is executed, for example, at an arbitrary cycle, for example, a cycle of several tens of microseconds to several seconds, after the control unit 133 detects that the engine of the vehicle 15 has stopped and the accessory power supply has been cut off, and instructs the battery device 17 to supply power to the monitoring system 16 to start monitoring inside the vehicle 15 by the monitoring system 16.

[0022] In S301, the control unit 133 determines whether or not the reference time has elapsed since the start of monitoring by the monitoring system 16. The control unit 133 starts timing when starting the monitoring by the monitoring system 16, obtains the elapsed time for each processing cycle in FIG. 2, compares it with the reference time, and determines whether or not the reference time has elapsed. The reference time is an arbitrarily set time, and is 10 minutes to several tens of minutes. When the reference time has not elapsed (Yes), the control unit 133 proceeds to step S302, and acquires a captured image. The control unit 133 acquires a captured image inside the vehicle 15 captured by the camera 14 of the monitoring system 16 from the monitoring system 16. The control unit 133 may acquire a captured image for each frame rate of the camera 14, or may acquire a captured image of an arbitrary frame, for example, the most recent captured image. When a plurality of cameras 14 are provided, the procedure in FIG. 2 can be executed for each camera. When the reference time has elapsed in step S301 (No), the control unit 133 ends the processing cycle in FIG. 2.

[0023] In S303, the control unit 133 detects a face image in the acquired captured image. The control unit 133 detects one or more passenger face images from the captured image by any image processing such as edge detection and pattern matching. For example, in the captured image 40a shown in FIG. 3(A), a face image 41 of a passenger inside the vehicle 15 and a face image 42 of a person outside the vehicle 15 through the vehicle window 43 are detected.

[0024] In step S304, the control unit 133 determines whether a face image has been detected inside the vehicle 15. If a face image 41 has been detected, for example, as in the captured image 40a in Figure 3(A) (Yes), the control unit 133 proceeds to step S305 and determines that the face image detected inside the vehicle 15 is subject to unspecified processing. If no face image has been detected inside the vehicle 15 in step S304 (No), the process proceeds to step S306.

[0025] In step S306, the control unit 133 determines whether a face image has been detected outside the vehicle 15. If a face image 42 has been detected, for example, as in the captured image 40a in Figure 3(A) (Yes), the control unit 133 proceeds to step S307 and determines that the face image detected outside the vehicle 15 is subject to unspecified processing. If no face image has been detected outside the vehicle 15 in step S306 (No), the control unit 123 proceeds to step S309.

[0026] In S308, the control unit 133 adds information to the captured image for outputting a warning. The warning is a string of text or an audio message indicating the presence of a person outside the vehicle, for example, "There is a person outside the vehicle."

[0027] In S309, the control unit 133 determines whether there are any facial images that have been determined to be subject to despecification processing either inside or outside the vehicle 15, or both. If there are facial images that have been determined to be subject to despecification processing (Yes), the control unit 133 proceeds to step S310 and performs despecification processing on the captured image. For example, as shown in Figure 3(B), the control unit 133 masks the facial images 41 and 42 to generate a captured image 40b in which the faces have been despecificated. In a modified example, as shown in Figure 3(C), the control unit 133 may despecificate the facial image 42 by masking the entire scenery outside the vehicle window 43. When multiple people's faces are reflected outside the vehicle, the processing load on the processor can be reduced by simplifying the process. In Figures 3(A) to 3(C), examples are shown where people exist both inside and outside the vehicle 15. If a person exists only inside the vehicle 15, the face of the person inside the vehicle 15 is masked. If a person exists only outside the vehicle 15, the face of the person outside the vehicle 15 is masked, or the window is masked. If, in step S309, there is no face image determined to be the target of the unspecified processing (No), the control unit 133 proceeds to step S311.

[0028] In S311, the control unit 133 sends out the captured images. The control unit 133 sends out to the server device 10 either the captured images that have been processed to despecify faces inside and outside the vehicle 15, or the captured images that do not contain any people and therefore do not contain faces, and are therefore not processed to despecify faces.

[0029] The procedure shown in Figure 2 is executed by the control unit 133 of the in-vehicle device 13, causing the server device 10 to acquire the captured image and send it to the monitor's terminal device 12. This makes it possible to output, i.e., display, the captured image on the monitor's terminal device 12. When the monitor visually inspects the captured image, if a person's face is captured, the image will be displayed in a manner that obscures the face. Therefore, the monitor can confirm the presence or absence of a person by, for example, checking only for the presence or absence of the obscured manner, thereby reducing the monitoring burden. In addition, if a person is present outside the vehicle 15, a warning is output along with the captured image, so the monitor can be sure to notice the presence of a person outside the vehicle 15. In that case, even if the person's face 42 is obscured as in the captured image 40b in Figure 3(B), or if the entire car window 43 is obscured as in the captured image 40c in Figure 3(C), the warning is output, allowing the monitor to recognize the presence of a person outside the vehicle. For example, a monitor can instruct the in-vehicle device 13 to output a warning sound via the server device 10 from the terminal device 12. The in-vehicle device 13 outputs a warning sound, such as a buzzer sound or a voice message such as "There is someone trapped inside the vehicle, please check," through the speaker 18 of the monitoring system 16, prompting a person outside the vehicle 15 to check inside the vehicle 15.

[0030] Furthermore, the procedure shown in Figure 2 is terminated when step S301 is executed, after a reference time has elapsed since the start of monitoring, thus avoiding unnecessary consumption of power from the battery device 17.

[0031] Furthermore, the procedure shown in Figure 2 may be performed on the server device 10. In that case, for example, the processor of the server device 10 would perform each step in Figure 2. In step S301, the server device 10 receives the captured image sent from the in-vehicle device 13 and acquires the captured image. Then, in step S311, the server device 10 sends the captured image to the monitor's terminal device 12. This makes it possible to distribute the processing load across the entire information provision system 1.

[0032] Thus, according to this embodiment, it becomes possible to reduce the burden of monitoring the vehicle interior. Furthermore, it becomes possible to contribute to the protection of personal information of individuals captured in the images.

[0033] As described above, embodiments have been explained based on various drawings and examples, but it should be noted that those skilled in the art will find it easy to make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of this disclosure. For example, the functions, etc., included in each means, each step, etc., can be rearranged in a logically consistent manner, and multiple means, steps, etc., can be combined into one or divided. [Explanation of Symbols]

[0034] 1 Information provision system 10 Server equipment 11 Network 12 Terminal equipment 13 Vehicle equipment 15 Vehicle 16 Monitoring system 17 Battery unit 131 Communication unit 132 Memory unit 133 Control unit 134 Positioning unit 135 Input unit 136 Output unit 137 Detection unit

Claims

1. An in-vehicle device mounted on a vehicle, comprising a communication unit and a control unit that performs communication using the communication unit, The control unit sends information to the information processing device for displaying the captured image of a passenger's face in the captured image of the vehicle in a first manner, causing the information processing device to display the captured image. In-vehicle device.

2. An in-vehicle device mounted on a vehicle, comprising a communication unit and a control unit that performs communication using the communication unit, The control unit sends information to the information processing device for displaying the captured image of a passenger's face in the captured image of the vehicle in a first manner. In-vehicle device.

3. In claim 2, If the captured image does not include an image of the passenger's face, the control unit sends information to the information processing device for displaying the captured image in a second manner, with the image of the area outside the vehicle in the captured image. In-vehicle device.

Images