Vehicle-mounted device

Abstract

The present invention addresses the problem of reducing the burden on monitoring in a vehicle interior. The in-vehicle device includes a communication unit and a control unit that performs communication by the communication unit, the in-vehicle device is mounted on a vehicle, and the control unit transmits, to the information processing device, information for displaying a captured image on the information processing device in such a manner that a face image of an occupant in the captured image in the vehicle is a first manner.

Description

Technical Field

[0001] This invention relates to a vehicle-mounted device. Background Technology

[0002] A technology is proposed that takes measures not only against accidents during driving but also against events around the vehicle by photographing the vehicle while it is in motion or parked. Furthermore, Patent Document 1 discloses an example of a system for monitoring the interior of a vehicle to prevent children from being forgotten or left inside.

[0003] Patent Document 1: Japanese Patent No. 7370073 Summary of the Invention

[0004] When a vehicle is parked, the vehicle monitoring system sends video images from the vehicle's interior to the monitoring center, and the monitors monitor these images. It is necessary to reduce the workload of the monitors.

[0005] The following discloses an in-vehicle device that can reduce the burden of monitoring inside the vehicle.

[0006] The vehicle-mounted device of the present invention has a communication unit and a control unit that communicates with the communication unit. The vehicle-mounted device is mounted on a vehicle. The control unit sends information for displaying a camera image of a passenger's face in a camera image inside the vehicle on an information processing device in a first manner.

[0007] Invention Effects

[0008] The vehicle-mounted device and the like according to the present invention can reduce the burden of monitoring inside the vehicle. Attached Figure Description

[0009] Figure 1 This is a diagram illustrating an example of the structure of an information provision system.

[0010] Figure 2 This is a flowchart illustrating the operational steps of an onboard device.

[0011] Figure 3 This is a diagram showing an example of a camera image. Detailed Implementation

[0012] The embodiments will now be described with reference to the accompanying drawings.

[0013] Figure 1This diagram illustrates a structural example of an information providing system according to one embodiment. The information providing system 1 includes one or more server devices 10, terminal devices 12, and an on-board unit 13 mounted on a vehicle 15, all of which are communicatively connected to each other via a network 11. The server devices 10 are, for example, cloud computing systems or other computing systems, functioning as server computers with various installed functions. The terminal devices 12 are information processing terminals that send and receive various information with the server devices 10, such as personal computers or tablet computers. The vehicle 15 is a bus that transports passengers such as toddlers and children, and is configured as an internal combustion engine vehicle, hybrid electric vehicle (HEV), or plug-in hybrid electric vehicle (PHEV) capable of charging the battery of the battery unit 17 while the engine is running. The on-board unit 13 is a computer with communication and information processing functions, communicatively connected to the monitoring system 16 and the battery unit 17 inside the vehicle, and controls the operation of the monitoring system 16 and the battery unit 17. The monitoring system 16 includes one or more cameras 14 installed in locations capable of capturing images inside the vehicle interior, along with their control circuitry, and a speaker 18 for voice output, along with its control circuitry. It operates by receiving power from a battery unit 17. The battery unit 17 includes batteries such as lithium-ion batteries capable of supplying power to various mechanisms of the vehicle 15 and the monitoring system 16, and control circuitry for controlling their charging and discharging. The network 11 includes, for example, a mobile communication network, the Internet, an ad hoc network, a local area network (LAN), a metropolitan area network (MAN), or other networks, or any combination thereof.

[0014] Information provision system 1 assists in providing information to supervisors and protectors of vehicle 15 to prevent passengers from being forgotten or locked inside the vehicle. If vehicle 15 finishes picking up and dropping off passengers and parks in a designated parking location such as a garage, under the control of onboard device 13, monitoring system 16 captures images of the vehicle interior via camera 14. Onboard device 13 obtains the video images from camera 14 and sends them to server device 10 via network 11. Server device 10 sends the video images to terminal device 12 used by a monitor at the monitoring center. The terminal device 12 displays the video images towards the monitor, allowing the monitor to visually assess the situation inside vehicle 15 and identify instances of passengers being forgotten or locked inside. If the monitor identifies a passenger being forgotten or locked inside, they can send a warning notification from their terminal device 12 to the terminal device 12 used by the passenger's supervisor or protector, or send a warning voice output instruction to onboard device 13 to cause the speaker 18 of monitoring system 16 to output a warning voice. When a monitor visually inspects the camera images, it is sufficient to confirm the presence or absence of passengers; images of passengers' faces are considered redundant information, increasing the visual burden. In this embodiment, to alleviate this burden, the vehicle-mounted device 13 or the server device 10 is activated.

[0015] The vehicle-mounted device 13 of this embodiment is mounted on a vehicle 15 used for transporting passengers. The vehicle-mounted device 13 sends information for displaying a camera image in a predetermined manner (hereinafter referred to as a de-identification method) using a passenger's facial image from an image captured within the vehicle 15 to an information processing device, namely a terminal device 12, so that the terminal device 12 displays the camera image. Here, the de-identification method refers to a method where the face cannot be recognized, including methods such as obscuring the face, blurring it, or replacing it with an arbitrary image. Furthermore, the image processing that de-identifies the face is referred to as de-identification processing. De-identification processing can be performed in the vehicle-mounted device 13 or in the server device 10 on the camera image sent from the vehicle-mounted device 13. Therefore, by displaying a camera image with the face de-identified in the terminal device 12, the burden of judging unnecessary information is reduced for the monitor who is visually viewing the camera image. Thus, the burden of monitoring inside the vehicle can be reduced.

[0016] The structure of the vehicle-mounted device 13 will be described below. The vehicle-mounted device 13 includes a communication unit 131, a storage unit 132, a control unit 133, a positioning unit 134, an input unit 135, an output unit 136, and a detection unit 137. These can be configured as a single control device, or as two or more control devices, or control devices combined with communication equipment and other devices. Control devices may include, for example, an Electronic Control Unit (ECU). Communication equipment may include, for example, a Data Communication Module (DCM). Each component can be connected to each other or to the equipment of the vehicle 15 via an in-vehicle network conforming to standards such as Controller Area Network (CAN). Furthermore, the vehicle-mounted device 13 may include a portion of equipment identical to that of the terminal device 12. The vehicle-mounted device 13 operates by receiving power from a built-in battery or battery device 17.

[0017] The communication unit 131 includes modules corresponding to mobile communication standards such as Long Term Evolution (LTE), 4th Generation (4G), or 5th Generation (5G), as well as modules corresponding to in-vehicle LANs such as CAN. The vehicle-mounted device 13 connects to the network 11 via the communication unit 131 through a nearby router or mobile communication base station, and communicates with other devices via the network 11, or with the monitoring system 16 and battery device 17 via the in-vehicle LAN.

[0018] Storage unit 132 includes one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these. Semiconductor memories are, for example, random access memory (RAM) or read-only memory (ROM). RAM is, for example, static random access memory (SRAM) or dynamic random access memory (DRAM). ROM is, for example, electrically erasable programmable read-only memory (EEPROM). Storage unit 132 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. Storage unit 132 stores information for the operation of control unit 133 and information obtained through the operation of control unit 133.

[0019] The control unit 133 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processor is a general-purpose processor such as a central processing unit (CPU) or a dedicated processor such as a graphics processing unit (GPU) for specific processing. The dedicated circuit is, for example, a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). The control unit 133 controls various parts of the vehicle-mounted device 13 while performing information processing related to the operation of the vehicle-mounted device 13. The functions of the control unit 133 are implemented by the processor included in the control unit 133 executing control / processing programs. Alternatively, some or all of the functions of the control unit 133 may also be implemented by the dedicated circuits included in the control unit 133.

[0020] The positioning unit 134 includes one or more Global Navigation Satellite System (GNSS) receivers. GNSS receivers include, for example, at least one of the following: Global Positioning System (GPS), Quasi-Zenith Satellite System (QZSS), BeiDou, GLONASS, and Galileo. The positioning unit 134 transmits the positioning results to the control unit 133, which then calculates the position information of the vehicle-mounted device 13.

[0021] The input unit 135 includes one or more input interfaces. These input interfaces may include, for example, a microphone for accepting voice input, physical keys, electrostatic capacitive keys, a pointing device, or a touchscreen integrated with the display. The input unit 135 accepts input information for the operation of the control unit 133 and sends the input information to the control unit 133.

[0022] The output unit 136 includes one or more output interfaces. The output interface may be, for example, a speaker or a display. The display may be, for example, a liquid crystal display (LCD) or an organic electroluminescence (EL) display. The output unit 136 outputs information obtained through the operation of the control unit 133.

[0023] The detection unit 137 has an interface with one or more sensors that detect the status of various parts of the vehicle 15. These sensors include sensors for detecting the on / off state of the vehicle 15's accessories, engine start / stop, etc., and vehicle speed sensors. The detection unit 137 transmits various information obtained from the sensors to the control unit 133.

[0024] Figure 2 This is a flowchart illustrating the operation steps of the vehicle-mounted device 13 in this embodiment. Figure 2 Each step in the process is an information processing step performed by the control unit 133. Figure 2 The steps, for example, are as follows: after the control unit 133 detects that the engine of the vehicle 15 has stopped and the power to the accessories has been cut off, it instructs the battery device 17 to supply power to the monitoring system 16 so that the monitoring system 16 can start monitoring inside the vehicle 15, and then executes the process at any interval, such as tens of microseconds to several seconds.

[0025] In S301, the control unit 133 determines whether a reference time has elapsed since the start of monitoring based on the monitoring system 16. The control unit 133 starts timing when the monitoring system 16 begins monitoring and calculates each... Figure 2 The processing cycle elapses and is compared with a reference time to determine whether it has elapsed before the reference time. The reference time is an arbitrarily set time, ranging from 10 minutes to several tens of minutes. If it has elapsed before the reference time, the control unit 133 proceeds to step S302 to acquire a camera image. The control unit 133 acquires camera images of the interior of the vehicle 15 captured by the camera 14 of the monitoring system 16. The control unit 133 can acquire camera images at the frame rate of the camera 14, or it can acquire any frame, such as the most recent camera image. In addition, when multiple cameras 14 are installed, Figure 2 The steps can be executed for each camera. In step S301, if the reference time has elapsed (no), the control unit 133 terminates. Figure 2 The processing cycle.

[0026] In S303, the control unit 133 detects facial images in the acquired camera images. The control unit 133 detects the facial images of more than one passenger from the camera images using arbitrary image processing such as edge detection and pattern matching. For example, in... Figure 3 In the camera image 40a shown in (A), the face image 41 of the passenger inside the vehicle 15 and the face image 42 of a person outside the vehicle 15 through the vehicle window 43 are detected.

[0027] In S304, the control unit 133 determines whether a facial image has been detected inside the vehicle 15. For example, the control unit 133... Figure 3If a face image 41 is detected as in camera image 40a (A), proceed to step S305, whereby the face image detected inside the vehicle 15 is identified as the object for de-identification processing. If no face image is detected inside the vehicle 15 in step S304 (No), proceed to step S306.

[0028] In S306, the control unit 133 determines whether a facial image has been detected outside the vehicle 15. For example, the control unit 133... Figure 3 If a face image 42 is detected as in the camera image 40a (A), the process proceeds to step S307, where the face image detected outside the vehicle 15 is identified as the object for de-identification processing. If no face image is detected outside the vehicle 15 in step S306 (No), the control unit 133 proceeds to step S309.

[0029] In S308, the control unit 133 attaches information for outputting a warning to the camera image. The warning is a string or voice message indicating the presence of a person outside the vehicle, such as "Someone is outside the vehicle".

[0030] In step S309, the control unit 133 determines whether a facial image of an object to be de-identified exists either inside or outside the vehicle 15. If a facial image of an object to be de-identified exists (yes), the control unit 133 proceeds to step S310 to perform de-identification processing on the camera image. For example, the control unit 133... Figure 3 (B) In this way, a camera image 40b is generated by masking the face images 41 and 42, and the face is de-identified. In a modified example, the control unit 133 can be as follows: Figure 3 As shown in (C), the entire landscape outside the car window 43 is masked to de-identify the face image 42. In cases where multiple faces are reflected outside the vehicle, this simplified processing reduces the processor's workload. Furthermore, in Figure 3 (A) ~ Figure 3 In (C), an example is shown where there are people both inside and outside the vehicle 15. However, if there are people only inside the vehicle 15, their faces are obscured. If there are people only outside the vehicle 15, their faces are obscured or the windows are covered. In step S309, if there is no face image of an object determined to be de-identified (No), the control unit 133 proceeds to step S311.

[0031] In S311, the control unit 133 sends the camera images. The control unit 133 sends the camera images of faces inside and outside the vehicle 15 that have been de-identified, or the camera images that do not contain people and do not contain faces and therefore do not require de-identification processing, to the server device 10.

[0032] Executed by the control unit 133 of the vehicle-mounted device 13 Figure 2 In the following steps, server device 10 acquires the camera image and sends it to the monitor's terminal device 12. Thus, the camera image can be output and displayed on the monitor's terminal device 12. When the monitor views the camera image, the image of a person's face is displayed in a de-identified manner. Therefore, the monitor can confirm the presence of a person, for example, by only checking for the presence or absence of de-identification, thereby reducing the monitoring burden. Furthermore, if a person is present outside vehicle 15, a warning is output along with the camera image, so the monitor can reliably notice the presence of a person outside vehicle 15. In this case, even if... Figure 3 Even in cases where the face 42 of a person in the photographic image 40b of (B) is de-identified, even if... Figure 3 In the case where the entire car window 43 is de-identified as shown in the camera image 40c (C), the monitor can identify the presence of a person outside the vehicle by outputting a warning. Then, for example, the monitor can instruct the on-board device 13 to output a warning sound from the terminal device 12 via the server device 10. The on-board device 13 can output a warning sound, such as a buzzer or a voice message like "Someone is locked inside the vehicle, please check," through the speaker 18 of the monitoring system 16, prompting the person outside the vehicle 15 to confirm that they are inside the vehicle 15.

[0033] In addition, by executing step S301, Figure 2 The process is stopped after a reference time has elapsed since the start of monitoring, thus avoiding unnecessary power consumption of the battery device 17.

[0034] and, Figure 2 The steps can also be performed in server device 10. In this case, for example, the processor of server device 10 performs the steps. Figure 2 The steps are as follows. In step S301, the server device 10 acquires a camera image by receiving the camera image sent from the vehicle-mounted device 13. Then, in step S311, the server device 10 sends the camera image towards the monitor's terminal device 12. This allows the processing burden of the information providing system 1 as a whole to be distributed.

[0035] Thus, according to this embodiment, the burden of monitoring inside the vehicle can be reduced. Furthermore, it helps protect the personal information of people captured in the camera images.

[0036] In the foregoing, the embodiments have been described based on the accompanying drawings and examples. However, those skilled in the art should note that various modifications and alterations can be easily made based on this invention. Therefore, it should be noted that these modifications and alterations are included within the scope of this invention. For example, the functions included in each mechanism, step, etc., can be reconfigured without logically contradicting each other, and multiple mechanisms, steps, etc., can be combined into one or divided.

[0037] Symbol Explanation

[0038] 1-Information providing system, 10-Server device, 11-Network, 12-Terminal device, 13-Vehicle device, 15-Vehicle, 16-Monitoring system, 17-Battery device, 131-Communication unit, 132-Storage unit, 133-Control unit, 134-Positioning unit, 135-Input unit, 136-Output unit, 137-Detection unit.

Claims

1. A vehicle-mounted device, characterized in that, The vehicle-mounted device, which includes a communication unit and a control unit for communication via the communication unit, is mounted on a vehicle. The control unit sends information for displaying the camera image in a first manner, using the face image of the passenger in the camera image inside the vehicle, to the information processing device, so that the information processing device displays the camera image.

2. A vehicle-mounted device, characterized in that, The vehicle-mounted device, which includes a communication unit and a control unit for communication via the communication unit, is mounted on a vehicle. The control unit sends information to the information processing device for displaying the camera image on the information processing device in a first manner, using the face image of the passenger in the camera image inside the vehicle.

3. The vehicle-mounted device according to claim 2, characterized in that, If the camera image does not contain the passenger's face, the control unit sends information to the information processing device for displaying the camera image on the information processing device in a second manner, using the image outside the vehicle in the camera image.

Images