Center device, in-vehicle device, and safety confirmation system

Abstract

To make it possible to confirm safety of vehicle occupants using an in-vehicle video image even when the in-vehicle video image is being turned off.SOLUTION: A center device according to an embodiment comprises a controller configured to receive vehicle information regarding the behavior of a vehicle from an in-vehicle device, request an in-vehicle camera footage from the in-vehicle device upon detection of abnormality from the vehicle information, transmit a control signal for forcing the in-vehicle camera footage to be transmitted when there is no response to the request from the in-vehicle device, and acquire the in-vehicle camera footage from the in-vehicle device.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The disclosed embodiments relate to a center device, an in-vehicle device, and a safety confirmation system. [Background technology]

[0002] Conventionally, a technology has been proposed in which, when a vehicle falls into an emergency state due to an accident or the like, a notification is sent from an on-board device mounted on the vehicle to a notification center (for example, see Patent Document 1). With this technology, when the acceleration of the vehicle exceeds a predetermined threshold, it is estimated that a collision or other accident with a large impact has occurred, and a notification is sent to the notification center.

[0003] In such a case, it is preferable to be able to confirm the safety of the vehicle occupants based on, for example, camera images. In this regard, a technology has been proposed in which a third party remotely monitors camera images from inside the vehicle (hereinafter, appropriately referred to as "in-vehicle images") (see, for example, Patent Document 2). It is considered that by using such a technology, it is possible to confirm the safety of the occupants using in-vehicle images. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-176566 A [Patent Document 2] JP 2023-007135 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned conventional technology has room for further improvement in terms of making it possible to check the safety of occupants using in-vehicle images even when the in-vehicle images are set to an off state.

[0006] For example, a vehicle occupant may prioritize privacy protection and set the in-vehicle camera that captures in-vehicle images to an off state. In this case, the in-vehicle images cannot be transmitted to the notification center, and it is difficult for an operator at the notification center to confirm the safety of the occupants using the in-vehicle images even if an accident or the like occurs.

[0007] One aspect of the embodiment has been made in consideration of the above, and aims to provide a center device, an in-vehicle device, and a safety confirmation system that make it possible to check the safety of occupants using in-vehicle images even when the in-vehicle images are set to an off state. [Means for solving the problem]

[0008] The center device in one aspect of the embodiment includes a controller that receives vehicle information regarding the behavior of the vehicle from an in-vehicle device, requests camera footage of the interior of the vehicle cabin from the in-vehicle device when an abnormality is detected from the vehicle information, and, if there is no response to the request from the in-vehicle device, sends a control signal to forcibly transmit the camera footage of the interior of the vehicle cabin, and acquires the camera footage of the interior of the vehicle cabin from the in-vehicle device. Effect of the Invention

[0009] According to one aspect of the embodiment, when the controller of the center device receives, for example, acceleration corresponding to a strong impact as vehicle information from the in-vehicle device, the controller transmits a control signal to the in-vehicle device to forcibly transmit camera images of the interior of the vehicle if the in-vehicle device is unable to acquire the camera images. The in-vehicle device forcibly transmits the camera images of the interior of the vehicle to the center device in response to the control signal, and the controller of the center device acquires the camera images of the interior of the vehicle that have been forcibly transmitted from the in-vehicle device. This allows the controller of the center device to acquire camera images of the interior of the vehicle in, for example, a situation in which an occupant is estimated to be in a critical condition, even if the occupant has set the in-vehicle camera to an off state to protect privacy. That is, according to one aspect of the embodiment, even if the in-vehicle camera is set to an off state, it is possible to realize confirmation of the safety of the occupant using the in-vehicle image. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic explanatory diagram of a safety confirmation method according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of a safety confirmation system according to the embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of the configuration of the drive recorder according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of the configuration of a server device according to the embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of the configuration of the operator terminal according to the embodiment. [Figure 6] FIG. 6 is a diagram (part 1) showing a processing sequence executed by the safety confirmation system according to the embodiment. [Figure 7] FIG. 7 is a diagram (part 2) showing a processing sequence executed by the safety confirmation system according to the embodiment. [Figure 8] FIG. 8 is a diagram (part 1) showing a specific example of the dialogue screen. [Figure 9] FIG. 9 is a diagram (part 2) showing a specific example of the dialogue screen. [Figure 10] FIG. 10 is a diagram (part 3) showing a specific example of the dialogue screen. [Figure 11] FIG. 11 is a diagram (part 3) showing a processing sequence executed by the safety confirmation system according to the embodiment. [Figure 12] FIG. 12 is a diagram showing a specific example of the notification screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the center device, the vehicle-mounted device, and the safety confirmation system disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below.

[0012] In the following, the center device according to the embodiment is assumed to be a server device 100 included in a safety confirmation system 1 (see FIG. 2 onwards). In the following, the in-vehicle device according to the embodiment is assumed to be a communication type drive recorder 10 (see FIG. 2 onwards) that is provided so as to be able to communicate with the server device 100. In the following, the safety confirmation method according to the embodiment is assumed to be a safety confirmation method executed by a controller 103 (see FIG. 4) of the server device 100.

[0013] In addition, the expressions "specific," "predetermined," and "constant" in the following description may be read as "predetermined."

[0014] First, an overview of the safety confirmation method according to the embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram illustrating the overview of the safety confirmation method according to the embodiment.

[0015] In the safety confirmation method according to the embodiment, the controller 103 of the server device 100 receives vehicle information related to the behavior of the vehicle from the drive recorder 10. When the controller 103 detects an abnormality from the vehicle information, the controller 103 requests an in-vehicle video from the drive recorder 10. When there is no response to this request from the drive recorder 10, the controller 103 transmits a control signal for forcibly transmitting the in-vehicle video, and acquires the in-vehicle video from the drive recorder 10.

[0016] Specifically, the controller 103 executes a stepwise process of confirming the safety of the vehicle occupants when the drive recorder 10 detects an acceleration equal to or greater than a predetermined threshold (corresponding to an example of "strong acceleration") as a strong impact. At this time, the controller 103 executes a stepwise process of confirming the safety of the vehicle occupants according to the on or off state of the in-vehicle camera 13b (see FIG. 3) of the drive recorder 10. The stage of the safety confirmation process is associated in advance with the danger level of the occupants, and only when the stage of the safety confirmation process indicates a high danger level exceeding the threshold, the controller 103 forcibly turns on the in-vehicle camera 13b if it is off.

[0017] More specifically, as shown in Fig. 1, the drive recorder 10 has two setting states: an "in-vehicle image on setting" in which the in-vehicle camera 13b is turned on, and an "in-vehicle image off setting" in which the in-vehicle camera 13b is turned off. In the "in-vehicle image off setting", the drive recorder 10 does not transmit the in-vehicle image to the server device 100 even if it receives a request for the in-vehicle image from the server device 100. Vehicle occupants often prioritize privacy protection and set the "in-vehicle image off setting".

[0018] However, when a strong impact occurs due to an accident or the like, it is necessary to confirm the safety of the occupants, so it is preferable to use the in-vehicle video in order to appropriately confirm the safety. However, if the occupants have the in-vehicle camera 13b turned off, it is preferable to be able to confirm the safety of the occupants without using the in-vehicle video as much as possible from the viewpoint of protecting their privacy.

[0019] Therefore, in the safety confirmation method according to the embodiment, the controller 103 executes a first stage safety confirmation process, a second stage safety confirmation process, and a third stage safety confirmation process in stages according to the on / off state of the in-vehicle camera 13b. This makes it possible to perform the minimum necessary safety confirmation while comparing the priority of privacy protection of the occupant and the danger level of the occupant according to the on / off state of the in-vehicle camera 13b.

[0020] Each stage of the safety confirmation process is associated with a level of danger to the occupants, with the first stage being the highest, followed by the second stage and then the third stage, as shown in FIG.

[0021] First, when an accident occurs, regardless of the stage of the safety confirmation process, the drive recorder 10 goes into "forced call mode." In this mode, the drive recorder 10 is unconditionally connected to the call center by the server device 100. This makes it possible for the call center to at least attempt to confirm the safety of the driver by voice when an accident occurs.

[0022] 1, when the in-vehicle camera 13b is in the on state, that is, "in-vehicle image on setting", the controller 103 executes the first stage of safety confirmation processing to confirm the safety using the in-vehicle image, the in-vehicle image, and the audio, as shown in FIG. 1. When the in-vehicle image on setting is received from the server device 100, the drive recorder 10 transmits the in-vehicle image to the server device 100.

[0023] In addition, in the case of "in-vehicle image off setting" where the in-vehicle camera 13b is off, if dialogue with the occupant by voice or interactive operation is possible, the controller 103 executes a second stage safety confirmation process in which the safety confirmation is performed by using the outside-vehicle image and voice or interactive operation. In the second stage safety confirmation process, the priority of protecting the privacy of the occupant is higher than in the first stage safety confirmation process. As a result, when the in-vehicle camera 13b is off, it is possible to perform a safety confirmation that prioritizes privacy protection more than when the in-vehicle camera 13b is on.

[0024] The interactive operation refers to an operation via an interactive screen that the controller 103 causes the drive recorder 10 to display. Specific examples of the interactive screen will be described later with reference to FIGS.

[0025] In addition, in the case of "in-vehicle image off setting", if dialogue with the occupant by voice or interactive operation is not possible, the controller 103 judges that the danger level of the occupant is high and forcibly turns on the in-vehicle camera 13b ("forcible in-vehicle image on"). In the case of "forcible in-vehicle image on", when the drive recorder 10 receives a request for in-vehicle image from the server device 100, it forcibly transmits the in-vehicle image to the server device 100. Then, the controller 103 executes a third-stage safety confirmation process to confirm the safety of the occupant using the outside-vehicle image and the in-vehicle image. In the third-stage safety confirmation process, the priority of privacy protection of the occupant is lower than in the second-stage safety confirmation process. As a result, when the in-vehicle camera 13b is off and dialogue is not possible, it is possible to perform a safety confirmation with a lower priority of privacy protection than when the in-vehicle camera 13b is off and dialogue is possible.

[0026] Note that the controller 103 also executes the third-stage safety confirmation process when the response from the occupant is interrupted during the second-stage safety confirmation process. That is, in a situation where the occupant is estimated to be in a critical condition, the controller 103 executes the third-stage safety confirmation process in which the in-vehicle camera 13b is turned on, prioritizing privacy protection.

[0027] Then, the controller 103 executes arrangements according to the confirmation results in each safety confirmation process. Here, the controller 103 executes arrangements to dispatch necessary vehicles such as an ambulance, police car, tow truck, and fire engine, and personnel, etc., to the relevant requested destination (e.g., a medical institution, police, fire department, etc.) according to the confirmation results.

[0028] Thus, in the safety confirmation method according to the embodiment, the controller 103 receives vehicle information related to the behavior of the vehicle from the drive recorder 10. When the controller 103 detects an abnormality from the vehicle information, the controller 103 requests the vehicle interior video from the drive recorder 10. When the controller 103 does not respond to the request from the drive recorder 10, the controller 103 transmits a control signal for forcibly transmitting the vehicle interior video, and acquires the vehicle interior video from the drive recorder 10. Specifically, when the controller 103 receives a notification of strong acceleration detection (corresponding to an example of "vehicle information") from the drive recorder 10, the controller 103 detects an abnormality from the notification and executes a stepwise safety confirmation process for confirming the safety of the vehicle occupants. When the stage of the safety confirmation process previously associated with the danger level of the occupants exceeds a threshold, the controller 103 forcibly turns on the vehicle interior camera 13b if the vehicle interior camera 13b is in an off state. The controller 103 then acquires the vehicle interior video from the drive recorder 10.

[0029] As a result, according to the safety confirmation method of the embodiment, it is possible to confirm the safety of occupants using in-vehicle images even when the in-vehicle images are set to an off state.

[0030] Hereinafter, a configuration example of the safety confirmation system 1 including the server device 100 to which the safety confirmation method according to the above-described embodiment is applied will be described in more detail.

[0031] Fig. 2 is a diagram showing a configuration example of a safety confirmation system 1 according to an embodiment. As shown in Fig. 2, the safety confirmation system 1 includes drive recorders 10-1, 10-2, ... 10-m (m is a natural number equal to or greater than 1), a server device 100, operator terminals 200-1, 200-2, ... 200-n (n is a natural number equal to or greater than 1), and a request destination system 300. The server device 100 and each operator terminal 200 constitute a call center.

[0032] Each drive recorder 10, the server device 100, and the request destination system 300 are connected to each other so as to be able to communicate with each other via a network N1, which may be the Internet, a mobile phone network, a C-V2X (Cellular Vehicle to Everything) communication network, or the like.

[0033] The server device 100 and each operator terminal 200 are connected to each other so that they can communicate with each other via a network N2, which is a private network managed by a company that operates a call center. Also, each operator terminal 200 can communicate with each drive recorder 10 and the request destination system 300 connected to the network N1 via the server device 100.

[0034] The drive recorder 10 records vehicle data including exterior and interior images captured by an exterior camera 13a and an interior camera 13b mounted on the device, as well as various data indicating the vehicle's status, in a ring buffer memory in a manner that allows the data to be overwritten for a certain period of time.

[0035] When the in-vehicle image is set to off (that is, when the in-vehicle camera 13b is in an off state), the drive recorder 10 does not capture or record the in-vehicle image.

[0036] Furthermore, when the drive recorder 10 detects a specific event such as an accident or a near miss while the vehicle is running, the drive recorder 10 transmits a notification of the detection of the event to the server device 100. In this embodiment, when the drive recorder 10 detects a strong acceleration corresponding to a strong impact, the drive recorder 10 transmits a notification indicating the detection to the server device 100.

[0037] Furthermore, when the drive recorder 10 detects the specific event, the drive recorder 10 sets the vehicle data for a certain period of time before and after the detection time to be prohibited from being overwritten, or records the vehicle data for a certain period of time before and after the detection time on another recording medium.

[0038] Furthermore, when the drive recorder 10 detects a specific event, it transmits the vehicle data to be set to the above-mentioned overwrite prohibition to the server device 100. At this time, the drive recorder 10 may notify the server device 100 that it has detected the specific event, and transmit the vehicle data to the server device 100 in response to a request from the server device 100 that has received this notification.

[0039] The server device 100 is realized as, for example, a private cloud server. The server device 100 is managed by, for example, a business operator that operates a call center. The server device 100 collects vehicle data transmitted from each drive recorder 10.

[0040] Furthermore, the server device 100 executes a step-by-step safety confirmation process when the drive recorder 10 detects a strong acceleration. At this time, the server device 100 executes the step-by-step safety confirmation process according to the on / off state of the in-vehicle camera 13b, which has been described with reference to FIG.

[0041] The operator terminal 200 is a terminal device used by an operator of a call center, and is realized by a PC (Personal Computer), a smartphone, etc. In addition to these, the operator terminal 200 may be realized by, for example, a tablet terminal, a wearable device, etc.

[0042] The operator checks the safety of the occupants based on the outside and inside images of the vehicle on the monitor screen displayed on the operator terminal 200 and on conversations with the occupants through telephone calls or interactive operations. Based on the results of the confirmation, the operator also arranges for the necessary vehicles and personnel, such as an ambulance, police car, tow truck, and fire engine, to the relevant request destination. Specifically, the operator accesses the request destination system 300 via the operator terminal 200 and requests the request destination system 300 to dispatch the necessary vehicles and personnel, for example.

[0043] When the request destination system 300 receives a request for each required means from the call center, it executes the procedure to meet this request.

[0044] Next, a configuration example of the drive recorder 10 will be described. Fig. 3 is a diagram showing a configuration example of the drive recorder 10 according to the embodiment. As shown in Fig. 3, the drive recorder 10 has a communication unit 11, an HMI (Human Machine Interface) unit 12, a sensor unit 13, a storage unit 14, and a controller 15.

[0045] The communication unit 11 is realized by a network adapter etc. The communication unit 11 is wirelessly connected to the network N1, and transmits and receives information to and from the server device 100 via the network N1.

[0046] The HMI unit 12 is a component that provides interface components related to input and output to a passenger or the like who uses the drive recorder 10. The HMI unit 12 includes an input interface that accepts input operations from a passenger or the like. The HMI unit 12 also includes an output interface that presents visual information and audio information to a driver or the like.

[0047] 3, for example, the HMI unit 12 includes a touch panel display 12a, a microphone 12b, and a speaker 12c. The touch panel display 12a corresponds to the input interface and the output interface described above.

[0048] The touch panel display 12a displays, for example, operation components for the occupant to set the in-vehicle video to on or off. The touch panel display 12a also displays, for example, an interactive screen for the occupant to perform an interactive operation and the operation components thereof. Note that each operation component may be provided in the HMI unit 12 as a hardware component, rather than being displayed on the touch panel display 12a as a software component.

[0049] The microphone 12b corresponds to an input interface for a voice call between a call center operator and a passenger, and the speaker 12c corresponds to an output interface for the voice call.

[0050] The sensor unit 13 is a group of various sensors mounted on the drive recorder 10. The sensor unit 13 includes, for example, an exterior camera 13a, an interior camera 13b, a GPS (Global Positioning System) sensor 13c, and a G sensor 13d.

[0051] The exterior camera 13a is provided so as to be able to capture images outside the vehicle. The exterior camera 13a is attached near the windshield, near the dashboard, etc., and captures images in front of the vehicle. The exterior camera 13a may also be attached near the rear window, etc., and be able to capture images behind the vehicle.

[0052] The in-vehicle camera 13b is provided so as to be able to capture an image of the interior of the vehicle. The in-vehicle camera 13b is attached near the windshield, the dashboard, or the like so that at least all passengers are included in the imaging range.

[0053] The GPS sensor 13c measures the GPS position of the vehicle. The G sensor 13d measures the acceleration applied to the drive recorder .

[0054] In addition to the sensor unit 13, the drive recorder 10 is connected to an in-vehicle sensor 5, which is a group of various sensors mounted on a vehicle. The in-vehicle sensor 5 includes, for example, a vehicle speed sensor, an accelerator sensor, a brake sensor, etc. The in-vehicle sensor 5 is connected to the drive recorder 10 via an in-vehicle network such as a CAN (Controller Area Network).

[0055] The storage unit 14 is realized by a storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, etc. In the example of Fig. 3, the storage unit 14 stores user information 14a and vehicle record information 14b.

[0056] The user information 14a is information that links information about a user who uses the drive recorder 10 with identification information of the drive recorder 10. The information about the user is the identification information of the user, the identification information of the vehicle in which the drive recorder 10 is installed (for example, the vehicle registration number), etc.

[0057] The vehicle record information 14b is a database of vehicle data recorded by the drive recorder 10.

[0058] The controller 15 corresponds to a so-called processor. The controller 15 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphical Processing Unit), or the like. The controller 15 executes a program according to an embodiment (not shown) stored in the storage unit 14, using the RAM as a working area. The controller 15 can also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0059] The controller 15 executes information processing according to the processing sequences shown in Figures 6, 7 and 11. The explanation using Figures 6, 7 and 11 will be given later.

[0060] Next, a configuration example of the server device 100 will be described. Fig. 4 is a diagram showing a configuration example of the server device 100 according to an embodiment. As shown in Fig. 4, the server device 100 includes a communication unit 101, a storage unit 102, and a controller 103.

[0061] The communication unit 101 is realized by a network adapter, etc. The communication unit 101 is connected to a network N1 by wire or wirelessly, and transmits and receives information to and from the drive recorder 10 and the request destination system 300 via the network N1. The communication unit 101 is also connected to a network N2 by wire or wirelessly, and transmits and receives information to and from the operator terminal 200 via the network N2.

[0062] The storage unit 102 is realized by a storage device such as a ROM, a RAM, a flash memory, a HDD (Hard Disk Drive), etc. In the example of Fig. 4, the storage unit 102 stores a user information DB (Database) 102a, a confirmation information record DB 102b, and a screen generation information DB 102c.

[0063] The user information DB 102a is a database of the user information 14a collected from each drive recorder 10. In addition, the user information DB 102a may store the vehicle data collected from each drive recorder 10 for each piece of user information 14a.

[0064] The confirmation information recording DB 102b is a database in which confirmation information including each video used in the safety confirmation process, the dialogue content in the safety confirmation process, and the confirmation result is recorded. The dialogue content includes a history of dialogue by voice and a history of dialogue by dialogue operation. By recording such a history, it is possible to objectively evaluate whether a dialogue with high response quality is being conducted from the viewpoint of privacy protection.

[0065] The screen generation information DB 102c is a database that stores various parameters, GUIs (Graphical User Interfaces), and the like used when the controller 103 generates each screen such as the above-mentioned interactive screen and monitor screen. The various parameters include, for example, parameters related to the screen layout of each screen.

[0066] The controller 103 corresponds to a so-called processor. The controller 103 is realized by a CPU, an MPU, a GPU, or the like. The controller 103 executes a program according to an embodiment (not shown) stored in the storage unit 102, using a RAM as a working area. The controller 103 can also be realized by an integrated circuit such as an ASIC or an FPGA.

[0067] The controller 103, like the above-mentioned controller 15, executes information processing according to the processing sequences shown in Figures 6, 7 and 11. The explanation using Figures 6, 7 and 11 will be given later.

[0068] Next, a configuration example of the operator terminal 200 will be described. Fig. 5 is a diagram showing a configuration example of the operator terminal 200 according to the embodiment. As shown in Fig. 5, the operator terminal 200 has a communication unit 201, an HMI unit 202, a storage unit 203, and a controller 204.

[0069] The communication unit 201 is realized by a network adapter etc. The communication unit 201 is connected to the network N2 by wire or wirelessly, and transmits and receives information to and from the server device 100 via the network N2.

[0070] The HMI unit 202 is a component that provides interface components related to input and output to the operator who uses the operator terminal 200. The HMI unit 202 includes an input interface that accepts input operations from the operator. The HMI unit 202 also includes an output interface that presents visual information and audio information to the operator.

[0071] 5, for example, the HMI unit 202 includes a display 202a, a microphone 202b, a speaker 202c, and a keyboard 202d. The display 202a corresponds to the output interface described above. The display 202a displays a monitor screen including at least one or both of an outside image and an inside image of the vehicle after receiving a notification that a strong acceleration has been detected, for example.

[0072] The microphone 202b corresponds to an input interface for voice communication between an operator and a passenger, and the speaker 202c corresponds to an output interface for voice communication.

[0073] The keyboard 202d corresponds to an input interface that accepts an operator's operation on a monitor screen, for example. The input interface may also be realized by a mouse, a pen tablet, or the like. The input interface may also be realized by a software component.

[0074] Furthermore, the HMI unit 202 may provide the operator with an input interface and an output interface in one body, for example, by using a touch panel display.

[0075] The storage unit 203 is realized by a storage device such as a ROM, a RAM, a flash memory, a HDD, etc. In the example of Fig. 5, the storage unit 203 stores operator application information 203a.

[0076] The operator application information 203a is information related to an operator application executed in the operator terminal 200, and includes a program of the operator application. The operator application information 203a may be realized as a Web application with the server device 100 serving as a Web server.

[0077] The controller 204 corresponds to a so-called processor. The controller 204 is realized by a CPU, an MPU, a GPU, or the like. The controller 204 executes the above-mentioned operator application program stored in the storage unit 203, using the RAM as a working area. The controller 204 can also be realized by an integrated circuit such as an ASIC or an FPGA.

[0078] The controller 204, like the above-mentioned controller 15 and controller 103, executes information processing according to the processing sequences shown in FIGS.

[0079] Next, the information processing according to this processing sequence will be described in order. Fig. 6 is a diagram (part 1) showing a processing sequence executed by the safety confirmation system 1 according to the embodiment. Fig. 7 is a diagram (part 2) showing a processing sequence executed by the safety confirmation system 1 according to the embodiment. Fig. 6 corresponds to the processing procedure of the first stage of the safety confirmation processing described above. Fig. 7 corresponds to the processing procedure of the second stage of the safety confirmation processing described above.

[0080] 6, the controller 15 of the drive recorder 10 determines whether or not an impact (i.e., strong acceleration) is detected (step S101). If an impact is detected (step S101, Yes), the controller 15 transmits a notification that an impact has been detected to the server device 100 (step S102). If an impact is not detected (step S101, No), the controller 15 repeats step S101.

[0081] When the controller 103 of the server device 100 receives the notification in step S102, it forcibly connects a call between the drive recorder 10 and the operator terminal 200 ("forced call connection") (step S103).

[0082] In addition, the controller 15 of the drive recorder 10 transmits the camera images captured by the exterior camera 13a and the interior camera 13b to the server device 100 (step S104). The controller 103 of the server device 100 generates a monitor screen that displays the camera images captured in step S104 in real time (step S105) and transmits the generated monitor screen to the operator terminal 200 (step S106).

[0083] The controller 204 of the operator terminal 200 displays the transmitted monitor screen on the display 202a of the HMI unit 202 (step S107).

[0084] Furthermore, the controller 103 of the server device 100 determines whether or not the camera video transmitted from the drive recorder 10 includes an in-vehicle video, that is, whether or not the in-vehicle video is set to ON (step S108).

[0085] If there is in-vehicle video (step S108, Yes), the controller 103 causes a safety confirmation to be performed between the operator terminal 200 and the drive recorder 10 using the in-vehicle video and the video from outside the vehicle and a telephone call (step S109). If there is in-vehicle video due to the original "in-vehicle video on setting", it is considered that the priority for protecting the privacy of the occupants is low, so a safety confirmation can be performed using at least the in-vehicle video and a telephone call according to that priority.

[0086] The operator inputs the safety confirmation result at any time, and the controller 204 of the operator terminal 200 acquires the input of the confirmation result (step S110). Then, the controller 204 transmits the acquired confirmation result to the server device 100 (step S111).

[0087] The controller 103 of the server device 100 records confirmation information including the video, the dialogue content, and the confirmation result of step S111 as needed (step S112). In addition, the controller 204 of the operator terminal 200 acquires an arrangement operation according to the confirmation result by the operator (step S113), and transmits an arrangement request corresponding to this arrangement operation to the server device 100 (step S114).

[0088] Then, when the controller 103 of the server device 100 receives the dispatch request in step S114, it executes dispatch processing to the request destination in response to this dispatch request (step S115). In this dispatch processing, the controller 103 transfers the dispatch request to the request destination system 300 corresponding to the request (step S116). This makes it possible to automatically and appropriately request the dispatch of each vehicle and personnel required in response to the safety confirmation result.

[0089] If there is no in-vehicle video (step S108, No), that is, if the in-vehicle video is set to off, the controller 103 executes the second-stage safety confirmation process shown in Fig. 7. As shown in Fig. 7, in the second-stage safety confirmation process, the controller 103 of the server device 100 causes the operator terminal 200 to make a call to the drive recorder 10 (step S201).

[0090] If the occupant is able to respond by telephone call, the driver recorder 10 responds by telephone call to the operator terminal 200 (step S202).

[0091] In parallel with steps S201 and S202, the controller 103 of the server device 100 generates an interactive screen (step S203) and transmits it to the drive recorder 10 (step S204). The controller 15 of the drive recorder 10 displays this interactive screen on the touch panel display 12a of the HMI unit 12 (step S205). This makes it possible for the occupant to confirm their safety by interacting with the operator through this interactive operation, even if they cannot interact through voice, if they can perform an interactive operation on the interactive screen.

[0092] If the occupant is able to respond interactively to the displayed dialogue screen, the driver recorder 10 responds interactively to the operator terminal 200 (step S206).

[0093] Here, specific examples of the dialogue screen will be described with reference to Fig. 8 to Fig. 10. Fig. 8 is a diagram (part 1) showing a specific example of the dialogue screen. Fig. 9 is a diagram (part 2) showing a specific example of the dialogue screen. Fig. 10 is a diagram (part 3) showing a specific example of the dialogue screen.

[0094] As shown in FIG. 8, immediately after detecting an impact, the controller 103 causes the touch panel display 12a of the drive recorder 10 to display an interactive screen indicating that a response is possible by voice or touch button.

[0095] The occupant looks at the dialogue screen as shown in the example of Figure 8 and responds by voice if it is possible, or responds by dialogue by touching the touch button "OK" if it is not possible to respond by voice.

[0096] Furthermore, as shown in FIG. 9, the controller 103 causes the touch panel display 12a of the drive recorder 10 to display, for example, a dialogue screen asking whether or not the driver can move his / her body.

[0097] The occupant looks at the dialogue screen as shown in the example of Figure 9 and responds by voice if a voice response is possible, or the occupant responds by dialogue operation by touching the touch button "Yes" or "No" if a voice response is not possible.

[0098] Furthermore, as shown in FIG. 10, the controller 103 causes the touch panel display 12a of the drive recorder 10 to display, for example, a dialogue screen asking whether or not the vehicle can be moved.

[0099] The occupant looks at the dialogue screen as shown in the example of Figure 10 and responds by voice if a voice response is possible, or the occupant responds by dialogue operation by touching the touch button "Yes" or "No" if a voice response is not possible.

[0100] Returning to the description of Fig. 7, the controller 103 of the server device 100 then determines whether or not a dialogue is possible based on the response by telephone call in step S202 or the response by dialogue operation in step S206 (step S207).

[0101] In step S207, the controller 103 determines that dialogue is not possible if there is no response by dialogue operation and, further, if there is no speech from the occupant for a certain period of time or more in response to a call, for example. This makes it possible to estimate a situation in which the occupant is in such a serious condition that he or she cannot perform dialogue operation or speak.

[0102] Furthermore, when there is no response by interactive operation and, regarding a response by phone call, for example, the occupant's speech cannot be interpreted by voice recognition, the controller 103 determines that dialogue is not possible. This makes it possible to estimate a situation in which the occupant has spoken but is emitting a moan or the like that cannot be interpreted by voice recognition, and the occupant is in a critical condition without being able to perform an interactive operation.

[0103] Furthermore, when there is no voice response and no response through interactive operations for a certain period of time, the controller 103 determines that interaction is not possible. This makes it possible to estimate a situation in which the occupant is in a serious condition in which he or she is unable to respond through either voice or interactive operations.

[0104] Then, when the controller 103 determines in step S207 that dialogue is possible (step S207, Yes), it causes a safety confirmation to be performed between the operator terminal 200 and the drive recorder 10 using a voice call or dialogue operation and an outside-vehicle image, although there is no inside-vehicle image (step S208). This allows the privacy of the occupants to be protected by the absence of an inside-vehicle image, while the safety confirmation can be performed using an outside-vehicle image and a voice call or dialogue operation.

[0105] The operator inputs the confirmation result of the safety confirmation at any time, and the controller 204 of the operator terminal 200 acquires the input operation of the confirmation result (step S209). Then, the controller 204 transmits the acquired confirmation result to the server device 100 (step S210).

[0106] The controller 103 of the server device 100 records confirmation information including the video, the dialogue content, and the confirmation result of step S210 as needed (step S211). The controller 103 constantly determines whether the response is interrupted during the confirmation in steps S208 to S211 (step S212). The determination condition in step S212 may be the same as the determination condition described for step S207.

[0107] Then, if there is no interruption of the response during the confirmation (step S212, No), the controller 103 executes steps S113 to S116 shown in Fig. 6. On the other hand, if the controller 103 determines that dialogue is not possible in step S207 (step S207, No), it executes the third stage of safety confirmation processing described above. Even if the response is interrupted during the confirmation in step S212 (step S212, Yes), the controller 103 executes the third stage of safety confirmation processing.

[0108] Fig. 11 is a diagram (part 3) showing a processing sequence executed by the safety confirmation system 1 according to the embodiment. Fig. 11 corresponds to the processing procedure of the third stage of safety confirmation processing. Fig. 12 is a diagram showing a specific example of a notification screen.

[0109] 11, in the third stage of the safety confirmation process, the controller 103 of the server device 100 determines that the danger level of the occupants exceeds a threshold (step S301). That is, the controller 103 determines that the situation is one in which the lives of the occupants should be prioritized over the protection of the occupants' privacy, and generates a notification screen that forcibly turns on the in-vehicle video (step S302).

[0110] Then, the controller 103 transmits the generated notification screen to the drive recorder 10 (step S303). The controller 15 of the drive recorder 10 displays this notification screen on the touch panel display 12a of the HMI unit 12 (step S304).

[0111] 12, the notification screen displays a message to the effect that the in-vehicle camera 13b will be automatically turned on since there is no reaction from the occupant. Although there is a possibility that the occupant cannot actually see this notification screen, by storing the history including this notification screen in the confirmation information record DB 102b, it is possible to leave a record that the in-vehicle camera 13b will be turned on while considering the privacy protection of the occupant.

[0112] Returning to the description of Fig. 11, the controller 103 of the server device 100 then transmits to the drive recorder 10 a "forced in-vehicle image on signal" which is a control signal for forcibly turning on the in-vehicle camera 13b (step S305).

[0113] The controller 15 of the drive recorder 10 receives the "forced in-vehicle image on signal" and switches the drive recorder 10 to the forced in-vehicle image on mode (step S306). That is, the controller 15 forcibly switches the in-vehicle camera 13b, which was in the off state, to the on state.

[0114] Then, the controller 15 transmits the camera images including the in-vehicle video to the server device 100 (step S307). The controller 103 of the server device 100 generates a monitor screen that displays the camera video of step S307 in real time (step S308) and transmits it to the operator terminal 200 (step S309).

[0115] The controller 204 of the operator terminal 200 displays the transmitted monitor screen on the display 202a of the HMI unit 202 (step S310). Then, the controller 103 of the server device 100 causes a safety confirmation to be performed between the operator terminal 200 and the drive recorder 10 using the outside-vehicle video and the inside-vehicle video (step S311).

[0116] In this way, in the third stage safety confirmation process, only when the danger level of the occupants exceeds the threshold, the controller 103 forcibly turns on the in-vehicle camera 13b if it is off. In other words, the controller 103 forcibly transmits the in-vehicle video to the drive recorder 10. Then, the controller 103 causes the safety confirmation to be performed using the outside video and the inside video. This makes it possible to realize the minimum necessary confirmation of the safety of the occupants while protecting the privacy of the occupants.

[0117] Thereafter, the controller 103 executes steps S113 to S116 shown in FIG.

[0118] As described above, the server device 100 (corresponding to an example of a "center device") according to the embodiment includes the controller 103. The controller 103 receives vehicle information related to the behavior of the vehicle from the drive recorder 10 (corresponding to an example of an "in-vehicle device"). When the controller 103 detects an abnormality from the vehicle information, the controller 103 requests an in-vehicle image (corresponding to an example of an "in-vehicle camera image"). When the controller 103 does not respond to the request from the drive recorder 10, the controller 103 transmits a control signal for forcibly transmitting the in-vehicle image, and acquires the in-vehicle image from the drive recorder 10. Specifically, when the controller 103 receives a notification that a predetermined strong acceleration has been detected from the drive recorder 10 having the in-vehicle camera 13b that captures the in-vehicle image of the vehicle, the controller 103 detects an abnormality from the notification and executes a step-by-step safety confirmation process for confirming the safety of the vehicle occupants. Furthermore, when the stage of the safety confirmation process, which is associated in advance with the danger level of the occupant, exceeds a threshold, if the in-vehicle camera 13b is in an off state, the controller 103 transmits a control signal to the drive recorder 10 to forcibly turn on the in-vehicle camera 13b. Then, the controller 103 acquires an in-vehicle video from the drive recorder 10.

[0119] Therefore, when the controller 103 receives, for example, acceleration corresponding to a strong impact as vehicle information from the drive recorder 10, it transmits a control signal to the drive recorder 10 to forcibly transmit the in-vehicle image if the in-vehicle image cannot be acquired from the drive recorder 10. The drive recorder 10 forcibly transmits the in-vehicle image to the server device 100 in response to this control signal, and the controller 103 acquires the in-vehicle image forcibly transmitted from the drive recorder 10. As a result, even if the occupant has set the in-vehicle camera to an off state for privacy protection, the controller 103 can acquire the in-vehicle image in, for example, a situation in which the occupant is estimated to be in a serious condition. That is, according to the server device 100 according to the embodiment, even if the in-vehicle image is set to an off state, it is possible to realize the confirmation of the safety of the occupant using the in-vehicle image.

[0120] Further advantages and modifications may readily occur to those skilled in the art. Thus, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof. [Explanation of symbols]

[0121] 1. Safety confirmation system 5. Vehicle sensors 10. Drive recorder 11 Communications Department 12 HMI section 13 Sensor section 14 Storage section 15 Controller 100 Server device 101 Communications Department 102 Storage section 103 Controller 200 Operator terminal 201 Communications Department 202 HMI Department 203 Storage section 204 Controller 300 Requested System

Claims

1. receiving vehicle information relating to a behavior of the vehicle from an in-vehicle device; When an abnormality is detected from the vehicle information, a camera image of the vehicle interior is requested from the vehicle-mounted device; If there is no response to the request from the in-vehicle device, Transmitting a control signal to forcibly transmit the camera image in the vehicle interior; a controller for acquiring camera images of the interior of the vehicle from the in-vehicle device; A center device comprising:

2. The controller: when a notification that a predetermined strong acceleration has been detected is received as the vehicle information, a call is connected to the in-vehicle device. The center device according to claim 1 .

3. The controller: executing a safety confirmation process for confirming the safety of an occupant of the vehicle in a stepwise manner according to an on / off state of an in-vehicle camera that captures a camera image inside the vehicle cabin; The center device according to claim 2 .

4. The controller: The safety confirmation process is executed with a higher priority given to privacy protection when the in-vehicle camera is in an off state than when the in-vehicle camera is in an on state. The center device according to claim 3.

5. The controller: When the in-vehicle camera is in an off state, the safety confirmation process is executed with a lower priority of privacy protection when there is no response from the occupant to the safety confirmation process than when there is a response from the occupant. The center device according to claim 3.

6. The controller: When the in-vehicle camera is turned on, a camera image of the inside of the vehicle is received from the in-vehicle device; Executing the safety confirmation process using at least the voice through the call connection and the camera image inside the vehicle cabin. The center device according to claim 3.

7. The controller: transmitting, to the in-vehicle device, an interactive screen that can be interactively operated by the occupant, when the in-vehicle camera is in an off state; The center device according to claim 3.

8. The controller: determining whether or not a dialogue with the occupant can be performed by voice via the call connection or by the interactive operation, and when it is determined that a dialogue is possible, executing the safety confirmation process by voice via the call connection or by the interactive operation; The center device according to claim 7.

9. The controller: determining that dialogue is not possible when there is no speech from the occupant via the call connection for a predetermined period of time or longer; The center device according to claim 8.

10. The controller: determining that a dialogue is not possible when the utterance of the occupant via the call connection cannot be interpreted by voice recognition; The center device according to claim 8.

11. The controller: determining that interaction is not possible when the occupant does not perform an interactive operation on the interactive screen for a predetermined period of time or longer; The center device according to claim 8.

12. The controller: When it is determined that the conversation is not possible, a control signal for forcibly turning on the in-vehicle camera is transmitted to the in-vehicle device; receiving a camera image of the interior of the vehicle from the in-vehicle device; Executing the safety confirmation process using a camera image inside the vehicle cabin. The center device according to claim 8.

13. The controller: Transmitting a request for arrangements according to the result of the safety confirmation process to a requested system; The center device according to any one of claims 3 to 12.

14. An in-vehicle device that communicates with a center device, A setting is provided for determining whether or not to transmit the camera image in the vehicle cabin to the center device, When a user selects a setting not to transmit the camera image in the vehicle cabin to the center device, If there is no response to the request from the center device, receiving a control signal transmitted from the center device; a controller that changes the setting to a setting to be transmitted in accordance with the control signal; An in-vehicle device comprising:

15. A center device and an in-vehicle device that communicates with the center device, The center device includes: receiving vehicle information relating to a behavior of the vehicle from the in-vehicle device; When an abnormality is detected from the vehicle information, a camera image of the vehicle interior is requested from the vehicle-mounted device; If there is no response to the request from the in-vehicle device, Transmitting a control signal to forcibly transmit the camera image in the vehicle interior; Acquire a camera image of the interior of the vehicle from the in-vehicle device; The in-vehicle device includes: A setting is provided for determining whether or not to transmit the camera image in the vehicle cabin to the center device, When a user selects a setting not to transmit the camera image in the vehicle cabin to the center device, If there is no response to the request from the center device, receiving the control signal transmitted from the center device; changing the setting to a setting to be transmitted in accordance with the control signal; Safety confirmation system.

Images