Information processing device and information processing method

Abstract

To provide an information processing device capable of reducing unnecessary notification to a user terminal during monitoring a vehicle.SOLUTION: An information processing device mounted on a vehicle has a control part for executing: recording first video data picked up by an on-vehicle camera provided by the vehicle and being video data when a first event occurs in a prescribed storage device, and the transmission of first information of notifying a user of occurrence of the first event to a user terminal in the case that the first event occurs in the vehicle; and recording second video data picked up by the on-vehicle camera and being video data when a second event occurs in the storage device without transmitting information of notifying the user of occurrence of the second event to a user terminal in the case that the second event having a different kind from the first event occurs in the vehicle in a state in which a travel system of the vehicle is not activated.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] This disclosure relates to vehicle monitoring.

Background Art

[0002] When a vehicle is parked, there is a known technique in which a device for monitoring the vehicle is operated by a predetermined device held by a user, and the vehicle state is notified to the device. In this regard, for example, Patent Document 1 discloses a unit that allows a user to select one of a plurality of vehicle monitoring functions on a user terminal held by the user and notifies the user terminal of the monitoring result when the vehicle is not in a traveling state.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of this disclosure is to reduce unnecessary notifications to a user terminal during vehicle monitoring.

Means for Solving the Problems

[0005] An information processing apparatus according to one aspect of this disclosure is Information processing device mounted on a vehicle, comprising a control unit that, when a first event occurs in the vehicle while the vehicle's driving system is not running, records first video data, which is video data of the time the first event occurred and is captured by an on-board camera installed in the vehicle, into a predetermined storage device and transmits first information notifying the occurrence of the first event to a user terminal; and when a second event of a different type from the first event occurs in the vehicle, records second video data, which is video data of the time the second event occurred and is captured by the on-board camera, into the storage device without transmitting information notifying the occurrence of the second event to the user terminal.

[0006] A user terminal according to one aspect of the present disclosure is a user terminal having a control unit that performs the following actions: receiving information that notifies the vehicle that a first event or a second event of a different type than the first event has occurred, while the vehicle's driving system is not running; and notifying the user of the occurrence of the first event via a predetermined interface if the information indicates that the first event has occurred in the vehicle.

[0007] A program according to one aspect of this disclosure is a program that causes a computer to perform the following actions: receive information notifying the vehicle that a first event or a second event of a different type than the first event has occurred, while the vehicle's driving system is not running; and, if the information indicates that the first event has occurred in the vehicle, notify the user of the occurrence of the first event via a predetermined interface.

[0008] An information processing method according to one aspect of the present disclosure is an information processing method performed by an information processing device mounted on a vehicle, wherein when the vehicle's driving system is not activated, a first event occurs in the vehicle, and the vehicle's onboard camera captures the following: The system includes recording first video data, which is video data of the moment the first event occurs, in a predetermined storage device, and transmitting first information notifying the user terminal of the occurrence of the first event; and, if a second event of a different type from the first event occurs in the vehicle, recording second video data, which is video data of the moment the second event occurs, captured by the in-vehicle camera, in the storage device without transmitting information notifying the user terminal of the occurrence of the second event.

[0009] Other embodiments include a method performed by the above-mentioned device, a program for causing a computer to perform the method, or a computer-readable storage medium that non-temporarily stores the program. [Effects of the Invention]

[0010] According to this disclosure, it is possible to reduce unnecessary notifications to the user terminal when monitoring a vehicle. [Brief explanation of the drawing]

[0011] [Figure 1] A conceptual diagram of an information processing device according to the first embodiment. [Figure 2] A diagram illustrating the components included in the system according to the first embodiment. [Figure 3] A flowchart of the processes performed by an information processing device. [Figure 4] A flowchart of the processes performed by the user terminal. [Figure 5] An example of a screen displayed on a user's terminal. [Figure 6] A flowchart of the processes performed by an information processing device. [Figure 7] A flowchart of the processes performed by an information processing device. [Figure 8] A diagram illustrating the components of an information processing device according to the fourth embodiment. [Figure 9] A flowchart of the process performed by the information processing device in the fourth embodiment. [Figure 10]Flowchart of the process executed by the information processing apparatus in the fourth embodiment.

Mode for Carrying Out the Invention

[0012] There is known an information processing apparatus mounted on a vehicle that monitors the vehicle while parked and notifies a user terminal (hereinafter referred to as the user terminal) possessed by the user of the monitoring result.

[0013] However, in the conventional information processing apparatus, when notifying the user of the detected result, the notification method in the case of detecting an event with low urgency and high occurrence frequency has not been appropriately considered.

[0014] When an event with low urgency is detected during vehicle monitoring, it is preferable not to frequently notify the user of its occurrence.

[0015] For example, for theft or intrusion prevention, a person approaching the vehicle may be detected. However, if a notification is sent triggered by detecting a person, the notification will be sent every time a person passes by the vehicle or gets on or off an adjacent vehicle, resulting in a decrease in usability. However, it is not preferable to avoid detecting these events with low urgency. Even when an event with low urgency occurs, the user has a need to check the video later to see what events have occurred.

[0016] In order to solve this problem, it is required to selectively execute the issuance of a notification to the user and the recording of video according to the type of the event that has occurred.

[0017] The information processing apparatus of the present disclosure solves such a problem.

[0018] An information processing device according to one aspect of the present disclosure is an information processing device mounted on a vehicle, which includes a control unit that, when a first event occurs in the vehicle while the vehicle's driving system is not running, records first video data, which is video data of the time the first event occurred and is captured by an on-board camera provided in the vehicle, into a predetermined storage device and transmits first information notifying the occurrence of the first event to a user terminal, and when a second event of a different type from the first event occurs in the vehicle, records second video data, which is video data of the time the second event occurred and is captured by the on-board camera, into the storage device without transmitting information notifying the occurrence of the second event to the user terminal.

[0019] The first event is an event that could harm the vehicle, such as when an impact to the vehicle is detected, or when an intrusion into the vehicle is detected. Specifically, when an onboard camera or various sensors such as shock sensors or tilt sensors installed in the vehicle detect an impact or when a vehicle door is forced open, the control unit can determine that the first event has occurred. However, the first event is not limited to events that occur when an impact to the vehicle is detected, or when an intrusion into the vehicle is detected. The first event can be any event that occurs when an event that could harm the vehicle is detected.

[0020] The second event is an event that does not directly involve harm to the vehicle, such as an event that occurs when the vehicle detects the approach of a person. Specifically, when various sensors such as an on-board camera or motion sensor installed in the vehicle detect that a person is within a predetermined distance from the vehicle, the control unit can determine that a second event has occurred. The second event is a different type of event from the first event. Note that the second event is not limited to events that occur when an event such as the approach of a person is detected in the vehicle. The second event can be any event that occurs when an event that does not directly involve harm to the vehicle is detected. The types of the first and second events may be predetermined.

[0021] The first video data is video data captured by an on-board camera installed in the vehicle when the first event occurs. The first video data may be video data or one or more still images. The first video data may also be video data transmitted via streaming. The first video data may be captured by a camera other than the on-board camera, as long as it shows the interior or surroundings of the vehicle.

[0022] The second video data refers to video data captured by the vehicle's onboard camera when the second event occurs. The second video data may be video data or one or more still images. The second video data may also be video data transmitted via streaming. The second video data may be captured by cameras other than the onboard camera, as long as it shows the interior or surroundings of the vehicle.

[0023] The first information is information that notifies the occurrence of the first event, and is transmitted from the information processing device to the user terminal. For example, the first information may be a flag that is set when the first event occurs. The first information may also be information that causes the user terminal to take action to notify the occurrence of the first event. The information processing device may also transmit the first information to a predetermined external storage device.

[0024] The control unit, when the first event or the second event occurs, processes the first video data. Alternatively, the second video data is recorded in a predetermined storage device. In other words, the control unit writes either the first video data or the second video data to the memory of the predetermined storage device. To put it another way, the control unit causes the first video data or the second video data to be stored in the predetermined storage device. Here, the predetermined storage device may be a storage unit provided by an information processing device, or it may be an external server wirelessly connected to the vehicle.

[0025] Furthermore, the control unit transmits the first information to the user terminal when the first event occurs. In other words, the control unit notifies the user terminal of the occurrence of the first event. Alternatively, the control unit may instruct the user terminal to notify the user of the occurrence of the first event via a predetermined interface.

[0026] Furthermore, the control unit switches whether or not to send information to the user terminal (i.e., information to notify the user) depending on whether the first event occurs or the second event occurs. Specifically, the control unit sends information to the user terminal when the first event occurs, and does not send information to the user terminal when the second event occurs.

[0027] Furthermore, if the vehicle's driving system is not running and the first event occurs within a predetermined time period after the second event, the control unit may copy the second video data recorded in the first storage device at the time of the second event to the external server device.

[0028] The control unit may also determine the time of occurrence of the first event and the second event, and measure the elapsed time between the time of occurrence of the first event and the second event.

[0029] Furthermore, the control unit may switch whether or not to send the second information to the user terminal depending on the duration of the second event. For example, the control unit may be configured to send the second information to the user terminal when the duration of the second event is greater than or equal to a predetermined threshold, and not send the second information when the duration of the second event is less than the predetermined time.

[0030] Furthermore, the control unit may measure the length of the period during which the second event occurs repeatedly within a predetermined interval, as the length of the period during which the second event occurs continuously.

[0031] The second event is considered less urgent than the first event, but if it lasts for a long time, it may be advisable to notify the user.

[0032] Furthermore, the control unit may record the second video data on the server device if the first event occurs within a predetermined time after the second event has occurred.

[0033] This configuration reduces unnecessary notifications sent to user terminals when monitoring vehicles.

[0034] The following describes specific embodiments of this disclosure with reference to the drawings. Unless otherwise specified, the hardware configurations, module configurations, functional configurations, etc., described in each embodiment are not intended to limit the technical scope of the disclosure to those configurations alone.

[0035] (First embodiment) An overview of the information processing device according to the first embodiment will be described with reference to Figure 1. Figure 1 is a conceptual diagram of a system including the information processing device 100 according to the first embodiment.

[0036] Vehicle 10 is equipped with an information processing device 100. The information processing device 100 acquires data transmitted from an on-board camera 400 or sensor 500 mounted on vehicle 10 and determines whether or not a person has approached vehicle 10, an impact has occurred on vehicle 10, or an intrusion has occurred on vehicle 10.

[0037] The user terminal 300 is an information terminal owned by a user who uses the services provided by the information processing device 100. The user terminal 300 may be a device such as a smartphone or tablet. In this embodiment, the determination result from the information processing device 100 is notified to the user terminal 300.

[0038] Server 200 is a server device that stores the video footage captured by the in-vehicle camera 400.

[0039] There is a system that notifies the user's terminal when a device installed in the vehicle detects an impact or other incident to the vehicle.

[0040] However, conventional information processing devices with such a configuration, while intended to detect the approach of suspicious individuals, also detect the approach of people simply getting in or out of adjacent parked vehicles (i.e., people who are not suspicious). In other words, if people frequently approach vehicle 10, the detection results may be frequently notified to the user.

[0041] Frequent notifications of non-urgent events are cumbersome for users. Therefore, the information processing device 100 according to this embodiment selectively sends notifications to the user terminal 300 when the above-mentioned events occur.

[0042] For example, if the information processing device 100 detects an impact to the vehicle 10 or an intrusion into the vehicle 10, it sends a notification to the user terminal 300. Upon receiving the notification, the user terminal 300 displays a screen indicating the occurrence of the detected event and outputs video footage captured by the in-vehicle camera 400, which was transmitted from the information processing device 100. The information processing device 100 also stores the video footage captured by the in-vehicle camera 400 in its storage unit (described later) and simultaneously transmits the video footage to the server 200.

[0043] In contrast, when the information processing device 100 detects that a person is approaching the vehicle 10, it does not send a notification to the user terminal 300, but instead stores information indicating the occurrence of the detected event and images captured by the in-vehicle camera 400 in the storage unit of the information processing device 100.

[0044] In this way, the information processing device 100 decides whether or not to notify the user terminal 300 based on the type of event detected in the vehicle 10. This reduces the inconvenience to the user caused by frequent notifications to the user terminal 300.

[0045] Furthermore, the information processing device 100 determines, based on the type of event detected in the vehicle 10, whether to save the video and other data only to the local storage device or to also send it to the server device. This makes it possible to save relatively unimportant video only locally, while sharing important video with the server device.

[0046] Next, the components of the information processing device 100 will be explained using Figure 2. Figure 2 is a diagram illustrating the components included in the system according to this embodiment.

[0047] The information processing device 100 is an information processing device mounted on the vehicle 10, and is mounted on the vehicle 10 This device detects the occurrence of a first event or a second event based on data transmitted from an in-vehicle camera 400 and a sensor 500. In this embodiment, the first event occurs when an impact to the vehicle 10 is detected, or when an intrusion into the vehicle 10 is detected. The second event occurs when a person approaches the vehicle 10. In this embodiment, when the above events occur, the information processing device 100 selectively notifies the user terminal 300 and saves the video footage at the time of the event to a predetermined device.

[0048] The information processing device 100 comprises a control unit 110, a storage unit 120, and a communication unit 130. The information processing device 100 can communicate with a server 200, a user terminal 300, an in-vehicle camera 400, and a sensor 500 via the communication unit 130. The user terminal 300 comprises a control unit 310, a storage unit 320, a communication unit 330, and a display 340.

[0049] This section provides a detailed explanation of each element that makes up the system.

[0050] First, let's explain the control unit 110.

[0051] The control unit 110 is implemented using a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) and memory. The control unit 110 includes a recording unit 111, an event detection unit 112, an event notification unit 113, and a transmission unit 114 as functional modules. These functional modules may be implemented by executing a program using the control unit 110.

[0052] The recording unit 111 receives from the event detection unit 112 (described later) that a first event or a second event has occurred. When a first event occurs, the recording unit 111 records first video data, which is video data of the first event as captured by the onboard camera 400 of the vehicle 10, into a predetermined storage device. The predetermined storage device is either the storage unit 120 of the information processing device 100 or an external server 200 of the information processing device 100.

[0053] Furthermore, when a second event occurs, the recording unit 111 records the second video data, which is video data of the second event captured by the in-vehicle camera 400, only in the storage unit 120.

[0054] If the in-vehicle camera 400 includes multiple cameras, the recording unit 111 may record the video data captured by each camera in the storage unit 120.

[0055] As explained above, when the first event occurs, the recording unit 111 saves first video data recording the first event, and when the second event occurs, it saves second video data recording the second event. The first video data is stored in both the storage unit 120 and the server 200, while the second video data is stored only in the storage unit 120.

[0056] The event detection unit 112 detects whether a first event or a second event has occurred. Specifically, the event detection unit 112 acquires data from the in-vehicle camera 400 or sensor 500 via the communication unit 130, and determines whether a first event or a second event has occurred based on the acquired data.

[0057] As mentioned above, the first event is when an impact is detected on vehicle 10, or when the vehicle The first event occurs when an intrusion into vehicle 10 is detected. The second event occurs when a person approaches vehicle 10.

[0058] The event detection unit 112 transmits the result of the determination to the recording unit 111 and the event notification unit 113.

[0059] When the event detection unit 112 detects that a first event has occurred, the event notification unit 113 sends first information to the user terminal 300 to notify it of the occurrence of the first event. However, when the event detection unit 112 detects that a second event has occurred, the event notification unit 113 does not send information to the user terminal 300 to notify it of the occurrence of the second event.

[0060] The transmitting unit 114 transmits the video data captured by the in-vehicle camera 400 to the user terminal 300. Specifically, when the transmitting unit 114 receives a video transmission request (first request) from the user terminal 300, it transmits the video data captured by the in-vehicle camera 400 to the user terminal 300 in response to the first request. The transmitting unit 114 may also have a server function for providing video data to the user terminal 300.

[0061] Next, we will describe the components of the information processing device 100 other than the control unit 110.

[0062] The memory unit 120 is a main memory device such as RAM or ROM, an EPROM, a hard disk drive, and an auxiliary memory device such as removable media. The auxiliary memory device stores the operating system (OS), various programs, various tables, etc., and by executing the programs stored therein, each function that matches the predetermined purpose of each part of the control unit 110 can be realized. However, some or all of the functions may be realized by hardware circuits such as ASICs or FPGAs.

[0063] The storage unit 120 stores data used or generated in the processing performed by the control unit 110. The storage unit 120 may also store data used or generated in the processing for various functions provided by the vehicle 10 (for example, a security monitoring function or a remote control function) to perform services.

[0064] The communication unit 130 is composed of a communication circuit that performs wireless communication. The communication unit 130 may be, for example, a communication circuit that performs wireless communication using 4G, or a communication circuit that performs wireless communication using 5G. Alternatively, the communication unit 130 may be a communication circuit that performs wireless communication using LTE, or a communication circuit that performs communication using LPWA (Low Power Wide Area). Furthermore, the communication unit 130 may include a wired communication circuit, and may communicate with a user terminal 300 or the like that is connected to the communication unit 130 by a wire.

[0065] Furthermore, the communication unit 130 may include a hardware interface that provides access to the in-vehicle network of the vehicle 10.

[0066] Next, we will describe the devices included in the system other than the information processing device 100.

[0067] First, let's describe the server 200. The server 200 is a server device installed outside the vehicle 10 that communicates wirelessly with the information processing device 100. The server 200 may be a server capable of storing and processing large amounts of data. The server 200 connects to the information processing device 100 via a wide-area network such as the internet. For example, the server 200 may be implemented as a cloud server.

[0068] Next, the user terminal 300 will be described. The user terminal 300 is a computer owned by the user of vehicle 10.

[0069] The control unit 310 of the user terminal 300 is implemented using a processor such as a CPU or GPU and memory, similar to the control unit 110. The control unit 310 includes a receiving unit 311 and a notification unit 312 as functional modules. These functional modules may be implemented by executing a program using the control unit 310.

[0070] The receiving unit 311 receives first information from the information processing device 100 (event notification unit 113) that a first event has occurred in the vehicle 10.

[0071] When the receiving unit 311 receives the first information, the notification unit 312 notifies the user of the occurrence of the first event via a predetermined interface. For example, if the notification unit 312 determines that the first event has occurred, it may output information indicating the occurrence of the first event to the display unit. Alternatively, when the receiving unit 311 receives the first information, the notification unit 312 may output audio indicating the occurrence of the first event from the output unit.

[0072] The display 340 is a display unit that displays images, etc. The display 340 may be a touch panel display, an organic EL (Electro-Luminescence) display, or a liquid crystal display.

[0073] Next, I will describe the equipment provided by vehicle 10.

[0074] The in-vehicle camera 400 is a digital camera equipped with an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) sensor, and is mounted on the vehicle 10. The in-vehicle camera 400 may be, for example, equipped in a drive recorder or the like, or it may be a digital camera installed independently of the drive recorder. The in-vehicle camera 400 may consist of multiple cameras. Each of the multiple cameras may, for example, capture images of the front, rear, sides, and interior of the vehicle 10.

[0075] Sensor 500 is a sensor mounted on the vehicle 10, and specifically includes ultrasonic sensors, tilt sensors, etc. The sensor may also include acceleration sensors, gyro sensors, GPS (Global Positioning System) sensors, millimeter-wave radar, 3D-LiDAR (Light Detection and Ranging), etc. The event detection unit 112 of the information processing device 100 may acquire sensor data detected by sensor 500.

[0076] Note that the configuration shown in Figure 2 is just one example, and all or part of the illustrated functions may be performed using specially designed circuits. Furthermore, program storage and execution may be performed using combinations of main memory and auxiliary memory other than those shown.

[0077] Next, we will explain the specific details of the processing performed by the information processing device 100. Figure 3 is a flowchart of the processing performed by the control unit 110. The illustrated processing is repeatedly executed while the vehicle 10's driving system is stopped and the vehicle 10 is being monitored by the information processing device 100.

[0078] First, in step S11, the event detection unit 112 determines whether or not a first event has occurred in the vehicle 10. Specifically, based on data acquired from the in-vehicle camera 400 or sensor 500, the event detection unit 112 determines whether or not an impact has occurred in the vehicle 10, or whether or not an impact has occurred in the vehicle 10. The system determines whether a first event, such as an intrusion, has occurred. If the event detection unit 112 determines that a first event has occurred in the vehicle 10, this step results in a positive determination.

[0079] If the result in this step is positive, the process proceeds to step S12.

[0080] If the result in this step is negative, the process proceeds to step S14.

[0081] When the process transitions to step S12, the recording unit 111 stores the first video data in a predetermined storage device. Specifically, the recording unit 111 stores the first video data in both the storage unit 120 of the information processing device 100 and the server 200 outside the information processing device 100. When the recording unit 111 stores the first video data in the server 200, it may also connect to a wide-area network such as the Internet via the communication unit 130 and upload the first video data to the server 200.

[0082] Next, in step S13, the event notification unit 113 transmits first information to the user terminal 300 to notify it of the occurrence of the first event. In other words, the event notification unit 113 notifies the user terminal 300 that an impact to the vehicle 10 or an intrusion into the vehicle 10 has occurred.

[0083] If the process transitions to step S14, the event detection unit 112 determines whether a second event has occurred in the vehicle 10. Specifically, the event detection unit 112 determines whether a second event, which is the approach of a person to the vehicle 10, has occurred, based on data acquired from the in-vehicle camera 400 or sensor 500. If the event detection unit 112 determines that a second event has occurred in the vehicle 10, this step results in a positive determination.

[0084] If the result in this step is positive, the process proceeds to step S15.

[0085] If the result in this step is negative, the process proceeds to step S11. Then, the above process is repeated.

[0086] Step S14 may be performed before step S11. In that case, if the result in step S14 is positive, the process proceeds to step S15; if the result in step S14 is negative, the process proceeds to step S11. If the result in step S11 is positive, the process proceeds to step S12; if the result in step S11 is negative, the process proceeds to step S14.

[0087] If the process transitions to step S15, the event notification unit 113 does not send information notifying the user terminal 300 of the occurrence of the second event, and the recording unit 111 stores the second video data in a predetermined storage device. At this time, the recording unit 111 records the second video data only in the storage unit 120 of the information processing device 100, and does not record it in the server 200 outside of the information processing device 100.

[0088] Next, we will explain the specific details of the processing performed by the user terminal 300. Figure 4 is a flowchart of the processing performed by the user terminal 300 when it notifies the user. The processing shown is executed when the first information is transmitted from the information processing device 100 to the user terminal 300.

[0089] First, in step S21, the receiving unit 311 receives the first information transmitted from the information processing device 100.

[0090] Next, in step S22, the notification unit 312 notifies the user that the first event has occurred. Notification is provided. Specifically, the notification unit 312 outputs information to the display unit indicating that the first event has occurred. The notification unit 312 may also output an audio notification indicating that the first event has occurred, or it may output a vibration notification indicating that the first event has occurred.

[0091] Next, in step S23, the notification unit 312 of the user terminal 300 outputs an interface for viewing the video from the in-vehicle camera 400 to the display unit. In this step, the user terminal 300 outputs an icon that can be pressed by the user to the display or other device provided by the user terminal 300. This icon is an example of an "interface for viewing the video obtained by the in-vehicle camera".

[0092] Next, in step S24, the notification unit 312 determines whether or not an icon for checking the video from the in-vehicle camera 400 has been pressed. Specifically, the notification unit 312 may determine whether or not it has detected a signal emitted when the icon is pressed.

[0093] If the notification unit 312 determines that the icon for checking the video feed from the in-vehicle camera 400 has been pressed, this step is considered a positive result.

[0094] If the result in this step is positive, the process proceeds to step S25.

[0095] If the result in this step is negative, the process terminates.

[0096] If the process proceeds to step S25, the notification unit 312 displays the video captured by the in-vehicle camera 400 on the display 340. In this step, the notification unit 312 outputs a user interface for logging into the server provided by the information processing device 100 (transmission unit 114), and logs into the server based on the input information. Once login is complete, the notification unit 312 requests the information processing device 100 (transmission unit 114) to transmit video. Upon receiving this request, the transmission unit 114 of the information processing device 100 reads the video from the storage unit 120 at the time the first event occurred and transmits it to the user terminal 300. The notification unit 312 then outputs the video transmitted by the transmission unit 114. The notification unit 312 displays the video captured by the in-vehicle camera 400, which has been transmitted from the information processing device 100, on a screen or the like. The video transmitted from the information processing device 100 may be the video at the time the first event occurred (i.e., the video recorded in the storage unit 120), or it may be video acquired in real time. Furthermore, these videos may be still images or moving images.

[0097] In this example, the information processing device 100 reads video from the storage unit 120 and transmits it to the user terminal 300. However, the video transmitted to the user terminal 300 may be video stored in the server 200. In this case, the user terminal 300 may send a request to the server 200, and the server 200 may transmit the video to the user terminal 300 based on this request. Alternatively, the user terminal 300 may send a request to the information processing device 100, and upon receiving this request, the information processing device 100 may instruct the server 200 to transmit the video, and the server 200 may transmit the video to the user terminal 300 accordingly.

[0098] Thus, the user terminal 300 displays an icon on its display unit, such as a screen, that accepts instructions to display the video captured by the in-vehicle camera 400, and when the icon is pressed, it outputs the video to the display unit. In other words, when the user terminal 300 receives information indicating that the first event has occurred, The system outputs a user interface that allows the user to view the video captured by the in-vehicle camera 400.

[0099] Next, we will explain the user interface output by the user terminal 300, as described in Figure 4. Figure 5 shows an example of a screen output by the user terminal 300. The screen shown is output when the user clicks the icon to log in to the server function of the transmission unit 114 of the information processing device 100, and the first information is notified to the user terminal 300.

[0100] As shown in Figure 5(a), the display screen 1000 shows a user ID input field 1110 and a password input field 1120 for entering the user's account information, and a login button 1130 for logging into the server using the entered account information.

[0101] When the login button 1130 is pressed, as shown in Figure 5(b), a real-time button 1210 for viewing images captured by the in-vehicle camera 400 in real time and a monitoring history button 1220 for viewing images captured by the in-vehicle camera 400 in the past are displayed. For example, if the user wants to view the images currently captured by the in-vehicle camera 400 in real time, they press the real-time button 1210.

[0102] When the real-time button 1210 is pressed, a screen for selecting one of the multiple on-board cameras mounted on vehicle A is displayed, as shown in Figure 5(c). For example, a front camera button 1310, a rear camera button 1320, an interior camera button 1330, a 360° / BGM camera button 1340, etc., are displayed. If vehicle A is mounted on only one on-board camera, only one button for selecting the camera is displayed.

[0103] The user selects one of the in-car cameras whose footage they want to view and presses the corresponding icon. For example, if the user wants to view the footage from the front camera, they press the front camera button 1310.

[0104] When the front camera button 1310 is pressed, the real-time video 1410 is displayed on the display screen 1000, as shown in Figure 5(d). In this case, the real-time video 1410 is the video captured by the front camera mounted on vehicle A.

[0105] When the monitoring history button 2220 shown in Figure 5(b) is pressed, the screen switches to an interface that provides video footage previously captured by the in-vehicle camera and stored in the storage unit 120 or server 200. On this interface, the user can select and view the previously captured video footage.

[0106] As described above, the information processing device according to this embodiment switches whether or not to send a notification to the user terminal 300 when a first event occurs and when a second event occurs. Specifically, the control unit 110 sends information to the user terminal 300 when a first event occurs, and does not send information to the user terminal 300 when a second event occurs. With this configuration, the information processing device 100 can reduce unnecessary notifications to the user terminal 300 while the vehicle 10 is being monitored by the information processing device 100.

[0107] Furthermore, when a first event occurs, the control unit 110 records the video data in both the storage unit 120 and the server 200, and when a second event occurs, it records the video data only in the storage unit 120. With this configuration, it becomes possible to appropriately select the storage location for the video data according to its importance.

[0108] (Modification of the first embodiment) In the first embodiment, the information processing device 100 transmitted the first information to the user terminal 300 only when the first event occurred. In contrast, in this modified example, the information processing device 100 transmits notification information to the user terminal 300 regardless of the type of event, and the user terminal 300 decides whether or not to notify the user.

[0109] As described in the first embodiment, when a first event occurs, the information processing device 100 (event notification unit 113) transmits the first information to the user terminal 300. On the other hand, in this modified example, when a second event occurs, the information processing device 100 (event notification unit 113) also transmits the second information to the user terminal 300. The second information is information that notifies the user that a second event has occurred.

[0110] In other words, in this modified example, the information processing device 100 (event notification unit 113) notifies the user terminal 300 that an event has occurred, regardless of the type of event.

[0111] In this modified example, the notification unit 312, as described in Figure 1, determines whether or not a first event has occurred in the vehicle 10 based on the first or second information received by the receiving unit 311. Specifically, the notification unit 312 determines that a first event has occurred in the vehicle 10 if the receiving unit 311 receives the first information, and determines that a second event has occurred in the vehicle 10 if the receiving unit 311 receives the second information.

[0112] As mentioned earlier, the second event is one in which users should not be notified immediately. Therefore, in this modified version, the user terminal 300 switches between notifying the user when it receives the first information and when it receives the second information. Specifically, when the user terminal 300 receives the first information, it notifies the user of the occurrence of the first event via a predetermined interface, and when it receives the second information, it does not notify the user.

[0113] Thus, the process of switching whether or not to send notifications depending on the type of event may be performed on the user terminal 300 side.

[0114] (Second Embodiment) In the first embodiment, when the information processing device 100 detects a first event, it uploads the video footage of the moment the first event occurred (for example, when an impact was detected) to the server 200. In other words, only the video footage of the moment the first event was detected is uploaded to the server 200.

[0115] On the other hand, if a second event (e.g., a person was detected) was detected before a first event (e.g., an impact was detected), it is preferable for the information processing device 100 to retrospectively upload the video in order to confirm both events.

[0116] The second embodiment addresses this by uploading, in addition to the video corresponding to the first event, the video corresponding to the second event (second video data) to the server 200 if the first event occurs within a predetermined time after the second event occurs.

[0117] In the first embodiment, when the event detection unit 112 determines that a first event has occurred, the recording unit 111 records the first video data in a predetermined storage device (storage unit 120 and server 200).

[0118] In contrast, in the second embodiment, if the event detection unit 112 determines that the first event occurred within a predetermined time after the second event occurred, the recording unit 111 reads the second video data that was recorded when the second event occurred from the storage unit 120 and transmits it to the server 200. In other words, if the first event occurs within a predetermined time after the second event occurs, both the first video data and the second video data are transmitted to the server 200.

[0119] By performing the above operations, the recording unit 111 can retrospectively save video data from when the first event occurred, as well as video data from when the second event occurred, to the server 200.

[0120] In the first embodiment, the second video data is recorded only in the storage unit 120, so in order to read the data, it is necessary to connect equipment or move the storage medium. On the other hand, in the second embodiment, the second video data, which is considered to be of high importance, can be recorded in the server 200. This makes it easy to perform services such as providing the recorded video to the user.

[0121] In the second embodiment, the event detection unit 112 determines whether the first event occurred within a predetermined time after the occurrence of the second event. For this purpose, the event detection unit 112 may determine the times of occurrence of the first and second events and measure the elapsed time between the occurrence times of the first and second events. The determination result is transmitted to the recording unit 111 and the event notification unit 113.

[0122] Furthermore, if the event detection unit 112 determines that the first event occurred within a predetermined time period after the occurrence of the second event, the event notification unit 113 may transmit both the first and second information to the user terminal 300. In other words, the event notification unit 113 may notify the user terminal 300 that the first event occurred following the second event.

[0123] Figure 6 is a flowchart of the process performed by the control unit 110 in the second embodiment. The illustrated process is performed after step S13 in Figure 3 is executed.

[0124] In step S31, the event detection unit 112 detects whether a second event occurred during a predetermined period prior to the occurrence of the first event determined in step S11. Specifically, the event detection unit 112 determines the time of occurrence of the first event (which occurred this time) and the second event (which occurred in the past), and measures the elapsed time between the occurrence times of the first and second events. If the event detection unit 112 determines that the first event occurred within a predetermined time period after the occurrence of the second event, this step results in a positive determination.

[0125] If the result in this step is positive, the process proceeds to step S32.

[0126] If the result in this step is negative, the process terminates.

[0127] If the process proceeds to step S32, the recording unit 111 sends (copies) the second video data recorded when the second event occurred to the server 200. As a result, the server 200 stores both the first video data transmitted in step S12 and the second video data transmitted in this step.

[0128] Next, in step S33, the event notification unit 113 sends the second information to the user terminal 300. To be sure, the event notification unit 113 sends a notification to the user terminal 300 informing it that a second event has occurred in the vehicle 10.

[0129] As a result, the user terminal 300 receives both the first information transmitted in step S13 and the second information transmitted in this step.

[0130] Furthermore, when the user terminal 300 receives the first and second pieces of information, it may output an interface for viewing the second video data in addition to the interface for viewing the first video data. This makes it possible to retrospectively review the video footage from when the second event occurred.

[0131] (Third embodiment) In the first embodiment, the user is notified only when a first event occurs (for example, when an impact is detected), and no notification is given when a second event occurs. On the other hand, it is preferable to notify the user that a second event is continuing to occur if the duration of the second event continues to occur exceeds a predetermined threshold.

[0132] The third embodiment addresses this by notifying the user terminal 300 of the occurrence of a second event when the duration of the second event continues to exceed a predetermined threshold.

[0133] In this embodiment, when the event detection unit 112 detects a second event, it determines whether the duration of the second event has exceeded a predetermined threshold. Here, "the second event occurring continuously" means that the second event occurs again within a predetermined time after the first event occurred. The duration of the second event refers to the time (period) during which the second event can be evaluated as occurring continuously. In other words, the event detection unit 112 monitors whether the second event occurs again within a predetermined time after the first event occurred. If the second event occurs again within the predetermined time, the time during which the second event repeatedly occurs within the predetermined time may be measured.

[0134] Furthermore, the event notification unit 113 transmits the second information to the user terminal 300 when the event detection unit 112 determines that the duration of the second event has exceeded a predetermined threshold.

[0135] Figure 7 is a flowchart of the process performed by the control unit 110 in the third embodiment. The illustrated process is performed after step S15 in Figure 3 is executed.

[0136] In step S41, the event detection unit 112 determines whether the duration of the second event determined in step S14 has exceeded a predetermined threshold. In this step, the event detection unit 112 determines whether a second event has occurred again within a predetermined time after the first event occurred, and whether its duration has exceeded a predetermined threshold.

[0137] If the event detection unit 112 determines that the duration of the second event has exceeded a predetermined threshold, this step results in a positive determination.

[0138] If the result in this step is positive, the process proceeds to step S42.

[0139] If the result in this step is negative, the process terminates.

[0140] If the process transitions to step S42, the event notification unit 113 sends the second information to the user terminal 300. In other words, the event notification unit 113 sends the second information to the user terminal 300, regarding the vehicle 10. Send a notification informing us that a second event has occurred. In this example, the event notification unit 113 sends the second information, that is, information simply notifying that a person has been detected, to the user terminal 300. However, instead, it may also send information (referred to as the third information) to the user terminal 300 notifying that a person has been continuously detected. Upon receiving the third information, the user terminal 300 may, for example, notify the user that a person has been continuously detected for a predetermined period of time. Furthermore, when the user terminal 300 receives the second information, it may display an icon for checking the video from the in-vehicle camera 400, and display an interface for checking the second video data on the display 340.

[0141] (Fourth embodiment) In the first embodiment, the user terminal 300, in response to receiving the first information, outputs an interface for viewing the camera image. On the other hand, the user terminal 300 may request the information processing device 100 to transmit the camera image, etc., at any time.

[0142] The fourth embodiment is one in which a user terminal 300 sends a request to the information processing device 100A at any time, and when the information processing device 100A receives the request, it executes processing according to the respective request.

[0143] Figure 8 is a diagram illustrating the components of the information processing device 100A according to the fourth embodiment. In the fourth embodiment, the control unit 110A of the information processing device 100A includes a device operation unit 115 in addition to the components described in the first embodiment. The device operation unit 115 is a functional module that operates one of several devices on the vehicle 10 in response to a request from the user terminal 300. The device operation unit 115 may also have server functionality, similar to the transmission unit 114. The user terminal 300 can request the operation of the aforementioned devices by logging into the server operated by the device operation unit 115.

[0144] In the fourth embodiment, the transmitting unit 114 receives a first request from the user terminal 300. Here, the first request is, for example, information instructing the information processing device 100A to transmit the video captured by the in-vehicle camera 400 to the user terminal 300. The first request may include the date and time the video was recorded and information specifying the camera. When the transmitting unit 114 receives the first request, it reads the corresponding video from the storage unit 120 or acquires real-time video from the in-vehicle camera 400 and transmits it to the user terminal 300.

[0145] In the fourth embodiment, the device operation unit 115 receives a second request. The second request is, for example, information in which the user terminal 300 instructs the information processing device 100A to operate one of several devices, including different types of devices, that are provided in the vehicle 10.

[0146] Examples of multiple devices include windshield wipers, car horns, or headlights.

[0147] Furthermore, when the device operation unit 115 receives a second request, it activates one of several devices, including different types of devices, that are provided by the vehicle 10. Here, devices provided by the vehicle 10 include, for example, wipers, car horns, or headlights.

[0148] Figure 9 is a flowchart of the process executed by the control unit 110A when the information processing device 100A receives a first request from the user terminal 300. Here, the first request is: As described above, this is information that the user terminal 300 instructs the information processing device 100A to transmit the video captured by the in-vehicle camera 400 to the user terminal 300.

[0149] First, in step S51, the transmission unit 114 determines whether or not it has received the first request from the user terminal 300. In this step, the user terminal 300 outputs a login screen for logging into the server provided by the information processing device 100A (transmission unit 114) in response to the user's operation. After the user logs in, the user terminal 300 generates the first request in response to a predetermined operation by the user and transmits it to the information processing device 100 (transmission unit 114).

[0150] If the transmitting unit 114 determines that it has received the first request, this step results in an affirmative determination.

[0151] If the result in this step is positive, the process proceeds to step S52.

[0152] If the result in this step is negative, the process terminates.

[0153] When the process transitions to step S52, the transmission unit 114 transmits the video data captured by the in-vehicle camera 400 to the user terminal 300. The transmission unit 114 may transmit video data that has been previously stored in the storage unit 120, or it may instruct the server 200 to transmit video data stored in the server 200 to the user terminal 300. Alternatively, the transmission unit 114 may transmit real-time video captured by the in-vehicle camera 400 to the user terminal 300 via streaming transmission.

[0154] Figure 10 is a flowchart of the process that the control unit 110A executes when the information processing device 100A receives a second request. Here, the second request is, as described above, information instructing the information processing device 100A by the user terminal 300 to activate one of several devices, including different types of devices, that are installed in the vehicle 10.

[0155] First, in step S61, the device operation unit 115 determines whether or not it has received a second request from the user terminal 300. In this step, the user terminal 300 outputs a login screen for logging into the server provided by the information processing device 100A (device operation unit 115) in response to the user's operation. After the user logs in, the user terminal 300 generates a second request in response to a predetermined operation by the user and sends it to the information processing device 100A (device operation unit 115). The second request may include information specifying the device to be operated, operating parameters, etc.

[0156] If the device operation unit 115 determines that it has received the second request, this step results in a positive determination.

[0157] If the result in this step is positive, the process proceeds to step S62.

[0158] If the result in this step is negative, the process terminates.

[0159] If the process proceeds to step S62, the equipment operation unit 115 activates the equipment specified by the second request from among the multiple pieces of equipment provided by the vehicle 10. The equipment operation unit 115 may activate any one of the multiple pieces of equipment provided by the vehicle 10, or it may activate multiple pieces of equipment simultaneously or sequentially.

[0160] Specifically, the device control unit 115 may, for example, sound the car horn of the vehicle 10 or operate the wipers of the vehicle 10. The device control unit 115 may also control the head of the vehicle 10. The headlights may be turned on. The equipment provided by vehicle 10 as described herein can be any equipment that can be used to alert people in the vicinity of vehicle 10 of an emergency, or to deter people who attempt to steal vehicle 10 or break into vehicle 10.

[0161] (modified version) Furthermore, a process described as being performed by a single device may be divided and executed by multiple devices. Conversely, a process described as being performed by different devices may be executed by a single device. In a computer system, the hardware configuration (server configuration) used to implement each function can be flexibly changed.

[0162] The present disclosure can also be realized by supplying a computer program implementing the functions described in the above embodiments to a computer, and having one or more processors in the computer read and execute the program. Such a computer program may be provided to the computer by a non-temporary computer-readable storage medium that can be connected to the computer's system bus, or it may be provided to the computer via a network. Non-temporary computer-readable storage mediums include, for example, any type of disk such as magnetic disks (floppy disks, hard disk drives (HDDs), etc.), optical disks (CD-ROMs, DVDs, Blu-ray discs, etc.), read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards, flash memory, optical cards, and any type of medium suitable for storing electronic instructions. [Explanation of Symbols]

[0163] 10.. Vehicles 100... Information Processing Device 110, 310... Control Unit 111... Records Department 112...Event detection unit 113...Event Notification Section 120, 320...Storage section 130, 330... Communications Department 200... Server 300...User terminals 400...In-car camera

Claims

1. An information processing device installed in a vehicle, When the vehicle's running system is not running, When a first event occurs in the vehicle, the first video data, which is video data of the moment the first event occurred, captured by the vehicle's onboard camera, is recorded in the first storage device and external server device of the information processing device, and first information notifying the occurrence of the first event is transmitted to the user terminal. If a second event of a different type from the first event occurs in the vehicle, instead of transmitting information notifying the user terminal of the occurrence of the second event, the second video data, which is video data of the moment the second event occurred and captured by the in-vehicle camera, is recorded in the first storage device. It has a control unit that performs the following: The control unit, When the vehicle's driving system is not running, if the first event occurs within a predetermined time period after the second event occurs, the second video data recorded in the first storage device at the time of the second event is copied to the external server device. Information processing device.

2. The control unit, Determine the time of occurrence of the first event and the second event, The elapsed time between the occurrence times of the first event and the second event is measured. The information processing apparatus according to claim 1.

3. The control unit, When the vehicle's driving system is not running, if the duration for which the second event continues to occur exceeds a predetermined threshold, a second piece of information notifying the occurrence of the second event is transmitted to the user terminal. The information processing apparatus according to claim 1.

4. The control unit, The length of the period during which the second event occurs continuously is measured as the length of the period during which the second event occurs repeatedly within a predetermined interval. The information processing apparatus according to claim 3.

5. The control unit, When a first request is received from the user terminal, the video data captured by the in-vehicle camera in response to the first request is transmitted to the user terminal. The information processing apparatus according to claim 1.

6. The control unit, In response to the first request, the video data is streamed to the user terminal. The information processing apparatus according to claim 5.

7. The control unit, When a second request is received from the user terminal, the vehicle activates one of several devices, including devices of different types. The information processing apparatus according to claim 1.

8. The aforementioned plurality of devices include at least one of the following: wipers, car horns, or headlights. The information processing apparatus according to claim 7.

9. The control unit, Based on sensor data obtained from sensors mounted on the vehicle, it is determined whether or not the first event occurred in the vehicle. The information processing apparatus according to claim 1.

10. The control unit determines that the first event is the detection of an impact on the vehicle or an intrusion into the vehicle. The information processing apparatus according to claim 9.

11. The control unit, Based on sensor data obtained from sensors mounted on the vehicle, it is determined whether or not the second event occurred in the vehicle. The information processing apparatus according to claim 1.

12. The control unit determines that the second event is the detection of a person approaching the vehicle. The information processing apparatus according to claim 11.

13. An information processing method performed by an information processing device installed in a vehicle, When the vehicle's running system is not running, When a first event occurs in the vehicle, the first video data, which is video data of the moment the first event occurred, captured by the vehicle's onboard camera, is recorded in the first storage device and external server device of the information processing device, and first information notifying the occurrence of the first event is transmitted to the user terminal. If a second event of a different type from the first event occurs in the vehicle, the vehicle will not send information notifying the user terminal of the occurrence of the second event, but will instead use the in-vehicle camera The second video data, which is video data of the second event that occurred as captured by the camera, is recorded in the first storage device. Includes, When the vehicle's driving system is not running, if the first event occurs within a predetermined time period after the second event occurs, the second video data recorded in the first storage device at the time of the second event is copied to the external server device. Information processing methods.

Images