Information processing apparatus

Abstract

An information processing device is mounted on a vehicle provided with a plurality of cameras. A control unit of the information processing apparatus determines a priority order of a plurality of recording data recorded by a plurality of cameras on the basis of a predetermined event occurring during parking of a vehicle. At this time, the control unit determines the priority order of the plurality of pieces of recording data such that the recording data, among the plurality of pieces of recording data, having the highest association with the predetermined event is the uppermost. The control unit uploads the plurality of recorded data to a predetermined server in an order corresponding to the specified priority order.

Description

Technical Field

[0001] The present disclosure relates to an information processing device. Background Art

[0002] There is known a technology in which a system issues an alarm when a suspicious person is detected from an image captured by a camera and changes the detection range of the suspicious person according to the parking position of the vehicle (for example, refer to Japanese Patent Application Laid-Open No. 2020-149088). Summary of the invention

[0003] An object of the present disclosure is to provide a technology that is effective in ensuring vehicle safety.

[0004] The present disclosure can be understood as an information processing device mounted on a vehicle equipped with a plurality of cameras.

[0005] The information processing device in this case, for example, includes a control unit, which is configured to perform the following processing: determining the priority of multiple recording data recorded by the multiple cameras based on a specified event occurring during the parking of the vehicle; and uploading the multiple recording data to an external server based on the determined priority.

[0006] The present disclosure may also be understood as an information processing method in which a computer mounted on a vehicle executes the processing of the above-mentioned information processing device. Furthermore, the present disclosure may also be understood as an information processing program for causing a computer mounted on a vehicle to execute the above-mentioned information processing method or a non-temporary storage medium storing the information processing program.

[0007] According to the present disclosure, it is possible to provide a technology that is effective in ensuring vehicle safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Features, advantages, and technical and industrial significance of exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which like reference numerals represent like elements, and in which:

[0009] Figure 1 It is a diagram showing an overview of a system in an embodiment.

[0010] Figure 2 It is a diagram showing an example of the software configuration of the vehicle-mounted device in the embodiment.

[0011] Figure 3 This is a diagram showing an example of the priority order in the embodiment.

[0012] Figure 4 This is a flowchart showing the flow of processing executed in the vehicle-mounted device in the embodiment. DETAILED DESCRIPTION

[0013] As a service to ensure the safety of a vehicle while parked, a service that uploads the recorded data of a camera to an external server when a predetermined event occurs in a vehicle while parked is being studied. In this service, the user of the vehicle can confirm the recorded data uploaded to the server by notifying the user of the occurrence of the predetermined event from the external server.

[0014] However, when implementing the above service, it is also assumed that the capacity of recorded data that can be uploaded from the vehicle to the server or the length of communication time that can be used for uploading each time a predetermined event occurs will be limited. Therefore, in a vehicle equipped with multiple cameras, recorded data that is highly relevant to the predetermined event may not be uploaded to the server. As a result, it may be difficult to ensure the safety of the vehicle.

[0015] Therefore, in the information processing device involved in the present disclosure, the control unit determines the priority of multiple recording data recorded by multiple cameras according to the specified event that occurs while the vehicle is parked. As an example, in a vehicle equipped with a first camera that shoots the front of the vehicle, a second camera that shoots the rear of the vehicle, a third camera that shoots the surroundings of the vehicle, and a fourth camera that shoots the interior of the vehicle, if the specified event is detection of intrusion into the interior of the vehicle, the control unit can determine the priority of the recording data of the fourth camera to be the highest. And, in a vehicle equipped with the first to fourth cameras, if the specified event is detection of an impact, the control unit can determine the priority of the recording data of the third camera to be the highest.

[0016] In addition, in a vehicle that is equipped with a sensor (for example, a multi-axis acceleration sensor, etc.) that can identify the direction in which the vehicle is impacted in addition to the first to fourth cameras, the control unit can determine the direction in which the vehicle is impacted based on the detection signal of the sensor, and determine the priority based on the determination result. As an example, in a case where it is determined that the vehicle is impacted from the front, the control unit can determine the priority of the recorded data of the first camera as the highest. Also, in a case where it is determined that the vehicle is impacted from the rear, the control unit can determine the priority of the recorded data of the second camera as the highest. Also, in a case where it is determined that the vehicle is impacted from the side, the control unit can determine the priority of the recorded data of the third camera as the highest.

[0017] After determining the priority of the plurality of recording data according to the above steps, the control unit uploads the plurality of recording data to the server according to the determined priority, that is, the control unit first uploads the recording data with the highest priority to the server.

[0018] According to the information processing device involved in the present disclosure, among a plurality of recorded data recorded by a plurality of cameras, recorded data with high relevance to a specific event can be preferentially uploaded to a server. As a result, even if the capacity of recorded data that can be uploaded from a vehicle to a server or the length of communication time that can be used for uploading is limited each time a specified event occurs, recorded data with high relevance to the specified event can be uploaded to the server more reliably.

[0019] In addition, the recording by each of the multiple cameras may be performed all the time while the vehicle is parked, or may be started when a specified event occurs. When the recording by each of the multiple cameras is performed all the time while the vehicle is parked, only the recorded data including the occurrence of the specified event and the specified time length before and after the specified event may be uploaded to the server. Alternatively, all the recorded data from the recording start time to the upload start time may be uploaded to the server.

[0020] The following is a description of specific embodiments of the present disclosure based on the accompanying drawings. Unless otherwise specifically stated, the hardware structure, module structure, functional structure, etc. described in the following embodiments are not intended to limit the technical scope of the disclosure to these.

[0021] Implementation

[0022] In the present embodiment, an example in which the information processing device according to the present disclosure is applied to a system for monitoring a vehicle while parked will be described.

[0023] System Overview

[0024] Figure 1 1 is a diagram showing an overview of a system in this embodiment. The system in this embodiment includes an onboard device 100 and a server 200. The onboard device 100 and the server 200 are connected to each other via a network. The network is, for example, a world-wide public communication network WAN such as the Internet or other communication networks.

[0025] The vehicle-mounted device 100 is a computer installed in the vehicle 10, which corresponds to the "information processing device" involved in the present disclosure. Figure 1 As shown, the vehicle-mounted device 100 includes a processor 101, a main storage device 102, an auxiliary storage device 103, a front camera 104, a rear camera 105, an omnidirectional camera 106, an in-vehicle camera 107, a multi-axis acceleration sensor 109, and a communication unit 110. Figure 1 In the illustrated example, only the hardware structure related to parking monitoring is extracted and illustrated, but the vehicle-mounted device 100 may include other hardware structures.

[0026] The processor 101 is a calculation processing device such as a central processing unit (CPU) or a digital signal processor (DSP). The processor 101 loads a program stored in the auxiliary storage device 103 into the main storage device 102 and executes the program, thereby controlling the vehicle-mounted device 100.

[0027] The main storage device 102 is composed of semiconductor memories such as random access memory (RAM) and read only memory (ROM). The main storage device 102 provides a storage area and a work area for loading programs stored in the auxiliary storage device 103. In addition, the main storage device 102 is used as a buffer for the processor 101 to perform calculations.

[0028] The auxiliary storage device 103 is, for example, an erasable programmable read-only memory (EPROM) or a hard disk drive (HDD). The auxiliary storage device 103 may include a removable medium, i.e., a portable recording medium. The removable medium is, for example, a disk recording medium such as a Universal Serial Bus (USB) memory, a compact disc (CD), or a digital versatile disc (DVD). The auxiliary storage device 103 stores various programs and data used by the processor 101 when executing each program. In addition to the operating system (OS), the programs stored in the auxiliary storage device 103 also include special programs for the processor 101 to perform processing related to parking monitoring. In addition, the data stored in the auxiliary storage device 103 include the recording data of the front camera 104, the rear camera 105, the omnidirectional camera 106, and the in-vehicle camera 107.

[0029] The front camera 104 captures the front of the vehicle 10. The rear camera 105 captures the rear of the vehicle 10. The omnidirectional camera 106 captures the surroundings of the vehicle 10 (a 360° range centered on the vehicle 10 from a planar perspective). The in-vehicle camera 107 captures the interior of the vehicle 10. In addition, the front camera 104, the rear camera 105, the omnidirectional camera 106, and the in-vehicle camera 107 are sometimes collectively referred to as cameras 104 to 107 hereinafter.

[0030] The intrusion sensor 108 detects a moving body (e.g., a person) entering the interior of the vehicle 10. The intrusion sensor 108 is, for example, an infrared sensor or an ultrasonic sensor. The multi-axis acceleration sensor 109 detects acceleration in multiple directions including the front-back direction, the left-right direction, and the height direction of the vehicle 10. In this embodiment, the detection signal of the multi-axis acceleration sensor 109 is used to determine the direction of the impact to which the vehicle 10 is subjected during parking.

[0031] The communication unit 110 is a communication interface for connecting the vehicle-mounted device 100 to a network. In one example, the communication unit 110 is configured to include a network interface connected to the network using a mobile communication method (e.g., Long Term Evolution (LTE), LTE-Advanced, 5th Generation (5G) or 6th Generation (6G)) or a wireless communication method such as Wi-Fi (registered trademark). The communication unit 110 communicates with the server 200 through the network.

[0032] Software structure of vehicle-mounted device

[0033] according to Figure 2 The software structure of the vehicle-mounted device 100 in this embodiment is described. Figure 2 As shown in FIG. 1 , the vehicle-mounted device 100 includes a control unit F110 as a software module. Figure 2 In the examples shown, constituent elements may be added or replaced as appropriate.

[0034] The control unit F110 is implemented by the processor 101 of the vehicle-mounted device 100 loading the dedicated program stored in the auxiliary storage device 103 into the main storage device 102 and executing it. In addition, the control unit F110 can be implemented by a hardware circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0035] When the control unit F110 detects a predetermined event when the vehicle 10 is in the parked state, the cameras 104 to 107 start recording. The parked state of the vehicle 10 refers to a state in which the drive system is stopped (the ignition switch or the power switch is off), the doors are locked, and the passengers have gotten off. Whether the passengers have gotten off can be determined based on the camera data of the in-vehicle camera 107, or based on the detection signal of the seat sensor. Alternatively, whether the passengers have gotten off can be determined based on whether there is a smart key in the vehicle 10.

[0036] The predetermined events in this embodiment are the intrusion of a moving body (e.g., a person, etc.) into the interior of the vehicle 10 and the impact of the vehicle 10. The intrusion of the moving body into the interior of the vehicle 10 is detected by the intrusion sensor 108. In addition, the intrusion of the moving body into the interior of the vehicle 10 may be determined by whether unlocking of the door by an inappropriate method is detected. The impact of the vehicle 10 is detected by the multi-axis acceleration sensor 109.

[0037] If a specified event occurs while the vehicle 10 is parked, the control unit F110 starts recording through the cameras 104 to 107. At this time, in one example, the recording time of the cameras 104 to 107 can be a predetermined time length (for example, several minutes to several tens of minutes). In other examples, the recording time of the cameras 104 to 107 can be changed according to the type of the specified event. That is, when the specified event is an impact of the vehicle 10, the recording is performed for a predetermined time length. When the specified event is a moving object intruding into the interior of the vehicle 10, the recording can be performed until the moving object retreats from the interior.

[0038] Furthermore, the control unit F110 determines the type of a specified event. The "specified type of event" mentioned here includes a moving body intruding into the interior of the vehicle 10, an impact from the front of the vehicle 10, an impact from the rear of the vehicle 10, an impact from the side of the vehicle 10, and the like. When the specified event is detected by the intrusion sensor 108, the control unit F110 determines that the type of the specified event is "a moving body intruding into the interior of the vehicle 10." Furthermore, when the specified event is detected by the multi-axis acceleration sensor 109, the control unit F110 determines the direction of the impact received by the vehicle 10 by analyzing the detection signal of the multi-axis acceleration sensor 109. That is, the control unit F110 determines whether it is an impact from the front of the vehicle 10, an impact from the rear of the vehicle 10, or an impact from the side of the vehicle 10.

[0039] After determining the type of the specified event, the control unit F110 determines the priority of the four recording data recorded by the four cameras 104 to 107 according to the determined type of the specified event. At this time, the control unit F110 determines the priority of the recording data that is most related to the specified event as the highest. Here, an example of the priority of the recording data is shown in Figure 3 middle.

[0040] exist Figure 3 In the example shown, when the type of the predetermined event is a moving object intruding into the interior of the vehicle 10 ( Figure 3 The priority order when "intrusion" is detected) is set from the highest to "recording data of the in-vehicle camera 107", "recording data of the omnidirectional camera 106", "recording data of the front camera 104", and "recording data of the rear camera 105".

[0041] When the type of the predetermined event is an impact from the front of the vehicle 10 ( Figure 3 The priority order when detecting "front impact" in the image is set from the highest to "recording data of the front camera 104", "recording data of the omnidirectional camera 106", "recording data of the rear camera 105", and "recording data of the in-vehicle camera 107".

[0042] When the type of the predetermined event is an impact from the rear of the vehicle 10 ( Figure 3 The priority order in the case of "impact from behind" is set from the highest to "recording data of the rear camera 105", "recording data of the omnidirectional camera 106", "recording data of the front camera 104", and "recording data of the in-vehicle camera 107".

[0043] When the type of the predetermined event is an impact from the side of the vehicle 10 ( Figure 3 The priority order when detecting a "side impact" is set from the highest to "recording data of the all-round camera 106", "recording data of the front camera 104", "recording data of the rear camera 105", and "recording data of the in-vehicle camera 107".

[0044] in addition, Figure 3 The priority order shown is merely an example, and the priority order other than the highest priority (first priority) recorded data can be appropriately changed according to the embodiment.

[0045] If the priority order of the four recorded data recorded by the four cameras 104 to 107 is determined, the control unit F110 uploads the four recorded data to the server 200 according to the determined priority order. In one example, when the type of the specified event is the intrusion of a moving body into the interior of the vehicle 10, the recorded data is uploaded to the server 200 in the order of "recorded data of the interior camera 107", "recorded data of the omnidirectional camera 106", "recorded data of the front camera 104", and "recorded data of the rear camera 105". Furthermore, when the type of the specified event is an impact from the front of the vehicle 10, the recorded data is uploaded to the server 200 in the order of "recorded data of the front camera 104", "recorded data of the omnidirectional camera 106", "recorded data of the rear camera 105", and "recorded data of the interior camera 107". Furthermore, when the type of the specified event is an impact from the rear of the vehicle 10, the recorded data is uploaded to the server 200 in the order of "recorded data of the rear camera 105", "recorded data of the omnidirectional camera 106", "recorded data of the front camera 104", and "recorded data of the in-vehicle camera 107". Furthermore, when the type of the specified event is an impact from the side of the vehicle 10, the recorded data is uploaded to the server 200 in the order of "recorded data of the omnidirectional camera 106", "recorded data of the front camera 104", "recorded data of the rear camera 105", and "recorded data of the in-vehicle camera 107".

[0046] In addition, the uploading of the recorded data may be performed after the recording of the cameras 104 to 107 is completed, or may be performed in parallel with the recording of the cameras 104 to 107 .

[0047] Furthermore, the recording by the cameras 104 to 107 may be performed all the time while the vehicle 10 is parked. In this case, the control unit F110 may upload the recording data of the cameras 104 to 107 to the server 200, including the recording data of the predetermined event and the predetermined time length before and after the event.

[0048] Processing Flow

[0049] Next, according to Figure 4 The flow of processing executed in the vehicle-mounted device 100 in this embodiment will be described. Figure 4 This is a flowchart of a processing routine executed in the vehicle-mounted device 100 when a predetermined event occurs as a trigger while the vehicle 10 is parked (the ignition switch or the power switch is off). Figure 4 The execution subject of the processing routine is the processor 101 of the vehicle-mounted device 100, but here, the software module (control unit F110) of the vehicle-mounted device 100 is explained as the execution subject.

[0050] exist Figure 4 In the processing routine, the control unit F110 detects the occurrence of a predetermined event by the intrusion sensor 108 or the multi-axis acceleration sensor 109 (S101). After executing the processing of S101, the control unit F110 executes the processing of S102.

[0051] In S102, the control unit F110 sends a recording start instruction to each of the cameras 104 to 107, thereby starting recording by the cameras 104 to 107. After executing the process of S102, the control unit F110 executes the process of S103.

[0052] In S103, the control unit F110 determines the type of the specified event. At this time, if the specified event is detected by the intrusion sensor 108, the control unit F110 determines the type of the specified event as "a moving body intrudes into the interior of the vehicle 10". Furthermore, if the specified event is detected by the multi-axis acceleration sensor 109, the control unit F110 determines the direction of the impact on the vehicle 10 by analyzing the detection signal of the multi-axis acceleration sensor 109. That is, the control unit F110 determines whether the type of the specified event is "an impact from the front of the vehicle 10", "an impact from the rear of the vehicle 10", or "an impact from the side of the vehicle 10". After executing the processing of S103, the control unit F110 executes the processing of S104.

[0053] In S104, the control unit F110 determines the priority order of the four recording data of the four cameras 104 to 107 according to the type of the predetermined event determined in S103. In one example, the control unit F110 may determine the priority order of the four recording data of the four cameras 104 to 107 according to the type of the predetermined event determined in S103. Figure 3 The priority order of the four recorded data is determined in the order shown in the example. After executing the process of S104, the control unit F110 executes the process of S105.

[0054] In S105, the control unit F110 uploads the four recording data of the four cameras 104 to 107 to the server 200 according to the priority order determined in S104. After executing the process of S105, the control unit F110 ends the execution of this processing routine.

[0055] Functions and effects of implementation methods

[0056] In the vehicle-mounted device 100 of the above-mentioned embodiment, when a predetermined event is detected while the vehicle 10 is parked, the type of the predetermined event is determined. Then, based on the determination result, the priority order of the four recorded data of the four cameras 104 to 107 is determined. In the vehicle-mounted device 100 of the present embodiment, the four recorded data are uploaded to the server 200 according to the determined priority order.

[0057] Therefore, according to the vehicle-mounted device 100 in this embodiment, the recorded data with the highest relevance to the prescribed event among the four recorded data recorded by the four cameras 104 to 107 can be preferentially uploaded to the server 200. As a result, even if the recorded data capacity that can be uploaded from the vehicle-mounted device 100 to the server 200 or the communication time that can be used for uploading is limited every time a prescribed event occurs, the recorded data with the highest relevance to the prescribed event can be uploaded to the server 200 more reliably. As a result, the safety of the vehicle 10 during parking can be more reliably ensured.

[0058] other

[0059] The above embodiment is only an example, and the present disclosure can be implemented with appropriate changes within the scope of the subject matter. For example, the number of cameras mounted on the vehicle 10 is not limited to 4, but may be 3 or less, or may be 5 or more. Here, when an impact is detected in a vehicle equipped with only an omnidirectional camera and an in-vehicle camera, the priority of the recorded data of the omnidirectional camera only needs to be determined as the highest.

[0060] Furthermore, when an impact is detected in a vehicle that is not equipped with a multi-axis acceleration sensor, the priority of the recording data of the omnidirectional camera may be determined to be the highest. Alternatively, it may be determined whether the impact is from the front or from the rear depending on whether the vehicle is parked forward or backward, and the priority may be determined based on the determination result.

[0061] Furthermore, a process described as being performed by one device may be shared and executed by a plurality of devices. Alternatively, a process described as being performed by different devices may be performed by one device. In a computer system, it is possible to flexibly change the hardware structure by which each function is realized.

Claims

1. An information processing device mounted on a vehicle equipped with a plurality of cameras, The information processing device is characterized in that A control unit is provided, and the control unit is configured to execute the following processing: determining a priority order of a plurality of recording data recorded by a plurality of the cameras according to a prescribed event occurring during parking of the vehicle; and The plurality of recording data are uploaded to a prescribed server in an order corresponding to the priority order.

2. The information processing device according to claim 1, characterized in that The plurality of cameras include: a first camera that captures the front of the vehicle; a second camera that captures the rear of the vehicle; a third camera that captures the surroundings of the vehicle; and a fourth camera that captures the interior of the vehicle. When the predetermined event is detection of intrusion into the interior of the vehicle, the control unit determines the priority of the recorded data of the fourth camera to be the highest.

3. The information processing device according to claim 2, characterized in that: When the predetermined event is detection of an impact, the control unit determines the priority of the recorded data of the third camera to be the highest.

4. The information processing device according to claim 2, characterized in that The vehicle is provided with a multi-axis acceleration sensor, When the predetermined event is detection of an impact, the control unit executes the following processing: Determining the direction in which the vehicle is impacted according to the detection signal of the multi-axis acceleration sensor; When it is determined that the vehicle is hit from the front, the priority of the recorded data of the first camera is determined to be the highest; When it is determined that the vehicle is hit from behind, the priority of the recorded data of the second camera is determined to be the highest; When it is determined that the vehicle is impacted from the side, the priority of the recorded data of the third camera is determined to be the highest.

Images