Information processing device, method, and program

Abstract

To improve a technique about storing an operational image.SOLUTION: An information processor 20 has a controller 23 which stores an image of areas around a vehicle 10, determines that storage of the image is to be maintained when it is determined that there is an abnormality in the behaviors of the vehicle 10 on the basis of at least one index other than an acceleration which shows behaviors of the vehicle 10 and determines that the image is to be deleted when it is not determined that there is an abnormality in the behaviors of the vehicle 10.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a method, and a program. [Background technology]

[0002] There has been a demand for technology relating to the evaluation and storage of vehicle driving conditions. For example, Patent Document 1 discloses a technology for evaluating driving conditions. Also, a technology relating to the storage of driving images when an accident occurs is known. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-187816 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional driving image storage only saves images when an accident is detected (when a large acceleration is detected), and there is room for improvement in the technology for saving driving images.

[0005] In view of the above circumstances, an object of the present disclosure is to improve the technology relating to the storage of driving images. [Means for solving the problem]

[0006] An information processing device according to an embodiment of the present disclosure includes: An information processing device, Save the captured image of the vehicle's surroundings, The control unit is provided with a control unit that decides to keep the image stored if it is determined that there is an abnormality in the behavior of the vehicle based on at least one indicator other than acceleration that indicates the behavior of the vehicle, and decides to delete the image if it is not determined that there is an abnormality in the behavior of the vehicle.

[0007] According to one embodiment of the present disclosure, a method comprises: A method executed by an information processing device, Storing images of the vehicle's surroundings; and Deciding to keep the image stored when it is determined that there is an abnormality in the behavior of the vehicle based on at least one indicator other than acceleration that indicates the behavior of the vehicle, and deciding to delete the image when it is not determined that there is an abnormality in the behavior of the vehicle. Includes:

[0008] A program according to an embodiment of the present disclosure includes: In the information processing device, Storing images of the vehicle's surroundings; and Deciding to keep the image stored when it is determined that there is an abnormality in the behavior of the vehicle based on at least one indicator other than acceleration that indicates the behavior of the vehicle, and deciding to delete the image when it is not determined that there is an abnormality in the behavior of the vehicle. Execute the following. [Effects of the Invention]

[0009] According to one embodiment of the present disclosure, techniques for saving driving images are improved. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram illustrating a schematic configuration of a system according to an embodiment of the present disclosure. [Figure 2] 1 is a block diagram showing a schematic configuration of a vehicle. [Figure 3] FIG. 1 is a block diagram showing a schematic configuration of an information processing device. [Figure 4] 10 is a flowchart illustrating an operation of the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described.

[0012] <Outline of the embodiment> An overview of a system 1 according to an embodiment of the present disclosure will be described with reference to Fig. 1. The system 1 includes a vehicle 10 and an information processing device 20. The vehicle 10 and the information processing device 20 are communicably connected to a network 30 including, for example, the Internet and a mobile communication network.

[0013] The vehicle 10 is, for example, an automobile, but is not limited to this and may be any vehicle. The automobile may be any type of automobile, such as a gasoline vehicle, a diesel vehicle, a hybrid electric vehicle (HEV), a plug-in hybrid electric vehicle (PHEV), a battery electric vehicle (BEV), or a fuel cell electric vehicle (FCEV). In this embodiment, the vehicle 10 is driven by a driver, but the driving may be automated at any level. The automation level is, for example, any of levels 1 to 5 in the classification of the Society of Automotive Engineers (SAE). The vehicle 10 may also be a vehicle dedicated to MaaS (Mobility as a Service). The number of vehicles 10 included in the system 1 may be determined arbitrarily.

[0014] In this embodiment, the vehicle 10 has a function of acquiring, storing, and / or transmitting images of the surroundings of the vehicle 10. The vehicle 10 also acquires and transmits information (indicators) indicating the behavior of the vehicle 10.

[0015] The information processing device 20 is, for example, a computer such as a server device, etc. The information processing device 20 is capable of communicating with the vehicle 10 via a network 30.

[0016] First, an overview of this embodiment will be described, and details will be provided later. The information processing device 20 saves images of the periphery of the vehicle 10. The information processing device 20 determines whether there is an abnormality in the behavior of the vehicle 10 based on at least one index other than acceleration that indicates the behavior of the vehicle 10. If it is determined that there is an abnormality in the behavior of the vehicle 10, the information processing device 20 decides to continue saving the images. If it is not determined that there is an abnormality in the behavior of the vehicle 10, the information processing device 20 decides to delete the images.

[0017] As described above, according to the present embodiment, when it is determined that the behavior of the vehicle 10 is abnormal based on at least one index other than acceleration, images of the periphery of the vehicle 10 are kept stored. Here, the behavior of the vehicle 10 includes not only behaviors determined to be dangerous based on acceleration (G value), but also behaviors that may lead to dangerous driving even if they are not determined to be dangerous based on acceleration (for example, the generation of an abnormal sound from the vehicle 10 or the user uttering a surprised voice). Both behaviors are equally likely to be behaviors for which the user is likely to require image storage. However, with conventional cameras such as drive recorders, images of behaviors that are not determined to be dangerous based on acceleration are deleted. For this reason, it has been difficult to keep all images of behaviors for which the user is likely to require image storage. On the other hand, according to the present embodiment, it is easier to keep all images of behaviors that may lead to dangerous driving regardless of acceleration. Therefore, technology related to the storage of driving images is improved in that it is easier to keep all images of behaviors for which the user is likely to require image storage.

[0018] Next, each component of the system 1 will be described in detail.

[0019] <Vehicle configuration> As shown in FIG. 2, the vehicle 10 includes a communication unit 11, an acquisition unit 12, an imaging unit 13, an output unit 14, an input unit 15, a storage unit 16, and a control unit 17.

[0020] The communication unit 11 includes one or more communication interfaces connected to the network 30. The communication interfaces are compatible with mobile communication standards such as, but not limited to, 4G (4th Generation) or 5G (5th Generation). In this embodiment, the vehicle 10 communicates with the information processing device 20 via the communication unit 11 and the network 30.

[0021] The acquisition unit 12 includes one or more devices that acquire location information of the vehicle 10. Specifically, the acquisition unit 12 includes, for example, a receiver compatible with the Global Positioning System (GPS), but is not limited thereto and may include a receiver compatible with any satellite positioning system. The acquisition unit 12 also includes any sensor module that can acquire information indicating the state of the vehicle 10. For example, the sensor module includes a vibration sensor, an infrared sensor, a speed sensor, an angular velocity sensor, an acceleration sensor, a geomagnetic sensor, a distance sensor such as a LiDAR (light detection and ranging), or a combination thereof.

[0022] The imaging unit 13 includes one or more cameras. Each camera included in the imaging unit 13 may be provided on the vehicle 10 so as to be able to capture an image of a subject outside or inside the vehicle. In this embodiment, the imaging unit 13 includes a front camera capable of capturing an image of a subject in front of the vehicle 10, a rear camera capable of capturing an image of a subject behind the vehicle 10, and an in-vehicle camera capable of capturing an image of a subject inside the vehicle 10. Alternatively, the imaging unit 13 may include an omnidirectional camera capable of capturing an image of a subject around the vehicle 10. Furthermore, the imaging unit 13 may include an input interface capable of communicating with a camera such as a drive recorder installed on the vehicle 10 so as to be able to capture an image of a subject outside or inside the vehicle, and may be able to acquire images of multiple users riding in the vehicle 10 from the camera.

[0023] The output unit 14 may include one or more output devices that output information to notify the user. Examples of the output devices include, but are not limited to, a display that outputs information visually or a speaker that outputs information audibly. The output unit 14 may include an interface for connecting an external output device.

[0024] The input unit 15 includes one or more input devices that detect user input. In this embodiment, the input unit 15 includes a microphone that receives input of the user's voice and sounds emitted by the vehicle 10. However, the input device is not limited to these examples and may include, for example, physical keys, capacitive keys, or a touch screen integrated with a display. The input unit 15 may also include an interface for connecting an external input device.

[0025] The storage unit 16 includes one or more memories. The memories may be, for example, semiconductor memories, magnetic memories, optical memories, etc., but are not limited to these. Each memory included in the storage unit 16 may function, for example, as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 16 stores any information used in the operation of the vehicle 10. For example, the storage unit 16 may store system programs, application programs, embedded software, etc. The information stored in the storage unit 16 may be updateable with information obtained from the network 30 via the communication unit 11, for example. In this embodiment, the storage unit 16 may store images (or videos) of the surroundings of the vehicle 10.

[0026] The control unit 17 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The processor may be, for example, a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process, but is not limited to these. The programmable circuit may be, for example, but is not limited to, an FPGA (Field-Programmable Gate Array). The dedicated circuit may be, for example, but is not limited to, an ASIC (Application Specific Integrated Circuit). The control unit 17 controls the overall operation of the vehicle 10.

[0027] <Configuration of information processing device> As shown in FIG. 3, the information processing device 20 includes a communication unit 21, a storage unit 22, and a control unit .

[0028] The communication unit 21 includes one or more communication interfaces connected to the network 30. The communication interfaces correspond to, for example, a mobile communication standard, a wired LAN (Local Area Network) standard, or a wireless LAN standard, but are not limited to these and may correspond to any communication standard. In this embodiment, the information processing device 20 communicates with the vehicle 10 via the communication unit 21 and the network 30.

[0029] The storage unit 22 includes one or more memories. Each memory included in the storage unit 22 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 22 stores any information used in the operation of the information processing device 20. For example, the storage unit 22 may store system programs, application programs, databases, map information, etc. The information stored in the storage unit 22 may be updateable with information obtained from the network 30 via the communication unit 21, for example. In this embodiment, the storage unit 22 can save images (or videos) of the surroundings of the vehicle 10 received from the vehicle 10.

[0030] The control unit 23 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The control unit 23 controls the operation of the information processing device 20 as a whole.

[0031] <Operation flow of information processing device> The operation of the information processing device 20 according to this embodiment will be described with reference to Fig. 4. The operation of Fig. 4 corresponds to the method according to this embodiment. The operation of Fig. 4 is repeatedly executed, for example, at a predetermined cycle. The predetermined cycle can be determined arbitrarily.

[0032] Step S100: The control unit 23 of the information processing device 20 stores an image of the periphery of the vehicle 10.

[0033] Specifically, the control unit 23 of the information processing device 20 receives, from the vehicle 10 via the communication unit 21 and the network 30, images of the surroundings of the vehicle 10 captured at an arbitrary frame rate by the imaging unit 13 of the vehicle 10 (hereinafter also referred to as "captured images"). The control unit 23 stores the received captured images in the memory unit 22, thereby saving the captured images of the surroundings of the vehicle 10. However, the method for saving the captured images is not limited to this, and any method can be adopted. Furthermore, although the captured images are stored in the memory unit 22 in this embodiment, the method is not limited to this. For example, the captured images may be stored in an external storage connected to the information processing device 20, or may be stored in the memory unit 16 of the vehicle 10.

[0034] Furthermore, in this embodiment, simultaneously with the imaging unit 13 of the vehicle 10 capturing an image, the input unit 15 records the behavior of the user aboard the vehicle 10 and sounds emitted by the vehicle 10, and the acquisition unit 12 measures vibrations emitted by the vehicle 10 and the distance between the vehicle 10 and objects located around the vehicle 10. Then, the control unit 23 of the information processing device 20 receives the above-mentioned captured image, as well as the following information (1) to (4) accompanying the captured image, from the vehicle 10 via the communication unit 21 and the network 30. (1) Information indicating the behavior of the user while riding in the vehicle 10 (hereinafter also referred to as “first information”) (2) Information indicating whether or not there is an abnormal sound from the vehicle 10 (hereinafter also referred to as “second information”) (3) Information indicating the presence or absence of abnormal vibration of the vehicle 10 (hereinafter also referred to as “third information”) (4) Information indicating the distance between the vehicle 10 and an object located around the vehicle 10 (hereinafter also referred to as “fourth information”)

[0035] The first information in (1) above may include any information indicating the user's behavior, such as the user's vocalizations or body movements while aboard the vehicle 10. There may be one or more users aboard the vehicle 10, but in the following description, it is assumed that only one user, i.e., the driver of the vehicle 10, is aboard the vehicle 10. The user's behavior may include any behavior, but in this embodiment, the first information is the user's utterance of a first voice, which is the user's calm voice, or a second voice, which is different from the first voice. Therefore, the first information in this embodiment is information indicating the user's voice recorded via the input unit 15 of the vehicle 10. The user's voice recorded via the input unit 15 includes the first voice or the second voice. The second voice, which is different from the user's first voice, which is the user's calm voice, may be any voice suggesting the user's critical situation, but in this embodiment, it is the user's surprised voice (typically, a scream).

[0036] The second information (2) above may include any information indicating a sound emitted by the vehicle 10, but in this embodiment, it is information indicating a sound emitted by the vehicle 10 that is recorded via the input unit 15 of the vehicle 10. The sound emitted by the vehicle 10 that is recorded via the input unit 15 includes normal sounds and abnormal sounds. A normal sound is a sound emitted by a component of the vehicle 10 when there is no accident or malfunction of the vehicle 10. An abnormal sound is a sound emitted by a component of the vehicle 10 when there is an accident or malfunction of the vehicle 10. For example, abnormal sounds include a sound caused by a collision or contact between the vehicle 10 and another object, a tire friction sound during sudden braking, or a sound caused by a vehicle malfunction such as engine trouble.

[0037] The third information (3) above may include any information indicating vibrations emitted by the vehicle 10, but in this embodiment, it is a measurement value of the vibrations emitted by the vehicle 10 measured via the acquisition unit 12 of the vehicle 10. The vibrations emitted by the vehicle 10 measured via the acquisition unit 12 include normal vibrations and abnormal vibrations. Normal vibrations refer to vibrations emitted by components of the vehicle 10 when no accident or malfunction of the vehicle 10 has occurred. Abnormal vibrations refer to vibrations emitted by components of the vehicle 10 when an accident or malfunction of the vehicle 10 has occurred. For example, abnormal vibrations include vibrations caused by a collision or contact between the vehicle 10 and another object, vibrations of the vehicle body during sudden braking, or vibrations caused by a vehicle malfunction such as engine trouble.

[0038] The fourth information (4) above may include any information indicating the distance between the vehicle 10 and an object located around the vehicle 10, but in this embodiment, it is a measurement value of the distance between the vehicle 10 and an object located around the vehicle 10 (hereinafter also referred to as "distance D") measured via the acquisition unit 12 of the vehicle 10. The object may be, for example, another vehicle, an obstacle, a person, an animal, or a combination thereof. Any method can be used to calculate the distance D. For example, the control unit 17 of the vehicle 10 may calculate the distance D by analyzing the position information of the vehicle 10 acquired via the acquisition unit 12, distance information to surrounding objects acquired from a distance sensor or the like, and / or captured images acquired via the imaging unit 13 of the vehicle 10.

[0039] The control unit 23 stores the received first information to fourth information in the storage unit 22. Then, each time the control unit 23 executes the operation of FIG. 4 , it acquires the first information, second information, third information, or fourth information associated with any number of frames of captured images by reading it from the storage unit 22. Note that the first information to fourth information acquired here can be used as at least one index other than acceleration in the vehicle behavior determination described below. However, the acquisition of the first information to fourth information is not limited to these examples, and any method can be adopted.

[0040] Step S101: The control unit 23 acquires at least one indicator other than acceleration that indicates the behavior of the vehicle 10. The at least one indicator includes any information other than acceleration that can identify the behavior of the vehicle 10. The at least one indicator may be an indicator acquired from outside the vehicle 10, an indicator inherent to the vehicle 10, or both.

[0041] Specifically, the control unit 23 acquires at least one indicator by reading out from the storage unit 22 at least one of the first information to the fourth information stored in the storage unit 22 in step S100. As will be understood from the following explanation, the more indicators acquired here, the more likely it is that images of behaviors that may lead to dangerous driving regardless of acceleration will be saved. Therefore, it becomes easier to save all images of behaviors that are likely to require saving by the user. Therefore, the more indicators acquired, the better. In the following explanation, it is assumed that the at least one indicator includes all of the first information to the fourth information.

[0042] Next, the control unit 23 of the information processing device 20 determines whether or not there is an abnormality in the behavior of the vehicle 10 based on at least one index acquired in step S101 (hereinafter also referred to as "vehicle behavior determination"). Specifically, the control unit 23 performs the vehicle behavior determination by executing the operations of steps S102 to S105, which will be described later. The vehicle behavior determination can be summarized as follows. If the determination result is affirmative in any of steps S102 to S105, the control unit 23 determines that there is an abnormality in the behavior of the vehicle 10. In this case, the captured image stored in the storage unit 22 in step S100 is maintained (step S107, which will be described later). On the other hand, if the determination results are negative in all of steps S102 to S105, the control unit 23 does not determine that there is an abnormality in the behavior of the vehicle 10. In this case, the captured image stored in the storage unit 22 in step S100 is deleted (step S106, which will be described later).

[0043] The vehicle behavior determination according to this embodiment will be described in detail below. In order to focus on making it easier to maintain the storage of captured images during periods where it has been difficult to do so in the past, the following description will be given on the assumption that the vehicle behavior determination is performed during periods when the acceleration of the vehicle 10 is equal to or less than a threshold. For periods when the acceleration of the vehicle 10 exceeds the threshold, the same processing as in the past may be performed. For example, when the control unit 23 detects that the acceleration of the vehicle 10 has exceeded the threshold, the control unit 23 may determine to maintain the storage of a predetermined number of frames of captured images in the storage unit 22 (for example, several seconds before and after the detection point). On the other hand, when the acceleration of the vehicle 10 is equal to or less than the threshold and the determination results in steps S102 to S105 are negative, the control unit 23 may determine to delete the captured images stored in the storage unit 22.

[0044] Step S102: The control unit 23 determines whether or not there is an abnormality in the user's behavior based on the first information acquired in step S101 (hereinafter also referred to as "user behavior determination"). If it is determined that there is an abnormality in the user's behavior (step S102-Yes), the process proceeds to step S107. On the other hand, if it is not determined that there is an abnormality in the user's behavior (step S102-No), the process proceeds to step S103.

[0045] Specifically, the control unit 23 determines whether the user's voice acquired in step S101 is a first voice or a second voice (hereinafter also referred to as "voice determination"). Any method can be employed for the voice determination. For example, the control unit 23 of the information processing device 20 may construct a model that classifies neutral voices and voices suggesting a critical situation. Any method can be employed for constructing the model. For example, the control unit 23 may construct a model by recording neutral voices and voices suggesting a critical situation of multiple men and women to collect a large number of samples, extracting physical features such as spectrograms or cepstrograms of each voice, and performing machine learning such as deep learning. The control unit 23 may perform voice analysis of the received user's voice based on the constructed model, and determine whether the voice is the user's first voice or the second voice based on the result of the voice analysis. If the received user's voice is determined to be the second voice as a result of the voice analysis, the control unit 23 determines that there is an abnormality in the user's behavior.

[0046] Step S103: If it is determined that the user's behavior is not abnormal (step S102-No), the control unit 23 determines whether or not an abnormal sound of the vehicle 10 has been detected, based on the second information acquired in step S101. If an abnormal sound of the vehicle 10 has been detected (step S103-Yes), the process proceeds to step S107. On the other hand, if an abnormal sound of the vehicle 10 has not been detected (step S103-No), the process proceeds to step S104.

[0047] Specifically, the control unit 23 determines whether the sound emitted by the vehicle 10 acquired in step S101 is a normal sound or an abnormal sound (hereinafter also referred to as "vehicle sound determination"). If the sound is an abnormal sound, the control unit 23 determines that an abnormal sound from the vehicle 10 has been detected. Any method can be adopted for vehicle sound determination. For example, the control unit 23 of the information processing device 20 may determine that the sound emitted by the vehicle 10 is a normal sound if a value (e.g., in dB) indicating the loudness of the sound emitted by the vehicle 10 is equal to or less than a predetermined threshold, and may determine that the sound emitted by the vehicle 10 is an abnormal sound if the value exceeds the predetermined threshold. The predetermined threshold can be set arbitrarily, and for example, a threshold may be set for each frequency band, such as high and low tones.

[0048] Step S104: If no abnormal sound of the vehicle 10 is detected (step S103-No), the control unit 23 determines whether or not abnormal vibration of the vehicle 10 is detected, based on the third information acquired in step S101. If abnormal vibration of the vehicle 10 is detected (step S104-Yes), the process proceeds to step S107. On the other hand, if abnormal vibration of the vehicle 10 is not detected (step S104-No), the process proceeds to step S105.

[0049] Specifically, the control unit 23 determines whether the vibrations emitted by the vehicle 10 acquired in step S101 are normal vibrations or abnormal vibrations (hereinafter also referred to as "vibration determination"). If the vibrations are abnormal vibrations, the control unit 23 determines that abnormal vibrations of the vehicle 10 have been detected. Any method can be employed for the vibration determination. For example, the control unit 23 of the information processing device 20 may determine that the vibrations emitted by the vehicle 10 are normal vibrations when a value (e.g., in Hz) indicating the frequency of the elastic vibrations emitted by the vehicle 10 is within a predetermined threshold range, and may determine that the vibrations emitted by the vehicle 10 are abnormal vibrations when the value exceeds the predetermined threshold range. The predetermined threshold range can be set arbitrarily. Alternatively, the control unit 23 of the information processing device 20 may determine that the vibrations emitted by the vehicle 10 are normal vibrations when a value indicating the vibration intensity (e.g., vibration amplitude) of the elastic vibrations emitted by the vehicle 10 is equal to or less than a predetermined threshold, and may determine that the vibrations emitted by the vehicle 10 are abnormal vibrations when the value exceeds the predetermined threshold. The predetermined threshold can be set arbitrarily.

[0050] Step S105: If abnormal vibration of the vehicle 10 is not detected (step S104-No), the control unit 23 determines whether or not an object approaching at a distance equal to or less than a predetermined threshold (hereinafter also referred to as an "approaching object") has been detected, based on the fourth information acquired in step S101. If an approaching object has been detected (step S105-Yes), the process proceeds to step S107. On the other hand, if an approaching object has not been detected (step S105-No), the process proceeds to step S106.

[0051] Specifically, the control unit 23 determines whether the distance D acquired in step S101 is equal to or less than a predetermined threshold (hereinafter also referred to as "distance determination"). If the distance D is equal to or less than the predetermined threshold, the control unit 23 determines that an approaching object has been detected. As described above, the expected approaching object may be another vehicle, an obstacle, a person, an animal, or a combination thereof. The predetermined threshold can be set arbitrarily, and, for example, a different value may be set for each expected approaching object or for each traveling speed of the vehicle 10. For example, the predetermined threshold for another vehicle may be set to any distance that does not violate the obligation to maintain a safe distance between vehicles as stipulated in the Road Traffic Act of Japan.

[0052] Step S106: If an approaching object is not detected (step S105-No), the control unit 23 determines to delete the captured image stored in the storage unit 22 in step S100. Then, the process ends.

[0053] Specifically, the control unit 23 deletes the captured image stored in the storage unit 22 in step S100 from the storage unit 22. Any method can be used to delete the captured image. For example, the control unit 23 may delete the captured image from the storage unit 22 after a predetermined time has elapsed since the captured image was saved in the storage unit 22.

[0054] Step S107: If it is determined that there is something abnormal in the user's behavior (step S102—Yes), if an abnormal sound is detected from the vehicle 10 (step S103—Yes), if an abnormal vibration is detected from the vehicle 10 (step S104—Yes), or if an approaching object is detected (step S105—Yes), the control unit 23 decides to maintain the storage of the captured image stored in the memory unit 22 in step S100. Then, the process ends.

[0055] Specifically, if any of the determination results of steps S102 to S105 is affirmative, the control unit 23 determines that there is an abnormality in the behavior of the vehicle 10 and decides to maintain storage of the captured image stored in the storage unit 22 in step S100. The control unit 23 does not delete the captured image from the storage unit 22, but maintains storage of the captured image in the storage unit 22. Any method can be employed to maintain storage of the captured image. For example, the control unit 23 may maintain storage of a predetermined number of frames of captured images in the storage unit 22 (for example, several seconds before and after the time when it is determined that there is an abnormality in the behavior of the vehicle 10). Note that, although the determination is performed in the order of steps S102 to S105 in the flowchart of FIG. 4, the steps S102 to S105 may be arranged in any order, and the determination of each step may be performed in any order. Furthermore, each step may be executed in parallel.

[0056] In this way, after steps S101 to S107, if the control unit 23 determines that there is an abnormality in the behavior of the vehicle 10 based on at least one indicator other than acceleration that indicates the behavior of the vehicle 10, it decides to maintain the storage of the captured image stored in the memory unit 22 in step S100, and if it does not determine that there is an abnormality in the behavior of the vehicle 10, it decides to delete the captured image.

[0057] As described above, the information processing device 20 according to this embodiment stores images of the periphery of the vehicle 10. If it is determined that there is something abnormal in the behavior of the vehicle 10 based on at least one index other than acceleration that indicates the behavior of the vehicle 10, the information processing device 20 decides to keep the images stored, and if it is not determined that there is something abnormal in the behavior of the vehicle 10, the information processing device 20 decides to delete the images.

[0058] According to this configuration, unlike the conventional technology, it is easier to maintain the storage of images of behaviors that may lead to dangerous driving regardless of acceleration. Therefore, the technology related to the storage of driving images is improved in that it is easier to store all images of behaviors that are likely to require the user to store images.

[0059] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art may make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each component or step can be rearranged so as not to be logically inconsistent, and multiple components or steps can be combined or divided into one.

[0060] For example, in a modified example of this embodiment, the control unit 23 of the information processing device 20 may determine whether or not there is an abnormality in the behavior of the vehicle 10 based on the acceleration of the vehicle 10 in addition to at least one indicator. That is, the control unit 23 may perform parallel processing of storing captured images based on the vehicle behavior determination according to this embodiment and storing captured images based on the conventional acceleration. In this case, in step S101, the control unit 23 receives, from the vehicle 10 via the communication unit 21 and the network 30, information indicating the acceleration of the vehicle 10 measured via the acquisition unit 12 of the vehicle 10 (e.g., a measured acceleration value) in addition to at least one indicator. The control unit 23 may determine whether or not there is an abnormality in the behavior of the vehicle 10 based on the received information indicating the acceleration. Any method may be employed for determining the vehicle behavior based on the acceleration. For example, when acceleration exceeding a predetermined threshold is detected in the vehicle 10 (e.g., during sudden braking), the control unit 23 may determine that there is an abnormality in the behavior of the vehicle 10.

[0061] Furthermore, for example, in the above-described embodiment, the user behavior determination is performed using voice, but the control unit 23 may perform another process for determining whether or not there is an abnormality in the user's behavior, in addition to or instead of the voice determination. For example, the control unit 23 may receive images showing the user's body movements captured at an arbitrary frame rate by an in-vehicle camera or the like of the vehicle 10 from the vehicle 10 via the communication unit 21 and the network 30. The control unit 23 may construct a model for classifying calm body movements from body movements suggesting a critical situation by performing machine learning such as deep learning described in step S102. The control unit 23 may perform image analysis of the user's body movements shown in the received images based on the constructed model, and perform user behavior determination based on the results of the image analysis.

[0062] In addition, for example, in the above-described embodiment, the configuration and operation of the information processing device 20 may be distributed among multiple computers that can communicate with each other. Also, for example, an embodiment in which some or all of the components of the information processing device 20 are provided in the vehicle 10 is possible. For example, a drive recorder mounted in the vehicle 10 may include some or all of the components of the information processing device 20.

[0063] Also, for example, an embodiment is possible in which a general-purpose computer functions as the information processing device 20 according to the above-described embodiment. Specifically, a program describing the processing content for realizing each function of the information processing device 20 according to the above-described embodiment is stored in the memory of the general-purpose computer, and the program is read and executed by a processor. Therefore, the present disclosure can also be realized as a program executable by a processor, or a non-transitory computer-readable medium storing the program.

[0064] Some embodiments of the present disclosure will be described below as examples, however, it should be noted that the embodiments of the present disclosure are not limited to these examples. [Appendix 1] An information processing device, Save the captured image of the vehicle's surroundings, An information processing device comprising a control unit that decides to keep the image stored if it is determined that there is an abnormality in the behavior of the vehicle based on at least one indicator other than acceleration that indicates the behavior of the vehicle, and decides to delete the image if it is not determined that there is an abnormality in the behavior of the vehicle. [Appendix 2] 10. The information processing device according to claim 1, the at least one indicator includes information indicative of a user's behavior while riding in the vehicle; The information processing device, wherein the control unit determines that there is an abnormality in the behavior of the vehicle when it is determined that there is an abnormality in the behavior of the user. [Appendix 3] 10. The information processing device according to claim 2, the user behavior is that the user utters a first voice that is a calm voice of the user or a second voice that is different from the first voice; The information processing device, wherein the control unit determines that there is something abnormal in the behavior of the user when the utterance of the second voice by the user is detected. [Appendix 4] An information processing device according to any one of Supplementary Notes 1 to 3, the at least one indicator includes information indicating whether or not an abnormal sound is present in the vehicle; The control unit determines that there is an abnormality in the behavior of the vehicle when the abnormal sound is detected. [Appendix 5] An information processing device according to any one of Supplementary Notes 1 to 4, the at least one indicator includes information indicating whether or not abnormal vibration of the vehicle is occurring; The information processing device, wherein the control unit determines that there is an abnormality in the behavior of the vehicle when the abnormal vibration is detected. [Appendix 6] An information processing device according to any one of Supplementary Notes 1 to 5, the at least one indicator includes a distance between the vehicle and an object located around the vehicle; The information processing device, wherein the control unit determines that there is an abnormality in the behavior of the vehicle when an object approaching at a distance equal to or less than a predetermined threshold is detected. [Appendix 7] An information processing device according to any one of Supplementary Notes 1 to 6, The control unit determines whether or not there is an abnormality in the behavior of the vehicle based on the acceleration of the vehicle in addition to the at least one index. [Appendix 8] A method executed by an information processing device, Storing images of the vehicle's surroundings; and Deciding to keep the image stored when it is determined that there is an abnormality in the behavior of the vehicle based on at least one indicator other than acceleration that indicates the behavior of the vehicle, and deciding to delete the image when it is not determined that there is an abnormality in the behavior of the vehicle. A method comprising: [Appendix 9] 9. The method of claim 8, the at least one indicator includes information indicative of a user's behavior while riding in the vehicle; The method further includes determining that a behavior of a front vehicle is abnormal when it is determined that a behavior of the user is abnormal. [Appendix 10] 10. The method of claim 9, the user's behavior is that the user utters a first voice that is a calm voice of the user or a second voice that is different from the first voice; The method further includes determining that there is an abnormality in the user's behavior when the user's utterance of the second voice is detected. [Appendix 11] 11. The method of any one of appendices 8 to 10, comprising: the at least one indicator includes information indicating whether or not an abnormal sound is present in the vehicle; The method further includes determining that there is an abnormality in the behavior of the vehicle if the abnormal sound is detected. [Appendix 12] 12. The method of any one of appendices 8 to 11, comprising: the at least one indicator includes information indicating whether or not abnormal vibration of the vehicle is occurring; The method further includes determining that there is an abnormality in the behavior of the vehicle if the abnormal vibration is detected. [Appendix 13] 13. The method of any one of appendices 8 to 12, comprising: the at least one indicator includes a distance between the vehicle and an object located around the vehicle; The method further includes determining that there is an abnormality in the behavior of the vehicle when an object is detected approaching at a distance equal to or less than a predetermined threshold. [Appendix 14] 14. The method of any one of appendices 8 to 13, comprising: The method further includes determining whether there is an abnormality in the behavior of the vehicle based on the acceleration of the vehicle in addition to the at least one indicator. [Appendix 15] In the information processing device, Storing images of the vehicle's surroundings; and Deciding to keep the image stored when it is determined that there is an abnormality in the behavior of the vehicle based on at least one indicator other than acceleration that indicates the behavior of the vehicle, and deciding to delete the image when it is not determined that there is an abnormality in the behavior of the vehicle. A program that executes the following. [Appendix 16] 16. The program of claim 15, the at least one indicator includes information indicative of a user's behavior while riding in the vehicle; The program further causes the information processing device to determine that there is an abnormality in behavior of a vehicle in front of the user when it is determined that there is an abnormality in behavior of the user. [Appendix 17] 17. The program of claim 16, the user's behavior is that the user utters a first voice that is a calm voice of the user or a second voice that is different from the first voice; The program further causes the information processing device to determine that there is something abnormal in the behavior of the user when the utterance of the second voice by the user is detected. [Appendix 18] 18. The program according to any one of appendices 15 to 17, the at least one indicator includes information indicating whether or not an abnormal sound is present in the vehicle; The program further causes the information processing device to determine that there is an abnormality in the behavior of the vehicle when the abnormal sound is detected. [Appendix 19] 19. The program according to any one of appendices 15 to 18, the at least one indicator includes information indicating whether or not abnormal vibration of the vehicle is occurring; The program further causes the information processing device to determine that there is an abnormality in the behavior of the vehicle when the abnormal vibration is detected. [Appendix 20] 20. The program according to any one of appendices 15 to 19, the at least one indicator includes a distance between the vehicle and an object located around the vehicle; The program further causes the information processing device to determine that there is an abnormality in the behavior of the vehicle when an object approaching at a distance equal to or less than a predetermined threshold is detected. [Explanation of symbols]

[0065] 1 System 10 vehicles 11 Communications Department 12 Acquisition Department 13 Imaging unit 14 Output section 15 Input section 16 Memory section 17 Control Unit 20 Information processing equipment 21 Communications Department 22 Memory section 23 Control Unit 30 Network

Claims

1. An information processing device, Save the captured image of the vehicle's surroundings, a plurality of pieces of information accompanying the image, which can be used as at least one indicator other than acceleration that indicates the behavior of the vehicle, including first information indicating the behavior of the user while riding in the vehicle, second information indicating whether or not there is an abnormal sound from the vehicle, third information indicating whether or not there is an abnormal vibration from the vehicle, and fourth information indicating a distance between the vehicle and an object located around the vehicle; reading at least one piece of information from the plurality of pieces of information to obtain the at least one index; An information processing device comprising a control unit that decides to keep the image stored if it is determined that there is an abnormality in the vehicle's behavior based on the at least one acquired indicator, and decides to delete the image if it is not determined that there is an abnormality in the vehicle's behavior.

2. 2. The information processing device according to claim 1, the user behavior of the first information is a vocalization or a body movement of the user, The information processing device, wherein the control unit determines that there is an abnormality in the user's behavior when it is determined that there is an abnormality in the user's vocalization or the user's body movement.

3. 2. The information processing device according to claim 1, the user behavior of the first information is that the user uttered a first voice that is a calm voice of the user or a second voice that is different from the first voice, The information processing device, wherein the control unit determines that there is something abnormal in the behavior of the user when the utterance of the second voice by the user is detected.

4. 4. The information processing device according to claim 1, The control unit determines whether or not there is an abnormality in the behavior of the vehicle based on the at least one indicator and also on the acceleration of the vehicle.

5. A method executed by an information processing device, Storing images of the surroundings of the vehicle; storing a plurality of pieces of information accompanying the image, which can be used as at least one indicator other than acceleration that indicates the behavior of the vehicle, including first information indicating the behavior of the user while riding in the vehicle, second information indicating the presence or absence of abnormal noise from the vehicle, third information indicating the presence or absence of abnormal vibration from the vehicle, and fourth information indicating the distance between the vehicle and an object located around the vehicle; reading at least one piece of information from the plurality of pieces of information to obtain the at least one indicator; and If it is determined that there is an abnormality in the behavior of the vehicle based on the at least one acquired indicator, it is determined to keep the image stored, and if it is determined that there is no abnormality in the behavior of the vehicle, it is determined to delete the image. A method comprising:

6. 6. The method of claim 5, the user behavior of the first information is a vocalization or a body movement of the user, The method further includes determining that there is an abnormality in the behavior of a previous previous user when it is determined that there is an abnormality in the user's vocalization or body movement.

7. 6. The method of claim 5, The user behavior of the first information is that the user utters a first voice that is a calm voice of the user or a second voice that is different from the first voice, The method further includes determining that the user's behavior is abnormal if the user's utterance of the second voice is detected.

8. 8. The method according to any one of claims 5 to 7, The method further includes determining whether there is an abnormality in the behavior of the vehicle based on the acceleration of the vehicle in addition to the at least one indicator.

9. In the information processing device, Storing images of the surroundings of the vehicle; storing a plurality of pieces of information accompanying the image, which can be used as at least one indicator other than acceleration that indicates the behavior of the vehicle, including first information indicating the behavior of the user while riding in the vehicle, second information indicating the presence or absence of abnormal noise from the vehicle, third information indicating the presence or absence of abnormal vibration from the vehicle, and fourth information indicating the distance between the vehicle and an object located around the vehicle; reading at least one piece of information from the plurality of pieces of information to obtain the at least one indicator; and If it is determined that there is an abnormality in the behavior of the vehicle based on the at least one acquired indicator, it is determined to keep the image stored, and if it is determined that there is no abnormality in the behavior of the vehicle, it is determined to delete the image. A program that executes the following.

10. 10. The program according to claim 9, the user behavior of the first information is a vocalization or a body movement of the user, The program further causes the information processing device to determine that there is an abnormality in the behavior of a previous previous user when it is determined that there is an abnormality in the user's vocalization or body movement.

11. 10. The program according to claim 9, The user behavior of the first information is that the user utters a first voice that is a calm voice of the user or a second voice that is different from the first voice, The program further causes the information processing device to determine that there is an abnormality in the behavior of the user when the utterance of the second voice by the user is detected.

12. A program according to any one of claims 9 to 11, The program further causes the information processing device to determine whether or not there is an abnormality in the behavior of the vehicle based on the acceleration of the vehicle in addition to the at least one indicator.

Images