Information processing device, information processing system, information processing method, and program

Abstract

To provide an information processing device, information processing system, information processing method, and program capable of sufficiently utilizing biological information in drive assistance.SOLUTION: An information processing device 10 includes a biological information acquisition section 11 and a moving image information storage section 12. The biological information acquisition section 11 acquires biological information of a crewman of a vehicle. The moving image information storage section 12 determines the length of a moving image based on the biological information of the crewman. The moving image information storage section 12 stores the moving image by the determined length.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing system, an information processing method, and a program. [Background technology]

[0002] Technology is being developed that uses human biometric information to assist driving. For example, Patent Document 1 discloses a driving support information accumulation system that acquires biometric information and improves the reliability of data accumulated as driving support information. In the driving support information accumulation system disclosed in Patent Document 1, when the biometric information exceeds a predetermined threshold, it is determined that a near miss has occurred, and a video of the near miss recorded on a drive recorder is saved. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-095542 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the driving support information accumulation system disclosed in Patent Document 1, if a near-miss is not determined to have occurred based on biometric information, video from the drive recorder is not saved. Therefore, biometric information is not fully utilized for driving support.

[0005] In view of the above-mentioned problems, an object of the present disclosure is to provide an information processing device, an information processing system, an information processing method, and a program that can fully utilize biological information for driving assistance. [Means for solving the problem]

[0006] The information processing device according to the present disclosure includes: A biometric information acquisition unit that acquires biometric information of a vehicle occupant; a video information storage unit that determines a length of a video based on the biometric information of the occupant and stores the video with the determined length; Equipped with.

[0007] The information processing system according to the present disclosure includes: A camera that photographs the faces of vehicle occupants; an information processing device including: a biometric information acquisition unit that acquires biometric information of an occupant based on an image received from the camera; and a video information storage unit that determines a length of a video based on the biometric information of the occupant and stores the video with the determined length; Equipped with.

[0008] The information processing method according to the present disclosure includes: Acquire biometric information of vehicle occupants; determining a length of the moving image based on the biological information of the occupant, and storing the moving image with the determined length; The processing is carried out by a computer.

[0009] The program according to the present disclosure is Acquire biometric information of vehicle occupants; determining a length of the moving image based on the biological information of the occupant, and storing the moving image with the determined length; The processing is executed by a computer. Effect of the Invention

[0010] The present disclosure makes it possible to provide an information processing device, an information processing system, an information processing method, and a program that are capable of fully utilizing biological information for driving assistance. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram illustrating an information processing device according to the present disclosure. [Diagram 2] 4 is a flowchart illustrating information processing according to the present disclosure. [Diagram 3] 11 is a diagram showing heart rates acquired by an information processing device according to the present disclosure. [Figure 4]1 is a schematic diagram showing a moving image stored by an information processing device according to the present disclosure. [Diagram 5] 1 is a schematic diagram showing a moving image stored by an information processing device according to the present disclosure. [Figure 6] 1 is a block diagram illustrating an information processing device according to the present disclosure. [Figure 7] FIG. 1 shows a portion of a commuter route for drivers of each vehicle. [Figure 8] 1 is a block diagram illustrating an information processing device according to the present disclosure. [Figure 9] 4 is a flowchart illustrating information processing according to the present disclosure. [Figure 10] 1 is a block diagram showing a configuration example of an information processing device according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the present disclosure will be described through embodiments, but the disclosure according to the claims is not limited to the following embodiments. In addition, all of the configurations described in the embodiments are not necessarily essential as means for solving the problems. In each drawing, the same elements are given the same reference numerals, and duplicated explanations are omitted as necessary.

[0013] <Embodiment 1> <Information processing device> Hereinafter, the configuration of an information processing device according to the present disclosure will be described with reference to the drawings. Fig. 1 is a block diagram illustrating an information processing device according to the present disclosure. As shown in Fig. 1, an information processing device 10 includes a biometric information acquisition unit 11 and a video information storage unit 12.

[0014] First, the biometric information acquiring unit 11 will be described. The biometric information acquiring unit 11 acquires biometric information of vehicle occupants. The occupants are the driver who drives the vehicle and passengers in the vehicle. In the case of a general household automobile, the occupants are the driver and passengers sitting in the passenger seat or the back seat. In the case of a vehicle used for public transportation such as a bus, the occupants are the driver and passengers sitting in the seats or standing around the seats.

[0015] The biological information includes pulse, heart rate, blood pressure, sweat rate, respiratory rate, brain waves, body temperature, brain waves, and information that can be derived from any one or a combination of these. The biological information includes, for example, the emotions of the occupant, such as tension, anxiety, surprise, anger, etc.

[0016] The biometric information acquiring unit 11 acquires biometric information of the vehicle occupant from, for example, a camera, a wearable device, or a sensor capable of communicating with the information processing device 10. Furthermore, the biometric information acquiring unit 11 may acquire the biometric information of the vehicle occupant via a server capable of communicating with the camera, the wearable device, or the sensor, and the information processing device 10. An example will be described in which the information processing device 10 is capable of communicating with a camera and a server, and the biometric information acquiring unit 11 acquires the biometric information of the vehicle occupant.

[0017] First, the camera captures a facial image of the occupant. Next, the camera transmits the captured facial image to the server. Next, the server analyzes the facial image of the occupant based on the facial features, eye opening degree, and gaze in the facial image. Next, the server calculates biometric information, such as heart rate, heart rate variability, tension level, and concentration level, by analyzing the facial image. Next, the server transmits the biometric information to the biometric information acquisition unit 11. In this manner, the biometric information acquisition unit 11 acquires the biometric information of the vehicle occupant.

[0018] An example will be described in which the biometric information acquiring unit 11 is capable of communicating with a wearable device and acquires biometric information of a vehicle occupant. First, the wearable device measures the pulse, heart rate, and blood pressure of the occupant. Next, the wearable device calculates biometric information, such as heart rate, heart rate variability, tension level, and concentration level, based on the measurement results. Next, the wearable device transmits the biometric information to the biometric information acquiring unit 11. In this manner, the biometric information acquiring unit 11 acquires the biometric information of the vehicle occupant.

[0019] Next, the video information storage unit 12 will be described. The video information storage unit 12 determines the length of the video based on the biometric information of the occupant, and stores the video with the determined length. The video is obtained by a camera such as a drive recorder capturing images of the inside and outside of the vehicle. The video is a video for confirming the surroundings including the occupant. More specifically, the video is a video for later confirming the surroundings including the occupant when a near miss is detected from the biometric information acquired by the biometric information acquisition unit 11 or when a traffic accident occurs, as in the embodiment described later. The length of the video indicates the time from the start time to the end time of the video. The length of the video determined by the video information storage unit 12 can be set arbitrarily according to the biometric information of the occupant.

[0020] For example, assume that the normal setting of the drive recorder is to capture a video for a total of 20 seconds, 10 seconds before and after the impact, when the vehicle detects an impact. When the vehicle detects an impact and the biometric information acquisition unit 11 acquires biometric information indicating that the occupant's heart rate is high, the video information storage unit 12 sets the length of the video captured using the drive recorder to 60 seconds, which is longer than the normal setting. The video information storage unit 12 stores the video for this 60 seconds.

[0021] The video information storage unit 12 may store the video and the biometric information of the occupant in association with each other. The storage destination is a storage unit such as a RAM (Random Access Memory) or a ROM (Read Only Memory) inside or outside the information processing device 10. For example, when the vehicle detects an impact and the biometric information acquisition unit 11 acquires biometric information indicating that the occupant's heart rate is high, the video information storage unit 12 stores 60 seconds of video and the biometric information indicating that the occupant's heart rate is high in association with each other.

[0022] The video information storage unit 12 is not limited to storing a video in association with the biometric information of the occupant when the vehicle detects an impact. Even if the vehicle does not detect an impact, when the biometric information acquisition unit 11 acquires predetermined biometric information, the video information storage unit 12 may determine the length of the video based on the biometric information and store the video in association with the biometric information of the occupant. More specifically, when the biometric information acquisition unit 11 acquires biometric information such as anxiety of the occupant, the video information storage unit 12 determines the length of the video to be, for example, 30 seconds. The video information storage unit 12 stores a 30-second video captured by a camera such as a drive recorder in a storage unit inside or outside the information processing device 10.

[0023] The video information storage unit 12 may be configured to vary the length of the video depending on the type of emotion. Also, the video information storage unit 12 may classify emotions into categories and determine the length of the video depending on which category the emotion acquired by the biometric information acquisition unit 11 belongs to. A more specific description will be given with reference to Table 1. Table 1 shows an example of the length of the video depending on each category.

[0024] [Table 1]

[0025] As shown in Table 1, in category 1, if the occupant is anxious or nervous, the length of the video is 30 seconds. In category 2, if the occupant is surprised, the length of the video is 60 seconds. In category 3, if the occupant is angry, the length of the video is 90 seconds. For example, if the emotion acquired by the biometric information acquisition unit 11 is anger, it belongs to category 3, so the video information storage unit 12 determines the length of the video to be 90 seconds. <Information processing method> Next, an information processing method according to the present disclosure will be described. Fig. 2 is a flowchart showing information processing according to the present disclosure. Each step is performed by the biometric information acquisition unit 11 or the video image information storage unit 12 shown in Fig. 1.

[0026] First, the biometric information acquiring unit 11 acquires biometric information of a vehicle occupant (step ST1). More specifically, the biometric information acquiring unit 11 acquires the biometric information of the vehicle occupant from a communicable camera, a wearable device, or a sensor.

[0027] Next, the video information storage unit 12 determines the length of the video based on the biological information of the occupant (step ST2). For example, when the biological information indicates a high heart rate, the video information storage unit 12 determines the length of the video to be 60 seconds, which is longer than usual.

[0028] Next, the video information storage unit 12 stores the video with the determined length (step ST3). For example, in the case of biometric information indicating a high heart rate, the video information storage unit 12 stores 60 seconds of video in a storage unit inside or outside the information processing unit. In addition, in the case of biometric information indicating a high heart rate, the video information storage unit 12 may store 60 seconds of video in a storage unit inside or outside the information processing unit in association with the biometric information indicating that the occupant has a high heart rate.

[0029] In this manner, in the information processing device 10 according to the present disclosure, the biometric information acquisition unit 11 acquires biometric information, and the video information storage unit 12 determines the length of the video based on the biometric information and stores the video with the determined length. With this configuration, the length of the video stored varies depending on the biometric information, so that the biometric information can be fully utilized for driving assistance.

[0030] <Embodiment 2> <Time Determination> Hereinafter, the configuration of the information processing device according to the present disclosure will be described with reference to the drawings. Each functional block of the information processing device according to the present disclosure is similar to that of the information processing device 10 shown in FIG. 1, and includes a biometric information acquisition unit 11 and a video information storage unit 12.

[0031] Here, it is assumed that the information processing device is capable of communicating with a camera. The camera photographs the inside and outside of the vehicle. The inside of the vehicle includes an occupant, and the camera can photograph the facial images of the occupant. The camera is provided, for example, in the rearview mirror of the vehicle. A plurality of cameras may be provided, and the system may be configured with a camera that photographs the inside of the vehicle and a camera that photographs the outside of the vehicle. The camera may also be a single camera that photographs the inside and outside of the vehicle. Furthermore, the camera may be a camera provided in a mobile terminal device.

[0032] The moving images are obtained by the camera capturing images of the inside and outside of the vehicle. Details of the biometric information acquisition unit 11 are the same as those in FIG. 1, so a detailed description will be omitted, and the moving image information storage unit 12 will be described in detail.

[0033] The video information storage unit 12 determines the length of the video for checking the surroundings including the occupant based on the occupant's biometric information, and further determines the start time and end time of the video. For example, when the biometric information indicates that the occupant has a high heart rate, the video information storage unit 12 sets the length of the video to 60 seconds, which is longer than the normal setting. In this case, the video information storage unit 12 determines, for example, the start time of the video to be 15:00 and the end time of the video to be 15:01.

[0034] A more specific description will be given with reference to FIG. 3. FIG. 3 is a diagram showing heart rates acquired by an information processing device according to the present disclosure. In FIG. 3, the vertical axis indicates heart rates, and the horizontal axis indicates time. In the following, when the acquired heart rate is equal to or greater than S1, the video information storage unit 12 determines the length of the video to be 60 seconds. Here, the description focuses on the time period 15:00 to 15:02, during which the heart rate is equal to or greater than S1. Note that the heart rate S1 is a predetermined value that can be set arbitrarily.

[0035] Of the time period 15:00-15:02 in which the heart rate is equal to or higher than S1, the heart rate increases in the time period 15:00-15:01, reaching a maximum at 15:01. On the other hand, of the time period 15:00-15:02, the heart rate decreases in the time period 15:01-15:02.

[0036] In this case, the video information storage unit 12 determines the start time to be 15:00:30 and the end time to be 15:01:30 so that 15:01, when the heart rate is at its maximum, is the central time of the 60-second video.

[0037] The video information storage unit 12 may store a predetermined length of video once the heart rate S1 is exceeded. The video information storage unit 12 may store video before and after the time (15:00) at which the heart rate S1 is exceeded. In the example shown in FIG. 3, the video before the time at which the heart rate S1 is exceeded is video having a length (time) going back from 15:00. In the example shown in FIG. 3, the video after the time at which the heart rate S1 is exceeded is video having a length (time) that has elapsed since 15:00.

[0038] The video information storage unit 12 can arbitrarily determine the time going back from 15:00 and the time elapsed since 15:00 based on the biological information. More specifically, when storing a 60-second video, the video information storage unit 12 may set the time going back from 15:00 to 20 seconds and the time elapsed since 15:00 to 40 seconds, and store the video from 14:59:40 to 15:00:40. In other words, the video information storage unit 12 may set the length of the video to be the same before and after the time (15:00) when the heart rate S1 is exceeded, or may not set the length of the video to be the same before and after the time (15:00) when the heart rate S1 is exceeded.

[0039] Naturally, the video information storage unit 12 may determine the time going back from 15:00 and the time elapsed from 15:00 according to the heart rate. Also, the video information storage unit 12 may determine the time going back from 15:00 and the time elapsed from 15:00 according to the type of emotion.

[0040] Furthermore, the video information storage unit 12 may determine the start time and end time as follows. In the time period from 15:00 to 15:01, the heart rate is increasing and therefore the video information storage unit 12 determines that the occupant is approaching a dangerous state. On the other hand, in the time period from 15:01 to 15:02, the heart rate is decreasing and therefore the video information storage unit 12 determines that the occupant is moving away from a dangerous state. In order to store videos including times when the occupant is approaching a dangerous state, the video information storage unit 12 may determine the start time of the videos to be 15:00 and 15:01.

[0041] In this way, the video information storage unit 12 can determine the start time and end time from within the time period in which the heart rate is equal to or higher than S1. Note that, although the heart rate has been used as an example here, the video information storage unit 12 can similarly determine the length to be stored based on the biometric information such as tension or concentration. That is, by digitizing the biometric information acquired by the biometric information acquisition unit 11, the video information storage unit 12 can determine the length to be stored according to the numerical value of the biometric information.

[0042] In other words, the video information storage unit 12 may determine the length and time of storage according to the biometric information. For example, assume that the biometric information acquisition unit 11 acquires an emotion such as anxiety as biometric information. In this case, the video information storage unit 12 determines that there is a possibility of approaching a dangerous state, and increases the length of the video to be stored. The video information storage unit 12 may store multiple videos by determining multiple start times and end times. For example, in FIG. 3, the video information storage unit 12 may store two videos, one for a time period from 15:00 to 15:01 and the other for a time period from 15:01 to 15:02.

[0043] On the other hand, when the biometric information acquisition unit 11 acquires a normal heart rate as biometric information, the video information storage unit 12 may shorten the length of the video to be stored. In this way, the video information storage unit 12 determines the length of the video to be stored and the start and end times according to the biometric information, thereby reducing the capacity of the storage unit where the video is stored, and making it possible to store video that is important to the occupant, for example, video of a dangerous state that may lead to an accident.

[0044] In this manner, the video information storage unit 12 stores the video obtained based on the determined start time and end time. With this configuration, the length and time of the video to be stored differ depending on the biometric information, so that the biometric information can be fully utilized for driving assistance.

[0045] Furthermore, the video information storage unit 12 may store a video in the storage unit for each vehicle occupant. With this configuration, the video information storage unit 12 can store a video for each renter, for example, when a vehicle is rented for car sharing or the like by multiple people. Furthermore, since the video is stored for each renter, the video can be shown as evidence to a person who rents a vehicle and engages in dangerous driving such as tailgating, and a penalty can be imposed. Note that the configuration may be such that only the administrator can delete the video data, so that a person who engages in dangerous driving cannot delete the video.

[0046] Here, the process of the video information storage unit 12 when the biometric information of the driver and the passenger among the occupants of the vehicle is different will be described. First, the camera captures the faces of the driver and the passenger of the vehicle. The biometric information acquisition unit 11 acquires the biometric information of both the driver and the passenger of the vehicle. Here, it is assumed that the acquired biometric information indicates that the driver of the vehicle is relaxed and the passenger is anxious.

[0047] In this case, the video information storage unit 12 may determine the length of the video for checking the surroundings including the passenger based on the passenger's biometric information, and further determine the start time and end time of the video. With this configuration, the length and time of the video to be stored can be determined according to the passenger's biometric information. For example, when driving on a highway, the driver of the vehicle may exceed the speed limit without noticing. In this case, the driver is in a relaxed state, but the passenger may feel anxious because the speed is exceeded. Therefore, the video information storage unit 12 determines that there is a possibility of approaching a dangerous state such as a traffic accident, and determines the length of the video for checking the surroundings including the passenger based on the passenger's biometric information, and further determines the start time and end time of the video. In this way, the information processing device can utilize the biometric information of not only the driver but also the passenger for driving assistance.

[0048] <How to save> Here, there are two storage methods for the video information storage unit 12 when storing video with a determined start time and end time. The first method is when the video information storage unit 12 stores video obtained by the camera starting or ending recording of the inside and outside of the vehicle based on the start time and end time. The second method is when the camera is constantly recording video inside and outside the vehicle, and the video information storage unit 12 acquires the video, extracts the video based on the start time and end time, and stores it. Each method will be described in detail.

[0049] First, with reference to FIG. 4, a case will be described in which video information storage unit 12 stores video obtained by a camera starting or ending shooting of the inside and outside of a vehicle based on a start time and an end time.

[0050] Fig. 4 is a schematic diagram showing a video stored by an information processing device according to the present disclosure. The upper part of Fig. 4 is a diagram showing the timing of shooting by the camera, and the lower part of Fig. 4 shows a video stored in memory unit 17 by video information storage unit 12. It is also assumed that video information storage unit 12 has determined the start time of the video to be 15:00 and the end time of the video to be 15:01.

[0051] As shown in the upper part of Fig. 4, the camera does not capture images until 15:00, captures images from 15:00 to 15:01, and does not capture images again after 15:01. That is, as shown in the upper part of Fig. 4, the camera starts at 15:00, the start time of the video, and starts capturing images of the inside and outside of the vehicle. The camera then stops at 15:01, the end time of the video, and capture of the inside and outside of the vehicle ends. In this way, a video 110 for checking the surroundings including the occupants is generated, with a start time of 15:00 and an end time of 15:01.

[0052] As shown in the lower part of FIG. 4, the video information storage unit 12 stores video 110 generated by the start and end of shooting with the camera in the memory unit 17. The video information storage unit 12 may store video 110 generated by the start and end of shooting with the camera in the memory unit 17 in association with biometric information of the occupant. With this configuration, the information processing device can fully utilize the biometric information for driving assistance. Furthermore, the video information storage unit 12 can reduce the amount of data stored in the memory unit 17.

[0053] Furthermore, the video image information storage unit 12 may set the time when the biometric information acquisition unit 11 acquires a predetermined emotion as the start time. The video image information storage unit 12 may set the time when the biometric information acquisition unit 11 no longer acquires a predetermined emotion as the end time. For example, when the biometric information acquisition unit 11 acquires the anger of the occupant, the video image information storage unit 12 sets that timing as the start time. When the anger of the occupant is resolved and the biometric information acquisition unit 11 no longer acquires the anger of the occupant, the video image information storage unit 12 sets that timing as the end time. As another example, when the biometric information acquisition unit 11 acquires a heart rate that exceeds a threshold, the video image information storage unit 12 sets that timing as the start time. When the heart rate of the occupant decreases and the biometric information acquisition unit 11 no longer acquires a heart rate that exceeds the threshold, the video image information storage unit 12 sets that timing as the end time.

[0054] Next, referring to Figure 5, we will explain the case where a camera is constantly taking pictures inside and outside the vehicle, and the video information storage unit 12 acquires the video, extracts the video based on the start time and end time, and stores it.

[0055] FIG. 5 is a schematic diagram showing a video stored by an information processing device according to the present disclosure. The upper part of FIG. 5 is a diagram showing the timing of shooting by a camera. The middle part of FIG. 5 shows a video stored by the video information storage unit 12. The lower part of FIG. 5 shows a video generated by the video information storage unit 12 extracting a video based on a start time and an end time. Here, as in FIG. 4, the storage destination is the memory unit 17. Also, as in FIG. 4, the video information storage unit 12 determines the start time of the video to be 15:00 and the end time of the video to be 15:01.

[0056] As shown in the upper part of Fig. 5, the camera constantly captures images of the inside and outside of the vehicle during all time periods including 15:00 to 15:01. As shown in the middle part of Fig. 5, the video information storage unit 12 stores in the memory unit 17 video images 111 that the camera constantly captures of the inside and outside of the vehicle.

[0057] As shown in the lower part of Fig. 5, the video information storage unit 12 extracts a video 110 for checking the surroundings including the occupant from the video 111 based on the start time and end time. In other words, the video information storage unit 12 cuts and edits the video 111 capturing the occupant based on the start time and end time to separate it into videos 110. The video information storage unit 12 then stores the video 110 in the memory unit 17. The video information storage unit 12 may store the video 110 in the memory unit 17 in association with the biometric information of the occupant. In this way, the information processing device can fully utilize the biometric information for driving assistance.

[0058] 5, video 111 is separated into video 110 based on the start time of 15:00 and the end time of 15:01. However, this is not limited to the above, and if there are multiple start times and end times, video information storage unit 12 can separate video 111 capturing the passengers into multiple videos by cutting and editing the video based on the respective start times and end times.

[0059] Cutting and editing may also be performed when the occupant wishes to check the corresponding video. For example, cutting and editing may be performed by the occupant inputting biometric information, time information of the video, and time stamp information into the information processing device. With this configuration, the information processing device cuts and stores only the video that the occupant wishes to check according to the needs of the occupant, thereby reducing the processing load of the information processing device.

[0060] Here, the information processing device is capable of communicating with a camera, and the information processing device is a so-called ECU (Electronic Control Unit) has been described. However, the present invention is not limited to this, and the information processing device may be provided in a drive recorder. Furthermore, the information processing device may be a server capable of communicating with a vehicle.

[0061] <Embodiment 3> <Identifying near misses> Hereinafter, the configuration of an information processing device according to the present disclosure will be described with reference to the drawings. Fig. 6 is a block diagram illustrating an information processing device according to the present disclosure. As shown in Fig. 6, an information processing device 20 includes a biometric information acquisition unit 21, a moving image information storage unit 22, and a generating unit 23. The biometric information acquisition unit 21 is similar to the biometric information acquisition unit 11 shown in Fig. 1, and therefore a description thereof will be omitted. Here, the moving image information storage unit 22 and the generating unit 23 will be described.

[0062] First, a description will be given of the video information storage unit 22. When a near miss is identified from biometric information, the video information storage unit 22 determines the length of a video for confirming the surroundings including the occupants, and stores the video.

[0063] A near miss is an incident that makes you feel close or makes you startle. When the biometric information acquisition unit 21 acquires a decrease in concentration, tension, anxiety, fear, anger, drowsiness, or fatigue level as biometric information, the video image information storage unit 22 determines that the vehicle occupant is approaching danger and identifies it as a near miss.

[0064] For example, assume that the normal setting of the drive recorder is to capture video images for a total of 20 seconds, 10 seconds before and after the impact when the vehicle detects an impact. When the video image information storage unit 22 identifies a near miss, it sets the length of the video images captured by the drive recorder to 60 seconds, which is longer than the normal setting.

[0065] Furthermore, when a near miss is identified, the video information storage unit 22 may further evaluate the near miss in stages and determine the length of the video in accordance with the evaluation. A more specific description will be given with reference to Table 2. Table 2 is a table showing the results of the staged evaluation of near misses. In Table 2, near misses are evaluated on a three-level scale from 1 to 3, with the higher the value indicating the higher the possibility of a near miss. Furthermore, in Table 2, a circle is indicated when the biometric information acquisition unit 21 has acquired biometric information, and an x ​​is indicated when the biometric information acquisition unit 11 cannot acquire biometric information.

[0066] [Table 2]

[0067] As shown in Table 2, when the biometric information acquisition unit 21 acquires only drowsiness, the video image information storage unit 22 assigns the near miss a rating of 1. When the biometric information acquisition unit 21 acquires anxiety and fatigue level, the video image information storage unit 22 assigns the near miss a rating of 2. When the biometric information acquisition unit 21 acquires anxiety, fatigue level, and tension, the video image information storage unit 22 assigns the near miss a rating of 3. The video image information storage unit 22 may evaluate the near miss in stages according to the number of items of biometric information acquired by the biometric information acquisition unit 21. The video image information storage unit 22 evaluates that the greater the number of items of biometric information acquired by the biometric information acquisition unit 21, the higher the possibility of a near miss.

[0068] As shown in Table 2, when the near miss is 1, the biometric information acquisition unit 21 determines the length of the video to be 60 seconds. When the near miss is 2, the biometric information acquisition unit 21 determines the length of the video to be 80 seconds. When the near miss is 3, the biometric information acquisition unit 21 determines the length of the video to be 100 seconds. In this way, the video information storage unit 22 determines an appropriate length of the video depending on the possibility of a near miss based on the biometric information. Therefore, the information processing device 20 can fully utilize the biometric information for driving assistance. Furthermore, the information processing device 20 can reduce the load on the memory unit, which is the storage destination.

[0069] In Table 2, an example has been described in which, when the video information storage unit 22 identifies a near miss, the near miss is evaluated in stages and the length of the video is determined according to the evaluation. However, this is not limited to this, and when the video information storage unit 22 identifies a near miss, the near miss may be evaluated in stages and the start time and end time of the video may be determined according to the evaluation.

[0070] Furthermore, the video image information storage unit 22 may change the criteria for identifying near misses depending on the attributes of the occupants, including age, sex, and driving habit. Driving habit includes driving frequency. For example, in Table 2, the video image information storage unit 22 may set the near miss to 1 when the biometric information acquisition unit 11 acquires only drowsiness from a young driver, and may set the near miss to 2 when the biometric information acquisition unit 11 acquires only drowsiness from an elderly driver.

[0071] As another example, in Table 2, when the monthly or annual driving frequency falls below a standard, the video image information storage unit 22 may set the near miss to 2 even if the biological information acquisition unit 11 acquires only drowsiness. The attributes of the occupant, including age, sex, and driving habit, are obtained, for example, by analyzing an image captured by a camera capable of communicating with an information processing device. More specifically, the video image information storage unit 22 analyzes the driver's driving habit by analyzing the driver's accelerator, brake, and steering operations from the image captured by the camera. The process of analyzing the driver's driving habit may be executed by the camera, the information processing device, and a server capable of communication, and the server may be configured to transmit the analysis result to the information processing device. The information processing device may also acquire information regarding the driving frequency from the vehicle.

[0072] <Statistical data> Next, the generating unit 23 will be described. The generating unit 23 generates statistical data including at least one of the location of the near miss, the date and time of the near miss, the traffic conditions at the time of the near miss, the weather at the time of the near miss, the vehicle operation, and the vehicle speed, related to the occupant, from the video image. The location of the near miss includes not only the position on the map but also the shape of the road, for example, a T-junction, a steep slope, and a steep angle. The date and time of the near miss includes the date, time, and time period. The traffic conditions at the time of the near miss include traffic conditions such as congestion and road closure at or near the location of the near miss. The weather at the time of the near miss includes the weather, temperature, humidity, and degree of penetration at or near the location of the near miss. The vehicle operation is turning left, going straight, parking, and the like. In the statistical data, the location of the near miss, the date and time of the near miss, the vehicle operation, and the speed may be combined to form the detailed situation of the near miss.

[0073] An example of generating statistical data by the generating unit 23 will be described with reference to FIG. 7. FIG. 7 is a diagram showing a part of the homeward journey of a driver who drives a vehicle to work. FIG. 7 describes a case in which the generating unit 23 generates statistical data including the location of a near-miss incident of an occupant, the time of the incident, and the weather at the time of the incident from a video. In FIG. 7, the information processing device 30 is mounted on the vehicle C1. The information processing device 40 is mounted on the vehicle C2. An example will be described below in which the information processing devices 30, 40 associate the video with the biometric information of the occupant and store the video.

[0074] In FIG. 7, vehicle C1 travels on road R1, turns left at T-junction T1, merges with road R2, and travels on road R2. Vehicle C2 travels on road R2. The timing when vehicle C1 turns left at T-junction T1 and the timing when vehicle C2 passes through T-junction T1 periodically overlap, and in FIG. 7, the timing is 5 p.m. In addition, in FIG. 7, it is assumed that the weather is clear, the sun is setting and the afternoon sun is shining in at T-junction T1, and the driver of vehicle C1 has difficulty seeing vehicle C2 due to the afternoon sun when turning left. In addition, it is assumed that the driver of vehicle C2 has difficulty seeing vehicle C1 merging from road R1 due to the afternoon sun when passing through T-junction T1.

[0075] When the driver of vehicle C1 turns left at T-junction T1 at 5 p.m., the setting sun makes it difficult for him to see vehicle C2. Therefore, the information processing device 30 acquires biological information such as tension and anxiety. The information processing device 30 then identifies a near miss from the biological information. The information processing device 30 determines the length of the video and associates the video with the biological information of the driver of vehicle C1 and stores it.

[0076] The generation unit 23 of the information processing device 30 generates statistical data including the location of the near miss, the time of the near miss, and the weather at the time of the near miss from the stored video image, by linking the data to the driver of the vehicle C1. In other words, the generation unit 23 of the information processing device 30 generates statistical data such as the T-junction T1 as the location of the near miss, 5 p.m. as the time of the near miss, and a sunny day with the setting sun shining in as the weather at the time of the near miss, by linking the data to the driver of the vehicle C1. The generation unit of the information processing device 30 may also generate statistical data further including a situation such as turning left at the T-junction T1 at 5 p.m. as a detailed situation of the near miss, by linking the data to the driver of the vehicle C1.

[0077] On the other hand, when vehicle C2 passes through T-junction T1 at 17:00, the setting sun makes it difficult to see vehicle C1. Therefore, the information processing device 40 acquires biological information such as tension and anxiety. Then, the information processing device 40 identifies it as a near miss from the biological information. The information processing device 40 determines the length of the video and associates the video with the biological information of the driver of vehicle C2 and stores it.

[0078] The generation unit 23 of the information processing device 40 generates statistical data including the location of the near miss, the time of the near miss, and the weather at the time of the near miss from the information of the stored video image, by linking it to the driver of the vehicle C2. In other words, the generation unit 23 of the information processing device 40 generates statistical data such as the T-junction T1 as the location of the near miss, 5 p.m. as the time of the near miss, and a sunny day with the setting sun shining in as the weather at the time of the near miss, by linking it to the driver of the vehicle C2. The generation unit of the information processing device 40 may also generate statistical data further including the situation when passing through the T-junction T1 at 5 p.m. as the detailed situation of the near miss, by linking it to the driver of the vehicle C2.

[0079] In this way, the information processing device 30 generates statistical data related to the driver of the vehicle C1 from the video, including the location of the near miss, the time of occurrence, and weather. Also, the information processing device 40 generates statistical data related to the driver of the vehicle C2 from the video, including the location of the near miss, the time of occurrence, and weather. In other words, the information processing devices 30 and 40 generate statistical data related to the occupants. This allows the driver to avoid near misses by referring to the statistical data when driving the same road regularly. In other words, it is possible to fully utilize biological information for driving assistance.

[0080] Here, the weather at the time of the near miss occurrence is acquired by a camera capable of communicating with the information processing device 20. A case where the weather at the time of the near miss occurrence is acquired by a camera capable of communicating with the information processing device 20 will be described.

[0081] The camera captures video images of the inside and outside of the vehicle, including the occupants. The camera transmits the video images to the information processing device 30. The generation unit 23 identifies the weather at the time of occurrence by analyzing the acquired video images. As another example, the weather at the time of occurrence of the near miss may be acquired by an environmental sensor capable of communicating with the information processing device. The environmental sensor acquires weather information on the travel route of the vehicle. The environmental sensor is, for example, a weather sensor, an illuminance sensor, a wind vane, an anemometer, a thermometer, and a hygrometer. The weather at the time of occurrence of the near miss may also be acquired by a camera and an environmental sensor capable of communicating with the information processing device. With such a configuration, the information processing device 30 can acquire more accurate weather information from the camera and the environmental sensor.

[0082] In FIG. 7, the generating unit 23 generates statistical data including the location of near miss occurrence, the time of occurrence, and the weather at the time of occurrence for each driver of the vehicles C1 and C2 from the video image. However, the present invention is not limited to this, and the generating unit 23 may generate statistical data including the location of near miss occurrence, the time of occurrence, and the weather at the time of occurrence, and store the statistical data as driving support information in a database that can communicate with the information processing device. With this configuration, the statistical data is accumulated, so that the driver can avoid near misses by referring to the statistical data. Furthermore, the road management company can repair the road so as to avoid near misses by referring to the statistical data.

[0083] <Embodiment 4> <Operation mode switching> Hereinafter, the configuration of the information processing device according to the present disclosure will be described with reference to the drawings. Fig. 8 is a block diagram illustrating an information processing device according to the present disclosure. As shown in Fig. 8, an information processing device 50 includes a biometric information acquisition unit 51, a video image information storage unit 52, and an operation mode switching unit 53. The biometric information acquisition unit 51 and the video image information storage unit 52 are similar to the biometric information acquisition units 11 and 21 and the video image information storage units 12 and 22 shown in Figs. 1 and 6, respectively, and therefore will not be described. Here, the operation mode switching unit 53 will be described.

[0084] In the following, the vehicle has a driving mode that can be switched between an automatic driving mode and a manual driving mode. Of course, in the automatic driving mode, the vehicle is controlled to run without the driver operating the steering wheel, brakes, or accelerator. In the manual driving mode, the driver operates the steering wheel, brakes, and accelerator to run the vehicle.

[0085] The driving mode switching unit 53 switches between the automatic driving mode and the manual driving mode of the vehicle based on the biometric information of the occupant acquired by the biometric information acquisition unit 51. More specifically, when the driving mode switching unit 53 identifies a near miss from the biometric information of the occupant, it switches between the automatic driving mode and the manual driving mode of the vehicle. When switching between the automatic driving mode and the manual driving mode of the vehicle, the driving mode switching unit 53 may notify the driver of the vehicle of the intention to switch the driving mode, and the driver of the vehicle may respond to the notification to switch the driving mode. The notification method is a notification by displaying on the car navigation system of the vehicle, or a voice notification by a speaker.

[0086] For example, when the biological information acquisition unit 51 acquires drowsiness of the occupant, the driving mode switching unit 53 identifies a near miss from the biological information of the occupant. The driving mode switching unit 53 switches the manual driving mode of the vehicle to the automatic driving mode.

[0087] In addition, the driving mode switching unit 53 may change the criteria for identifying a near miss depending on the attributes of the occupant, including age, gender, and driving habit. For example, when an older driver does not feel drowsy but is in a tense state, the driving mode switching unit 53 may identify it as a near miss. On the other hand, when a younger driver does not feel drowsy but is in a tense state, the driving mode switching unit 53 may identify it as not being a near miss. The attributes of the occupant, including age, gender, and driving habit, may be obtained, for example, by analyzing a video image of the driver's driving state captured by a camera capable of communicating with the information processing device 50.

[0088] In this way, when a near miss is identified from the biometric information of an occupant, the information processing device according to the present disclosure switches between the automatic driving mode and the manual driving mode of the vehicle. Also, when a near miss is identified from the biometric information, the information processing device according to the present disclosure determines the length, start time, and end time of a video for checking the surroundings including the occupant, and stores the video in association with the biometric information of the occupant.

[0089] With this configuration, the information processing device according to the present disclosure can display a moving image to the occupant when switching to autonomous driving, so that the occupant of the vehicle can understand what caused the switch to autonomous driving.

[0090] <Information processing method> Next, an information processing method according to the present disclosure will be described. Fig. 9 is a flowchart showing information processing according to the present disclosure. The subject of each step is either the biometric information acquisition unit 51, the video image information storage unit 52, or the driving mode switching unit 53 shown in Fig. 9. Also, step ST1 is the same as step ST1 shown in Fig. 2, so its description will be omitted. Here, steps ST21 and after will be described.

[0091] After the biometric information acquisition unit 51 acquires the biometric information of the vehicle occupant (step ST1), the video information storage unit 52 determines the length of the video for checking the surroundings including the occupant, which is obtained by photographing the inside and outside of the vehicle, based on the biometric information of the occupant (step ST21). More specifically, when a near miss is identified from the biometric information, the video information storage unit 52 determines the length of the video for checking the surroundings including the occupant. Next, the video information storage unit 52 stores the video with the determined length (step ST31). The video information storage unit 52 may store the video in association with the biometric information of the occupant.

[0092] Next, the driving mode switching unit 53 switches between the automatic driving mode and the manual driving mode of the vehicle based on the biometric information of the occupant acquired in step ST1 (step ST41). More specifically, when the driving mode switching unit 53 identifies a near miss from the biometric information of the occupant, it switches between the automatic driving mode and the manual driving mode of the vehicle. As a result, in the information processing method according to the present disclosure, when switching to automatic driving, a moving image can also be shown to the occupant. Therefore, the occupant of the vehicle can understand what caused the switching to automatic driving.

[0093] In step ST21 in Fig. 9, the video information storage unit 52 determines the length of the video based on the biometric information of the occupant. However, this is not limited to this, and in step ST21 in Fig. 9, the video information storage unit 52 may determine the length of the video and the start time and end time of the video based on the biometric information of the occupant.

[0094] FIG. 10 is a block diagram showing a configuration example of an information processing device according to the present disclosure. FIG. 10 is a block diagram showing a configuration example of the above-mentioned information processing devices 10, 20, and 50 (hereinafter referred to as information processing devices 10, etc.). Referring to FIG. 10, the information processing device 10, etc. includes a network interface 1201, a processor 1202, and a memory 1203. The network interface 1201 may be used to communicate with a network node. The network interface 1201 may include, for example, a network interface card (NIC) that complies with the IEEE 802.3 series. IEEE stands for Institute of Electrical and Electronics Engineers.

[0095] The processor 1202 reads out and executes software (computer programs) from the memory 1203 to perform the processing of the information processing device 10 and the like described using the flowcharts in the above-mentioned embodiments. The processor 1202 may be, for example, a microprocessor, an MPU, or a CPU. The processor 1202 may include multiple processors.

[0096] The memory 1203 is configured by a combination of volatile memory and non-volatile memory. The memory 1203 may include storage located away from the processor 1202. In this case, the processor 1202 may access the memory 1203 via an I / O (Input / Output) interface (not shown).

[0097] 10, the memory 1203 is used to store a group of software modules. The processor 1202 reads out and executes the group of software modules from the memory 1203, thereby performing the processing of the information processing device 10 and the like described in the above-mentioned embodiment.

[0098] As described with reference to FIG. 10, each of the processors included in the information processing device 10 or the like executes one or more programs including a group of instructions for causing a computer to execute the algorithm described with reference to the drawings.

[0099] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, the transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0100] While the present disclosure has been described above in accordance with the above-described embodiments, the present disclosure is not limited to the configurations of the above-described embodiments, and naturally includes various modifications, alterations, and combinations that may be made by a person skilled in the art within the scope of the claims of the present application.

[0101] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but to one or more other embodiments. As can be understood by those skilled in the art, various features or steps described with reference to any one drawing may be combined with features or steps shown in one or more other drawings to create an embodiment not explicitly shown or described, for example. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0102] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes. (Appendix 1) A biometric information acquisition unit that acquires biometric information of a vehicle occupant; a video information storage unit that determines a length of a video based on the biometric information of the occupant and stores the video with the determined length; An information processing device comprising: (Appendix 2) The video information storage unit determines a start time and an end time of the video based on biometric information of the occupant. 2. An information processing device according to claim 1. (Appendix 3) The moving image is Based on a start time of the moving image, an image of the surroundings including the occupant is started to be captured; a moving image generated by ending shooting of the surroundings including the occupant based on an end time of the moving image; 3. An information processing device according to claim 2. (Appendix 4) The video information storage unit includes: extracting a video for confirming the surroundings including the occupant from the video based on the start time and the end time, and storing the extracted video. 3. An information processing device according to claim 2. (Appendix 5) The video information storage unit, when identifying a near miss from the biometric information, determines a length of the video and stores the video with the determined length. 3. The information processing device according to claim 1 or 2. (Appendix 6) A generating unit that generates statistical data regarding the occupant from the video, including the location of the near miss, the time of the near miss, the traffic conditions at the time of the near miss, the weather at the time of the near miss, the vehicle operation, and the speed, 6. The information processing device according to claim 5. (Appendix 7) a generating unit that generates statistical data including at least one of the location where the near miss occurred, the date and time of the near miss, the traffic conditions at the time of the near miss, the weather at the time of the near miss, the vehicle operation, and the vehicle speed from the video image; The generation unit stores the statistical data in a communicable database as driving assistance information. 6. The information processing device according to claim 5. (Appendix 8) The biometric information acquisition unit acquires biometric information based on an image of the face of the occupant. 3. The information processing device according to claim 1 or 2. (Appendix 9) The vehicle has a driving mode that can be switched between an automatic driving mode and a manual driving mode, An operation mode switching unit that switches the operation mode based on the biological information, 3. The information processing device according to claim 1 or 2. (Appendix 10) The operation mode switching unit switches the operation mode when a near miss is identified from the biological information. 10. The information processing device according to claim 9. (Appendix 11) The video information cannot be deleted by the occupant. 3. The information processing device according to claim 1 or 2. (Appendix 12) A camera that photographs the faces of vehicle occupants; an information processing device including: a biometric information acquisition unit that acquires biometric information of an occupant based on an image received from the camera; and a video information storage unit that determines a length of a video based on the biometric information of the occupant and stores the video with the determined length; An information processing system comprising: (Appendix 13) Acquire biometric information of vehicle occupants; determining a length of the moving image based on the biological information of the occupant, and storing the moving image with the determined length; The processing is carried out by a computer. Information processing methods. (Appendix 14) Acquire biometric information of vehicle occupants; determining a length of the moving image based on the biological information of the occupant, and storing the moving image with the determined length; Have a computer carry out the process, program.

[0103] Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 2 to 11 that are dependent on Supplementary Note 1 may also be dependent on Supplementary Notes 12 to 14 in the same dependent relationship as Supplementary Notes 2 to 11. Some or all of the elements described in any Supplementary Note may be applied to various hardware, software, recording means for recording software, systems, and methods. [Explanation of symbols]

[0104] 10, 20, 30, 40, 50 Information processing device 11, 21, 51 Biometric information acquisition unit 12, 22, 52 Video information storage unit 17 Memory section 23 Generation part 53 Operation mode switching section 110, 111 Video 1201 Network Interface 1202 Processor 1203 Memory C1, C2 vehicles R1, R2 road S1 heart beats T1 T-junction

Claims

1. A biometric information acquisition unit that acquires biometric information of a vehicle occupant; a video information storage unit that determines a length of a video based on the biometric information of the occupant and stores the video with the determined length; An information processing device comprising:

2. The video information storage unit determines a start time and an end time of the video based on biometric information of the occupant. The information processing device according to claim 1 .

3. The moving image is Based on a start time of the moving image, an image of the surroundings including the occupant is started to be captured; a moving image generated by ending shooting of the surroundings including the occupant based on an end time of the moving image; The information processing device according to claim 2 .

4. The video information storage unit includes: extracting a video for confirming the surroundings including the occupant from the video based on the start time and the end time, and storing the extracted video. The information processing device according to claim 2 .

5. The video information storage unit, when identifying a near miss from the biometric information, determines a length of the video and stores the video with the determined length.

3. The information processing device according to claim 1 or 2.

6. A generating unit that generates statistical data including at least one of the location of the near miss, the date and time of the near miss, the traffic conditions at the time of the near miss, the weather at the time of the near miss, the vehicle operation, and the vehicle speed from the video image, The information processing device according to claim 5 .

7. The biometric information acquisition unit acquires biometric information based on an image of the face of the occupant.

3. The information processing device according to claim 1 or 2.

8. A camera that photographs the faces of vehicle occupants; an information processing device including: a biometric information acquisition unit that acquires biometric information of an occupant based on an image received from the camera; and a video information storage unit that determines a length of a video based on the biometric information of the occupant and stores the video with the determined length; An information processing system comprising:

9. Acquire biometric information of vehicle occupants; determining a length of the moving image based on the biological information of the occupant, and storing the moving image with the determined length; The processing is carried out by a computer. Information processing methods.

10. Acquire biometric information of vehicle occupants; determining a length of the moving image based on the biological information of the occupant, and storing the moving image with the determined length; Have a computer carry out the process, program.

Images