Information processing device, information processing system, and information processing method

Abstract

To provide a safe driving educational material that enables effective learning.SOLUTION: An information processing device according to an embodiment comprises a controller configured to acquire driving video footage captured by a vehicle while a driver of the vehicle is driving, and generate questions in the form of videos using the driving video footage, each question being designed to recognize it as a correct answer when a correct point being displayed in a portion of a video being played back is touched.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing system, and an information processing method. [Background technology]

[0002] Conventionally, there is a technology for setting questions via a touch panel. Patent Document 1 discloses a technology for generating questions about safe driving based on video captured by a drive recorder and distributing them to the driver.

[0003] Furthermore, Patent Document 2 discloses a technique for scoring based on whether or not a point where the display content differs between two moving images being played back on a touch panel is touched. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2023-923254 [Patent Document 2] Japanese Patent Application Laid-Open No. 2005-211287 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the prior art has room for improvement in providing educational materials for safe driving that are highly effective for learning. Specifically, Patent Document 1 uses multiple-choice questions, and drivers can answer the questions without looking at the content of the questions, so it is possible that they may not understand the content. Furthermore, Patent Document 2 is a technology related to games, and does not take into consideration providing educational materials for safe driving that are highly effective for learning.

[0006] The present invention has been made in view of the above, and has as its object to provide an information processing device, an information processing system, and an information processing method that can provide educational materials for safe driving that are highly effective in learning. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the objectives, the information processing device of the present invention has a controller, which acquires driving footage taken by the vehicle while the driver of the vehicle is driving, and generates questions in the form of video using the driving footage, in which the correct answer is given when the correct answer location displayed in a part of the video being played is touched. [Effects of the Invention]

[0008] According to the present invention, questions are provided that require the driver to watch a video being played and touch the correct answer, so that educational materials for safe driving that are highly effective for learning can be provided. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an overview of an information processing system. [Figure 2] FIG. 2 is a diagram illustrating a specific example of a problem according to the embodiment. [Figure 3] FIG. 3 is a block diagram of the in-vehicle device. [Figure 4] FIG. 4 is a block diagram of an information processing device. [Figure 5] FIG. 5 is a diagram illustrating an example of driver information stored in the storage unit. [Figure 6] FIG. 6 is a diagram showing an example of a question pattern. [Figure 7] FIG. 7 is a flowchart showing a processing procedure executed by the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an information processing apparatus, an information processing system, and an information processing method disclosed in the present application will be described with reference to the accompanying drawings. Note that the present invention is not limited to the following embodiments.

[0011] First, an overview of an information processing system according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an overview of the information processing system. As shown in Fig. 1, the information processing system S according to an embodiment is a system that generates and distributes e-learning teaching materials for safe driving instruction and the like. As shown in Fig. 1, the information processing system S includes an in-vehicle device 50, an information processing device 1, and a student terminal 200. The student terminal 200 corresponds to an example of a "terminal device."

[0012] The information processing device 1 analyzes the vehicle situation when a specific event such as an accident or a near miss is detected based on the vehicle data transmitted from the vehicle. The information processing device 1 also generates teaching materials based on the analysis results.

[0013] The in-vehicle device 50 is a video recording device (drive recorder) mounted on a vehicle C. The in-vehicle device 50 according to the embodiment has an in-camera that captures video images inside the vehicle and an out-camera that captures video images outside the vehicle.

[0014] While the vehicle is running, the in-vehicle device 50 records vehicle data, including images of the interior and exterior of the vehicle captured by the in-camera and the out-camera, in a ring buffer memory in an overwritable manner for a certain period of time. The certain period is, for example, 24 hours. The vehicle data may include various data indicating the status of the vehicle, such as date and time information, location information, vehicle speed, and G-force, in addition to the images of the interior and exterior of the vehicle.

[0015] The in-vehicle device 50 is also configured to be able to detect specific events such as the occurrence of an accident or a near miss. The in-vehicle device 50 detects a specific event when, for example, a change in vehicle speed or a change in G-value satisfies a predetermined event condition corresponding to the occurrence of a pre-set accident or a near miss.

[0016] In the information processing system S according to the embodiment, the information processing device 1 acquires vehicle data transmitted from a vehicle, for example, at a predetermined interval (step S1). The vehicle data includes an in-vehicle image, an outside-vehicle image, date and time information, location information, vehicle speed, G value, etc.

[0017] Next, the information processing device 1 analyzes the vehicle data acquired from the vehicle and generates teaching materials according to the analysis results (step S2). For example, the information processing device 1 analyzes the driving tendencies of the driver from in-vehicle video and generates teaching materials according to the driving tendencies.

[0018] Then, the information processing device 1 provides the generated teaching materials to the student terminal 200 (step S3). The student terminal 200 is a terminal device used by a student (driver) taking the e-learning course. The student terminal 200 is realized by a PC (Personal Computer), a smartphone, or the like. For example, the teaching materials are in the form of questions, and the student driver takes the e-learning course by answering the questions provided as the teaching materials.

[0019] However, systems that utilize e-learning like this are required to provide safe driving educational materials that are highly effective for learning. For example, when multiple-choice questions are provided to drivers through e-learning, drivers can answer without looking at the content of the questions, so it is possible that they may not understand the content. Therefore, there is room for improvement in terms of increasing the learning effect.

[0020] Therefore, the information processing device 1 according to the embodiment generates questions that are correct when the correct answer is touched on a part of the video being played. In other words, such questions require the driver to pay close attention to the video, which can improve learning effects.

[0021] Here, a specific example of a problem according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing a specific example of a problem according to the embodiment. Fig. 2 shows a case where a problem is created using a near-miss video from a driving video taken while a driver to whom the problem is to be provided is driving, and the problem is a problem in which the driver touches a location that should be checked in the near-miss video.

[0022] As shown in the first screen G1, when the playback button displayed on the screen is selected, the near-miss video is played once and then the screen transitions to the second screen G2. At this time, the second screen G2 instructs the driver to look back and see what part of the near-miss video they should pay attention to.

[0023] After that, the video is played back from the second screen G2, and for example, the video stops at a point just before the near miss, and the third screen G3 appears. On the third screen G3, a question such as "Please touch the point where the driver should check" is displayed.

[0024] The driver can answer the question by tapping the part of the image displayed on the third screen G3 that corresponds to the question. When the driver has finished answering, the screen transitions from the third screen G3 to the fourth screen G4.

[0025] The fourth screen G4 displays the results of scoring the driver's answers and explanations for the questions, etc. This allows the driver to review the questions.

[0026] In this way, the information processing device 1 according to the embodiment generates questions that become correct when the correct answer point displayed in a part of the video being played is touched, and provides the questions to the driver. As a result, the information processing device 1 according to the embodiment can provide educational materials that are premised on viewing of the video, and therefore can provide educational materials for safe driving that are highly effective for learning.

[0027] Next, a configuration example of the in-vehicle device 50 according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram of the in-vehicle device 50. As shown in Fig. 3, the in-vehicle device 50 is connected to an in-camera 101, an out-camera 102, an in-vehicle sensor 103, and an HMI (Human Machine Interface) unit 110.

[0028] The in-camera 101 is a camera that captures images inside the vehicle, and mainly captures images of the driver while driving. The out-camera 102 is a camera that captures images outside the vehicle, and captures images in front of the vehicle.

[0029] The on-vehicle sensors 103 include a GPS (Global Positioning System) sensor, a G sensor, a vehicle speed sensor, an accelerator sensor, a brake sensor, etc. The on-vehicle sensors 103 are connected to the on-vehicle device 50 via an on-vehicle network such as a CAN (Controller Area Network).

[0030] The HMI unit 110 includes an input interface that accepts input operations from the driver, etc. The input interface is realized by, for example, a touch panel. The input interface may also be realized by, for example, a keyboard, a mouse, a pen tablet, a microphone, etc. The input interface may also be realized by software components. The HMI unit 110 includes an output interface that presents image information and audio information to the driver, etc. The output interface is realized by, for example, a display, a speaker, etc.

[0031] 3, the in-vehicle device 50 has a communication unit 51, a storage unit 52, and a controller 53. The communication unit 51 is realized by a network adapter or the like. The communication unit 51 is connected to a network by wire or wirelessly, and transmits and receives information between the information processing device 1 and the student terminal 200 via the network.

[0032] The storage unit 52 is realized by a storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), or a flash memory. The storage unit 52 stores vehicle data and the like. The vehicle data includes various data indicating the status of the vehicle, such as in-vehicle and out-of-vehicle images, date and time information, location information, vehicle speed, and G-value.

[0033] Next, the controller 53 will be described. The controller 53 corresponds to a so-called processor. The controller 53 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphical Processing Unit), or the like. The controller 53 executes a program according to an embodiment (not shown) stored in the storage unit 52, using RAM as a work area. The controller 53 can also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0034] The controller 53 uploads the vehicle data to the information processing device 1 via the communication unit 51. The vehicle data includes an image of the inside of the vehicle, an image of the outside of the vehicle, date and time information, location information, vehicle speed, G value, etc. For example, the controller 53 uploads the vehicle data to the information processing device 1 at a predetermined period.

[0035] Furthermore, when a specific event occurs, the controller 53 may upload vehicle data for a period before and after the event to the information processing device 1. The specific event includes a case where the sensor value of the G sensor exceeds a threshold, i.e., a case where sudden braking or the like is performed.

[0036] Next, an example of the configuration of the information processing device 1 will be described with reference to Fig. 4. Fig. 4 is a block diagram of the information processing device 1. As shown in Fig. 4, the information processing device 1 includes a communication unit 2, a controller 3, and a storage unit 4.

[0037] The communication unit 2 is realized by a network adapter etc. The communication unit 2 is connected to a network by wire or wirelessly, and transmits and receives information between the in-vehicle device 50 and the student terminal 200 via the network.

[0038] The storage unit 4 is realized by a storage device such as a ROM, a RAM, a flash memory, an HDD (Hard Disk Drive), etc. The storage unit 4 stores vehicle data transmitted from each in-vehicle device 50, driver information about each driver, etc.

[0039] Fig. 5 is a diagram showing an example of driver information stored in the storage unit 4. As shown in Fig. 5, the driver information includes information on items such as "driver ID," "vehicle data," and "driver characteristics."

[0040] The "driver ID" field stores an identifier for identifying each driver. The driver ID may be an identifier for identifying each in-vehicle device 50, or an identifier for identifying each vehicle.

[0041] The "vehicle data" item stores vehicle data transmitted from the in-vehicle device 50 while the driver identified by the corresponding driver ID is driving. For example, newly acquired vehicle data is added to the "vehicle data" item whenever vehicle data is acquired.

[0042] The "driver characteristics" field stores information about the characteristics of the driver identified by the corresponding driver ID. For example, the driver characteristics are determined by analyzing vehicle data.

[0043] The driver characteristics include, for example, the driving area and frequency in which the driver drives, the driver's tendencies while driving, etc. The tendencies while driving include the tendency to not check the surroundings at intersections properly, the tendency to overlook road signs, etc.

[0044] Returning to the explanation of FIG. 4, the controller 53 will be described. The controller 3 corresponds to a so-called processor. The controller 3 is realized by a CPU, an MPU, a GPU, or the like. The controller 3 executes a program according to an embodiment (not shown) stored in the storage unit 4, using RAM as a work area. The controller 3 can also be realized by an integrated circuit such as an ASIC or an FPGA.

[0045] The controller 53 acquires a driving video taken by the vehicle while the driver of the vehicle is driving, and generates a question in the form of a moving image using the driving video, which becomes correct when the correct answer point displayed in a part of the video being played is touched.

[0046] First, the controller 53 estimates the driver's driving characteristics before generating a question. The driver's driving characteristics are estimated by analyzing vehicle data. For example, the controller 53 analyzes the transition of vehicle location information and the time period, thereby identifying the route the driver is driving and the frequency of driving.

[0047] Furthermore, the controller 53 estimates the driver's characteristics based on the in-vehicle video captured by the in-camera 101. For example, the controller 53 extracts the driver's line of sight information from the in-camera 101 and estimates the driver's characteristics using the extracted line of sight information.

[0048] Specifically, the controller 53 inputs the driver's viewpoint information and position information to a model (e.g., AI; Artificial Intelligence) that has undergone machine learning using correct answer data that indicates the point where the driver should look at at each point.

[0049] For example, the model outputs information about the driver's characteristics based on the degree of discrepancy between the correct answer data and the driver's gaze information. For example, if the driver looks at a different point when they should be paying attention to a road sign, the driver's characteristics will be "insufficient confirmation of road signs."

[0050] Also, for example, if the driver looks at a point other than the point to be watched (for example, a pedestrian at a crosswalk) at an intersection, the driver's characteristic will be "insufficient confirmation of the intersection." Note that the controller 53 may estimate the driver's driving characteristic using information other than the line of sight information.

[0051] Next, the controller 53 generates questions according to the driver's characteristics. For a driver whose driver characteristics include insufficient checking of road signs, the controller 53 generates questions about road signs. Furthermore, for a driver whose driver characteristics include insufficient checking of intersections, the controller 53 generates questions about intersections. In this way, the controller 53 can generate questions that are optimized according to the driver's characteristics.

[0052] For example, the controller 53 selects from the storage unit 4 a driving video corresponding to the question to be generated. At this time, the controller 53 generates the question using a driving video taken by the vehicle while the driver who will answer is driving. In this way, the controller 53 may generate the question using a driving video taken by another vehicle on the road on which the driver is driving. In this way, the controller 53 can generate the question using a video that is familiar to the driver, which is expected to increase the driver's motivation to answer the question.

[0053] For example, the controller 53 selects an image from the driving image that satisfies a selection condition. For example, the selection condition for generating a question about road signs is that a road sign is shown in the image, and the selection condition for generating a question about an intersection is that an intersection is shown in the image. Note that when generating a question about an intersection, a condition may be that an oncoming vehicle or a pedestrian is shown in addition to the intersection.

[0054] Next, the controller 53 generates questions based on the selected video. The controller 53 generates questions in one of a plurality of question formats. Here, an example of the question format will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of a question pattern.

[0055] Figure 6 explains two question formats: Pattern A and Pattern B. Pattern A asks the driver to name a sign that appears in the video. For example, in Pattern A, the driver checks the video and answers from the presented options.

[0056] Pattern B is a question pattern that asks the driver to tap on a road sign specified in the video. In pattern B, when the road sign specified in the question appears in the video, the driver answers by tapping the road sign. For example, the controller 53 uses various generative AIs to generate these questions, correct answers to the questions, and explanations for the questions. The controller 53 then distributes the generated questions to the driver's learning device 200.

[0057] Next, a processing procedure executed by the information processing device 1 according to the embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the processing procedure executed by the information processing device 1. The processing procedure shown below is repeatedly executed by the controller 53, for example, at a predetermined cycle.

[0058] 7, the controller 53 first acquires vehicle data from the vehicle (step S101), and then analyzes the acquired vehicle data to estimate the driver characteristics of the driver (step S102).

[0059] Next, the controller 53 determines the educational content based on the estimated driver characteristics (step S103). For example, if the driver characteristic is insufficient checking of road signs, the controller 53 determines the educational content to be educational content related to road signs.

[0060] Next, the controller 53 determines the driving video to be used for the question according to the determined educational content (step S104). For example, if the educational content is related to road signs, the controller 53 selects a video in which the road signs to be quizzed are captured from the driving video captured while the driver was driving.

[0061] Then, the controller 53 generates a question-type teaching material including the selected video (step S105), distributes the generated teaching material to the student terminal 200 of the driver to whom the teaching material is to be distributed (step S106), and ends the process.

[0062] Further advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. [Explanation of symbols]

[0063] 1. Information processing equipment 2. Communications Department 3 Controller 4 Storage section 50 Onboard equipment 51 Communications Department 52 Storage section 53 Controller 101 In-camera 102 Outer Camera 103 In-vehicle sensors 200 student terminals S Information Processing System

Claims

1. a controller; The controller A driving video taken by a vehicle while the driver of the vehicle is driving is acquired, A question is generated in the form of a moving image using the running video, and the correct answer is given when the correct answer point displayed in a part of the video being played is touched. Information processing device.

2. The controller Acquiring vehicle data from the vehicle; Estimating a driver characteristic of the driver based on the vehicle data; Generate the question according to the driver's characteristics The information processing device according to claim 1 .

3. The controller As the vehicle data, information on the driver's line of sight while driving is acquired, The question is generated according to the driver's characteristics estimated from the line of sight information. The information processing device according to claim 2 .

4. The controller determining whether the driver has insufficiently checked road signs based on the line-of-sight information; generating a question about the road sign for the driver who has determined that the driver has not sufficiently checked the road sign; The information processing device according to claim 3 .

5. The controller acquiring the driving images from a plurality of the vehicles; The problem is generated using the driving video captured by another vehicle on the road on which the driver is driving. The information processing device according to claim 1 .

6. an in-vehicle device that is mounted on a vehicle and captures driving video captured while the driver is driving; an information processing device that generates questions to be provided to a driver of the vehicle based on the driving video transmitted from the in-vehicle device; a terminal device that issues the question generated by the information processing device to the driver; Equipped with The information processing device includes: A driving video taken by a vehicle while the driver of the vehicle is driving is acquired, The question is in the form of a moving image using the running video, and a controller is provided for generating a question that becomes correct when a correct answer point displayed in a part of the video being played is touched. Information processing system.

7. An information processing method executed by a controller, an acquisition step of acquiring a driving video taken by the vehicle while the driver of the vehicle is driving; a generating step of generating a question in the form of a moving image using the running video, the question being correct when a correct answer point displayed in a part of the video being played is touched; An information processing method including:

Images