Program, information processing device, and information processing method

Abstract

To suitably support the creation of pre-trained models. [Solution] A program for causing a processor to execute the following: a process for acquiring multiple driving data obtained from multiple vehicles, each containing at least one of the following: information associated with the vehicle, location information of the vehicle, video footage from a camera mounted on the vehicle, and date and time information on when the video was captured; a process for acquiring search conditions related to at least one of the following: information associated with the vehicle, location information of the vehicle, and date and time information on when the video was captured; and a process for searching the multiple driving data based on the search conditions.

Description

Technical Field

[0001] The present invention relates to a program, a model creation support device, and a model creation support method.

Background Art

[0002] Conventionally, in order to automatically control a device, learning data used for learning a machine learning system that obtains a recommendation degree for setting the device to a predetermined setting by inputting at least one piece of state information regarding the device acquired by a state information acquisition unit is accumulated. An invention of a learning data management device is disclosed (Patent Document 1). This device includes a storage unit that stores a set of state information and a predetermined setting as learning data, and when new state information is acquired by the state information acquisition unit, among the learning data stored in the storage unit, from the number of learning data related to the predetermined setting, a confidence level representing the probability of the recommendation degree of the predetermined setting obtained by inputting the new state information into the machine learning system is calculated by a confidence level calculation unit, and when the confidence level satisfies a predetermined criterion indicating that the machine learning system needs to be learned, the set of the new state information and the predetermined setting is stored in the storage unit as learning data, and when the confidence level does not satisfy the predetermined criterion, a data accumulation determination unit that rejects the new state information, and the confidence level calculation unit obtains the confidence level using a function in which the confidence level increases as the recommendation degree increases.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Trained models generated by machine learning systems are used for a variety of applications, including autonomous driving technology that allows moving objects to move autonomously. The training data used to train these models consists of images (videos, still images) captured by cameras. While engineers are necessary to efficiently manage this training data, selecting appropriate training data requires engineers to selectively choose and verify suitable data from a large amount of existing driving data. However, conventional technologies have sometimes been unable to adequately support the creation of trained models.

[0005] This invention has been made in consideration of these circumstances, and one of its objectives is to provide a program, a model creation support device, and a model creation support method that can suitably support the creation of trained models. [Means for solving the problem]

[0006] One aspect of the present invention is a program that causes a processor to perform the following processes: acquiring a plurality of driving data obtained from a plurality of vehicles, each driving data including at least one of the following: information associated with the vehicle, location information of the vehicle, video footage from a camera mounted on the vehicle, and date and time information on when the video was captured; acquiring search conditions related to at least one of the following: information associated with the vehicle, location information of the vehicle, and date and time information on when the video was captured; and searching the plurality of driving data based on the search conditions.

[0007] Another aspect of the present invention is a model creation support device comprising: an acquisition unit that acquires a plurality of driving data obtained from a plurality of vehicles, each driving data including at least one of the following: information associated with the vehicle, location information of the vehicle, video footage from a camera mounted on the vehicle, and date and time information on when the video was captured; a search condition acquisition unit that acquires search conditions related to at least one of the following: information associated with the vehicle, location information of the vehicle, and date and time information on when the video was captured; and a search execution unit that searches the plurality of driving data based on the search conditions.

[0008] Another aspect of the present invention is a process in which a processor acquires a plurality of driving data obtained from a plurality of vehicles, each of which includes at least one of the following: information associated with the vehicle, location information of the vehicle, video footage from a camera mounted on the vehicle, and date and time information on when the video was captured; and a process in which a search condition is obtained related to at least one of the following: information associated with the vehicle, location information of the vehicle, and date and time information on when the video was captured. This is a model creation support method that performs the process of searching for the multiple driving data based on the aforementioned search conditions. [Effects of the Invention]

[0009] According to one aspect of the present invention, the creation of a trained model can be suitably supported. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an example of the configuration and operating environment of a model creation support device. [Figure 2] This is an illustrative diagram of the data included in the driving data. [Figure 3] This is a diagram showing an example of the first search screen. [Figure 4] This figure shows an example of the second search screen. [Figure 5] This figure shows an example of the state transitions on the second search screen. [Figure 6] This figure shows a portion of the second search screen in the first mode, where the magnification has been increased and the magnification is between the first and second thresholds. [Figure 7] This figure shows a portion of the second search screen when the magnification exceeds the first threshold and the system transitions to the second mode. [Figure 8] This figure shows an example of a video playback screen. [Modes for carrying out the invention]

[0011] [Summary] Embodiments of the program, model creation support device, and model creation support method of the present invention will be described below with reference to the drawings. The program is executed by, for example, one or more processors. The model creation support device may be a dedicated system, or it may be a program installed and functioning on a cloud server. The model creation support device is implemented by one or more processors. The model creation support device acquires driving data from multiple vehicles, accepts search operations to a terminal device, and displays the video obtained as a search result on the terminal device. In this way, the program, model creation support device, and model creation support method of the present invention can suitably support the creation of trained models. Note that "one or more processors" may be processors of a computer device separate from the terminal device, or processors built into the terminal device.

[0012] Figure 1 shows an example of the configuration and operating environment of the model creation support device 100. The model creation support device 100 communicates with multiple experimental vehicles (an example of a vehicle) 10 and one or more terminal devices 50 via a network NW. The network NW includes WAN (Wide Area Network), LAN (Local Area Network), the Internet, provider equipment, wireless base stations, etc.

[0013] At least some of the multiple experimental vehicles 10 are vehicles equipped with autonomous driving capabilities. In the following, all experimental vehicles 10 will be assumed to have autonomous driving capabilities. In addition to conventional equipment such as an engine, drive motor, brake system, steering system and their controls (gas pedal (accelerator pedal), brake pedal, steering wheel, shift lever, etc.), the experimental vehicles 10 are equipped with, for example, multiple cameras 12, vehicle sensor group 14, autonomous driving control device 16, positioning device 18, upload device 20, etc.

[0014] Each of the plurality of cameras 12 is disposed at an arbitrary position of the experimental vehicle 10 and images the external space of the experimental vehicle 10 to acquire a video. Each of the plurality of cameras 12 images a plurality of directions, for example, the front, diagonal front, left and right sides, and rear of the experimental vehicle.

[0015] The vehicle sensor group 14 includes, for example, a speed sensor, an acceleration sensor, a gyro sensor, and various sensors attached to an operator (such as a gas pedal sensor, a brake pedal sensor, a steering torque sensor, a shift position sensor, etc.).

[0016] The automatic driving control device 16 acquires information indicating the external environment of the experimental vehicle 10 from the vehicle sensor group 14 or an in-vehicle control device via, for example, a CAN (Controller Area Network) in the vehicle, and autonomously moves the experimental vehicle 10. Autonomously moving includes autonomously performing acceleration, deceleration, turning, etc. The automatic driving control device 16 starts operating when a control switch in the experimental vehicle 10 is operated, and stops operating at least temporarily when there is an intervention operation by the driver. The driving mode of the experimental vehicle 10 under the control of the automatic driving control device 16 is called a control experiment mode, and the driving mode of the experimental vehicle 10 without the control of the automatic driving control device 16, including after an intervention operation, is called a manual driving mode. The driving mode of the experimental vehicle 10 may include other driving modes such as a remote operation mode. The automatic driving control device 16 uses a learned model for at least part of the control decision-making, and the learned model is trained using the video collected in the manual driving mode as learning data.

[0017] An intervention operation refers to an operation on an operating element, including some or all of the following: gas pedal, accelerator pedal, brake pedal, steering wheel, etc. (for example, an operation to instruct acceleration, deceleration, or turning), that meets a predetermined standard. The predetermined standard refers to, for example, a physical quantity such as the amount of operation, the acceleration of operation, or the force of operation becoming greater than or equal to a predetermined value. Greater than or equal to a predetermined value may mean zero or greater, or a value greater than zero. When the amount of operation is used as the standard, the automatic driving control device 16 may determine that "the amount of operation is greater than or equal to the predetermined value" when the position of the steering wheel or gas pedal exceeds a predetermined position. In the case of an operation on the steering wheel, an intervention operation may be determined to have occurred when the left or right steering operation performed by the driver on the steering wheel is input at a predetermined angle or a predetermined angular velocity or greater. In the case of an operation on the gas pedal or brake pedal, an intervention operation may be determined to have occurred when the driver's pressing input is greater than or equal to a predetermined amount, or when the pressing acceleration is greater than or equal to a predetermined acceleration. Furthermore, if the vehicle is equipped with a camera for monitoring the driver, an intervention operation may be determined to have occurred if the driver's facial and body movements included in the video footage acquired by the camera meet predetermined criteria. After the intervention operation, the automatic driving control device 16 may automatically resume operation, or it may wait for the operation of a control switch before resuming operation.

[0018] The positioning device 18 determines the position of the experimental vehicle 10. The positioning device 18 includes, for example, a GNSS receiver and determines the position of the experimental vehicle 10 based on information contained in radio waves arriving from satellites. The positioning device 18 may also determine the position of the experimental vehicle 10 by supplementing information from the wireless base station with which the upload device 20 communicates, or, in environments such as inside a tunnel, it may determine the position of the experimental vehicle 10 based on odometry data (physical quantities such as speed, steering angle, and acceleration detected by the vehicle sensor group 14 in the experimental vehicle 10, and other information).

[0019] The upload device 20 includes, for example, a wireless communication device and a computer device, accesses the network NW, and transmits various data (including position information, video, video capture time, and odometry data) obtained in the experimental vehicle 10 to the model creation support device 100 in association with a common time respectively. When there is the above-described intervention operation, the upload device 20 stores it in the travel data as a flag associated with the time information indicating that there was an intervention operation. The upload device 20 may store the time at the start of the intervention as a flag, or may store the entire period during which the intervention is occurring as the period during which the intervention occurred.

[0020] The terminal device 50 is a computer device including a display device, a communication device, an input / output device, a processor, etc. For example, a general-purpose personal computer, a smartphone, a tablet terminal, etc. may be used as the terminal device 50. In the terminal device 50, a browser or an application program for reproducing the content provided from the model creation support device operates.

[0021] The model creation support device 100 includes, for example, a communication device 110, an acquisition unit 120, a data management unit 130, a search screen display control unit 140, a search execution unit 150, a video display control unit 160, and a storage unit 180. The components other than the storage unit 180 are realized, for example, when a processor such as a CPU (Central Processing Unit) executes a program (software, instruction). Some or all of these components are LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable The program may be implemented by hardware (including circuitry) such as a Gate Array, GPU (Graphics Processing Unit), or SOC (System On Chip), or by the collaboration of software and hardware. The program may be stored in advance in memory or storage device (a storage device with a non-transient storage medium) such as an HDD (Hard Disk Drive) or flash memory, or it may be stored in a removable storage medium (a non-transient storage medium) such as a DVD or CD-ROM and installed when the storage medium is inserted into a drive device.

[0022] The storage unit 180 can be RAM (Random Access Memory), HDD, flash memory, etc. The storage unit 180 stores driving data 182, map information 184, driver profile information 186, etc. The storage unit 180 is not attached to the model creation support device 100, but may be a NAS (Network Attached Storage) that the model creation support device 100 can access via a network. The driver profile information 186 includes information such as the driver's history of driving the experimental vehicle 10 in the past, age, and gender. The driver profile information 186 is provided to the terminal device 50 as reference information for the content described below.

[0023] The communication device 110 communicates with the experimental vehicle 10 and the terminal device 50, respectively, via the network NW.

[0024] The acquisition unit 120 acquires multiple driving data from multiple experimental vehicles 10 via communication using a communication device, and stores each of them as driving data 182 in the storage unit 180. However, the acquisition route for driving data is not limited to this; data stored in a memory device (not shown) in the experimental vehicle 10 may be transferred to a portable memory device, which is then attached to the model creation support device 100 and uploaded.

[0025] The driving data 182 includes at least the location information of the experimental vehicle (the route defined by its changes) and video footage from multiple cameras 12 mounted on the experimental vehicle. The driving data 182 may also include some or all of the following: odometry data, the time when various data including the video capture time were acquired, the identification information of the experimental vehicle (vehicle ID), the identification information of the driver (driver ID), and information that the driver intervened in the autonomous driving function of the experimental vehicle 10. The time when various data were acquired is, for example, the time set by the in-vehicle equipment (control devices such as an ECU (Electronic Control Unit), in-vehicle communication devices such as a CAN driver). The driver's identification information may be the driver's name.

[0026] The data management unit 130 edits the driving data 182 acquired by the acquisition unit 120 into a format suitable for searching. Figure 2 is an illustrative diagram of the data included in the driving data 182. The driving data 182 is stored in the storage unit 180 as time-series information associated with a common time, with labels such as vehicle ID, driver ID, and driving date, and includes video data (divided into 20-second segments, each assigned a scene ID), data indicating the driving mode, odometry data, and data indicating whether or not intervention operations were performed. Each of the labels, such as Video 1, Video 2, Video 3, etc., contains multiple videos corresponding to the number of cameras.

[0027] The search screen display control unit 140 displays a search screen on the terminal device 50 for searching the driving data 182 in, for example, a schedule format or a map format. The search screen includes, for example, a first search screen IM1 in schedule format and a second search screen IM2 in map format. Displaying various screens may mean sending the image itself and displaying it in a browser, etc., or it may mean sending only the screen parameters and text data to the terminal device 50 and having the terminal device 50 perform the rendering. Also, when sending video to the terminal device 50, the model creation support device 100 may have the terminal device 50 download the video chunk by chunk, or it may send the video in a streaming format. The search screen display control unit 140 is an example of a "search condition acquisition unit".

[0028] Figure 3 shows an example of the first search screen IM1. The first search screen IM1 allows users to switch dates, and displays the operating status of the experimental vehicle 10 for each vehicle, driver, and time period on the selected date. Area A1, provided for each vehicle ID, shows the operating status of the driver for each time period. The search screen display control unit 140 searches for multiple driving data 182 using the date specified by the user, extracts the necessary information from the acquired driving data 182, and generates the first search screen IM1. In the figure, D-ID is the driver ID. At least one of the vehicle ID and driver ID is an example of "information associated with the vehicle." What is displayed in area A1 is a single driving history identified by the vehicle, driver, and date. The bar-shaped display object OB1 associated with area A1 (corresponding to the driving history) indicates the driving mode of the experimental vehicle 10 by differences in color, etc. Area A2 shows the time periods when the experimental vehicle 10 moved while capturing video. The gaps in area A2 indicate rest periods, etc. When a search operation is instructed on the first search screen IM1 to specify a driving history (for example, when the area displayed as "A Mountain B Man" is selected with a mouse), the search execution unit 150 performs a search using the specified driving history as the search condition, and the video resulting from the search results is played on the terminal device 50. When the map display button B1 is operated, the system transitions to the second search screen IM2. These search conditions are related to information associated with the experimental vehicle 10 and information about the date and time the video was captured.

[0029] Figure 4 shows an example of the second search screen IM2. In the second search screen IM2, route TJs derived from the location information of each experimental vehicle are displayed on the map using color coding, etc. The search screen display control unit 140 may use map information 184 held in the storage unit 180 of the model creation support device 100, or it may use map information obtained from an external map providing device. The search screen display control unit 140 generates the route of the experimental vehicle 10 by sampling the location information contained in the driving data 182 at predetermined distance intervals. In the second search screen IM2, it is possible to narrow down the results in area A3 by date and time, driving mode, driver ID, vehicle ID, etc. When these narrowing conditions are entered, the search screen display control unit 140 displays only the corresponding route TJs on the terminal device 50. When a search operation by specifying a route (for example, clicking on a route TJ with the mouse) is instructed in the second search screen IM2 (including those shown in Figures 6 and 7), the search execution unit 150 performs a search using the specified route as the search condition, and a video of the search results is played on the terminal device 50. Note that these search criteria are related to the location information of experimental vehicle 10.

[0030] On the second search screen IM2, the user can zoom in or out on the map using their own actions (e.g., mouse, keyboard, touch panel, or voice). At a certain zoom level, the display mode switches between a first mode, where the number of location samples is relatively low (i.e., the resolution is low), and a second mode, where the number of location samples for the experimental vehicle 10 is higher than in the first mode (i.e., the resolution is high). In the second mode, the route is drawn with tens of times more samples than in the first mode. Figure 5 shows an example of the state transitions of the second search screen IM2. When the zoom level is above the first threshold Th1, the second mode is displayed; when it is below the first threshold Th1, the first mode is displayed. In the first mode, if the zoom level is below the second threshold Th2 and multiple routes exist on the same road, the average route over a predetermined period (a route derived from the average value of location information, or a route averaged over multiple routes) is displayed. If the zoom level is above the second threshold Th2 and multiple routes exist on the same road, individual routes (routes for each vehicle, routes for each trip) are displayed. Figure 4 shows how the average route is displayed. By switching the resolution in stages in this way, it is possible to prevent excessive processing load on wide-area maps and to achieve smooth screen transitions. The average value of location information is the average value (point) of the XY coordinates (latitude and longitude) of multiple location information acquired within a predetermined period that are close to each other, and the average route is constructed by connecting these average value coordinates as the endpoints of line segments.

[0031] Figure 6 shows a portion of the second search screen IM2 when the zoom level is increased in the first mode, and the zoom level is between the first threshold Th1 and the second threshold Th2. As shown, when the zoom level increases in the first mode and individual paths are displayed, multiple paths that are not smooth are displayed due to the low resolution. On the other hand, Figure 7 shows a portion of the second search screen IM2 when the zoom level is greater than or equal to the first threshold Th1 and the system has transitioned to the second mode. As shown, in the second mode, the number of samples related to the position of the experimental vehicle 10 increases, so the paths are drawn smoothly. In addition, in the second search screen IM2 in the second mode, a moving lighting effect (a part with higher brightness than other parts of the path) is displayed on the path, and the direction of movement of this lighting effect represents the direction of movement of the experimental vehicle 10, and the speed of the lighting effect represents the speed of the experimental vehicle 10. By displaying the lighting effect in this way, the general state of the experimental vehicle 10 can be conveyed, and the user's video selection can be appropriately supported.

[0032] The search execution unit 150 searches for the relevant driving data 182 from a plurality of driving data 182 based on the content of the search operation on the first search screen IM1 or the second search screen IM2 (based on the search conditions), and notifies the video display control unit 160 of the identification information of the obtained driving data 182.

[0033] The search screen display control unit 140 may acquire search conditions based on the presence or type of intervention operation in either the first search screen IM1, the second search screen IM2, or both, or in a different search screen. The search screen display control unit 140 may accept an operation to specify the presence or type of intervention operation as a filtering condition for the driving data 182, or it may display a screen that lists intervention operations on the terminal device 50 and accept an operation to identify the intervention operation itself. In this case, the search execution unit 150 may acquire driving data 182 that includes not only the period during which the intervention operation was performed, but also the period before and after (driving data 182 from a predetermined time before to a predetermined time after the intervention operation period).

[0034] The video display control unit 160 causes the terminal device 50 to display a video playback screen IM3 that includes the video contained in the driving data 182. Figure 8 shows an example of the video playback screen IM3. The video playback screen IM3 is a screen that displays some or all of the odometry data, video capture time, location information, and driver identification information contained in the driving data 182 together with the video.

[0035] In area A4 of the video playback screen IM3, videos of the experimental vehicle 10 taken from the front, diagonally in front, side, and rear are displayed in parallel and synchronized with the time. The "CONTROL" displayed at the top lights up when the driving mode of the experimental vehicle 10 that captured the video is the control experiment mode. "STEER," "BRAKE," and "ACCEL" light up when steering (turning), braking (deceleration), and acceleration intervention operations are performed, respectively. "CONTROL" turns off when an intervention operation is performed.

[0036] Area A5 of the video playback screen IM3 displays odometry data along with date, time, driver ID, vehicle ID, driving mode, and model (trained model). The information displayed in areas A4 and A5 is synchronized with the time contained in the driving data 182.

[0037] In area A6 of the video playback screen IM3, the changes in speed, acceleration, steering angle, yaw rate, gas pedal opening, and brake pressure over a period of several tens of minutes are displayed in the form of a line graph, along with an indicator ICT that shows the current point in the video playback. The line graph is updated as the action is played back, for example, by moving to the left. Area A6 also functions as a slider bar, allowing the user to change the video playback point by manipulating area A6 left or right with a mouse or other device. This makes it easy for the user to select (seek) a scene in which the experimental vehicle 10 exhibits a desired behavior (such as sudden deceleration or a sharp turn) and easily review the video of that scene. The video playback point can also be changed by operating the control bar CB located at the bottom of the video.

[0038] In area A7 of the video playback screen IM3, the changes in speed, steering angle, acceleration, etc. over a shorter period than in area A6 are displayed in a more magnified view than in area A6. Users can view area A6 and select a scene showing the desired behavior, then view detailed changes in speed, steering angle, acceleration, etc. for that scene in area A7. Area A7 also displays an indicator object OB2 indicating that an intervention operation has occurred. In other words, area A7 indicates whether or not an intervention operation has occurred. The indicator object OB2 may also indicate the type of intervention operation (acceleration, deceleration, steering) using different colors. An indicator ICT showing the current point in the video playback is also displayed in area A7.

[0039] The video display control unit 160 uses the time information included in the driving data 182 to display each part of the video playback screen IM3 described above on the terminal device 50 so that information for the same time is displayed synchronously.

[0040] According to the embodiments described above, the creation of a trained model can be suitably supported. (1) By having the processor perform the following: a process to acquire multiple driving data acquired from multiple vehicles, each including at least vehicle location information and video footage from a camera mounted on the vehicle; a process to display a search screen on a terminal device that allows the user to search for driving data in a schedule format or map format; and a process to display a video playback screen on the terminal device that includes video footage associated with the driving data acquired based on the content of the search operation, driving data that can be used as training data for a trained model can be displayed on the terminal device with intuitive and easy-to-understand operation. (2) By having the processor perform the following: a process to acquire multiple driving data acquired from multiple vehicles, each including at least one of the following: information associated with the vehicle, vehicle location information, video footage from a camera mounted on the vehicle, and date and time information on when the video was captured; a process to acquire search conditions related to at least one of the following: information associated with the vehicle, vehicle location information, and date and time information on when the video was captured; and a process to search the multiple driving data based on the search conditions, the search for driving data that can be used as training data for a trained model can be suitably supported.

[0041] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of symbols]

[0042] 10 Experimental Vehicles 50 Terminal devices 100 Model Creation Support Devices 110 Communication equipment 120 Acquisition Department 130 Data Management Department 140 Search screen display control unit 150 Search Execution Unit 160 Video Display Control Unit 180 Storage section 182 Driving Data 184 Map Information IM1 First Search Screen IM2 Second Search Screen IM3 video playback screen

Claims

1. In the processor, A process for acquiring driving data which includes at least one of the following: information associated with the vehicle, location information of the vehicle, video footage from a camera mounted on the vehicle, and date and time information of when the video was captured. A process for obtaining search conditions related to at least one of the following: information associated with the vehicle, location information of the vehicle, and information on the date and time the video was captured. A process for searching the multiple driving data based on the aforementioned search conditions, A program to execute, At least some of the aforementioned multiple vehicles are vehicles equipped with autonomous driving capabilities. The aforementioned driving data includes information on the driver's intervention actions with respect to the automated driving function. The processor is instructed to obtain search conditions based on the presence or type of the intervention operation in the process of obtaining the search conditions. program.

2. The information associated with the vehicle is the identification information of the vehicle and / or the identification information of the driver. The program according to claim 1.

3. The aforementioned driving data is time-series information to which time information is attached in the vehicle. The program according to claim 1.

4. The aforementioned processor further, The process involves displaying a video playback screen on the terminal device, which includes a video associated with the driving data obtained based on the aforementioned search conditions. The program according to claim 1.

5. The processor, In the process of displaying the aforementioned video playback screen on the terminal device, the presence or absence of driver intervention operations is displayed on the terminal device along with the video. The program according to claim 4.

6. The intervention operation is an operation that instructs the vehicle to accelerate, decelerate, or turn. The aforementioned processor, In the process of displaying the aforementioned video on the terminal device, the terminal device is made to display information indicating the type of intervention operation along with the video. The program according to claim 5.

7. The aforementioned driving data is time-series information to which time information is attached in the vehicle. The aforementioned processor, In the process of displaying the aforementioned video playback screen on the terminal device, the terminal device is made to display multiple types of information representing the vehicle's behavior based on the driving data, synchronized with the time information. The program according to claim 4.

8. In the process of searching for the plurality of driving data in the aforementioned processor, The system will acquire driving data for the period including the period during which the intervention operation was performed. The program according to claim 1.

9. The processor, in the process of acquiring the search conditions, causes the terminal device to display a search screen that allows the user to search for the driving data. The aforementioned search screen includes a first search screen that displays the driving history for each driver or vehicle on the schedule display screen and allows the user to perform a search operation by specifying the driving history. The program according to claim 1.

10. At least some of the aforementioned multiple vehicles are vehicles equipped with autonomous driving capabilities. In the process of causing the processor to display the first search screen on the terminal device, The vehicle's driving mode is displayed on the terminal device in association with the driving history. The program according to claim 9.

11. The processor includes: A process for acquiring driving data which includes at least one of the following: information associated with the vehicle, location information of the vehicle, video footage from a camera mounted on the vehicle, and date and time information of when the video was captured. A process for obtaining search conditions related to at least one of the following: information associated with the vehicle, location information of the vehicle, and information on the date and time the video was captured. A process for searching the multiple driving data based on the aforementioned search conditions, A program to execute, The processor, in the process of acquiring the search conditions, causes the terminal device to display a search screen that allows the user to search for the driving data. The aforementioned search screen includes a second search screen that displays a route on a map and allows the user to perform a search by specifying the route. The second search screen is displayed by switching between one of several modes, including the first mode and the second mode, which has a higher number of location-related samples compared to the first mode. program.

12. The second search screen is a screen that can accept filtering operations based on at least some of the date and time information, vehicle identification information, or driver identification information. In the process of causing the processor to display the second search screen on the terminal device, The terminal device is made to display the route corresponding to the driving data that matches the content of the filtering operation on the map. The program according to claim 11.

13. In the process of causing the processor to display the second search screen on the terminal device, The terminal device is instructed to display the route on the map on the second search screen, color-coding the routes for each vehicle. The program according to claim 11.

14. When the processor causes the terminal device to display the second search screen in the first mode, The route is generated based on the average value of location information acquired within a predetermined period. The program according to claim 11.

15. When the processor causes the terminal device to display the second search screen in the second mode, The terminal device is instructed to display a second search screen on the aforementioned route, in which a lighting effect indicating the movement of the vehicle is displayed along the aforementioned route, the direction of movement of the lighting effect represents the direction of movement of the vehicle, and the speed of the vehicle is represented by the speed of the lighting effect. The program according to claim 11.

16. An acquisition unit that acquires multiple driving data from multiple vehicles, each including at least one of the following: information associated with the vehicle, location information of the vehicle, video footage from a camera mounted on the vehicle, and date and time information of when the video was captured. A search condition acquisition unit acquires search conditions related to at least one of the following: information associated with the vehicle, location information of the vehicle, video footage from a camera mounted on the vehicle, and date and time information of the video footage. A search execution unit that searches for the plurality of driving data based on the search conditions, Equipped with, At least some of the aforementioned multiple vehicles are vehicles equipped with autonomous driving capabilities. The aforementioned driving data includes information on the driver's intervention actions with respect to the automated driving function. The search condition acquisition unit, in the process of acquiring the search conditions, acquires search conditions based on the presence or type of the intervention operation. Information processing device.

17. The processor, A process for acquiring driving data which includes at least one of the following: information associated with the vehicle, location information of the vehicle, video footage from a camera mounted on the vehicle, and date and time information of when the video was captured. A process for obtaining search conditions related to at least one of the following: information associated with the vehicle, location information of the vehicle, video footage from a camera mounted on the vehicle, and date and time information of the video footage. A process for searching the multiple driving data based on the aforementioned search conditions, Execute, At least some of the aforementioned multiple vehicles are vehicles equipped with autonomous driving capabilities. The aforementioned driving data includes information on the driver's intervention actions with respect to the automated driving function. The processor, in the process of obtaining the search conditions, causes the processor to obtain search conditions based on the presence or type of the intervention operation. Information processing methods.

Images