Information processing device, information processing method, and program

Abstract

To provide technology for evaluating the appropriateness of driving from predetermined perspectives. [Solution] An information processing device according to one embodiment includes: an acquisition unit that acquires information representing the situation of a target; a text generation unit that generates a first text describing the situation in natural language based on the information; an evaluation unit that evaluates the appropriateness of the situation represented by the first text based on criteria for evaluating the appropriateness of the situation from a predetermined viewpoint; and a notification unit that notifies a predetermined recipient of the result of the evaluation.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In recent years, various technologies have been proposed as technologies for realizing autonomous driving. For example, in Patent Document 1, a technology has been proposed that can perform highly safe driving in consideration of precautions generated based on laws and regulations and case laws. Further, for example, in Non-Patent Document 1, a technology has been proposed that can generate an appropriate route plan by detailing the driving environment obtained from an image in natural language.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, there is no technology that can evaluate the appropriateness of driving (including autonomous driving) from a predetermined perspective.

[0006] This disclosure is made in view of the above points and aims to provide a technology for evaluating the appropriateness of driving from predetermined perspectives. [Means for solving the problem]

[0007] An information processing device according to one aspect of the present disclosure includes: an acquisition unit that acquires information representing a target situation; a text generation unit that generates a first text describing the situation in natural language based on the information; an evaluation unit that evaluates the appropriateness of the situation represented by the first text based on criteria for evaluating the appropriateness of the situation from a predetermined viewpoint; and a notification unit that notifies a predetermined recipient of the results of the evaluation. [Effects of the Invention]

[0008] The appropriateness of driving can be evaluated from predetermined criteria. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of the hardware configuration of the operation evaluation device according to this embodiment. [Figure 2] This figure shows an example of the functional configuration of the operation evaluation device according to this embodiment. [Figure 3] This flowchart shows an example of the operation of the operational evaluation device according to this embodiment. [Figure 4] This is a diagram showing an example of the first text. [Figure 5] This is a diagram showing an example of the second text. [Figure 6] This figure shows an example of the results of comparing the meaning of the first text and the second text. [Modes for carrying out the invention]

[0010] Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings. In the following embodiment, a driving evaluation device 10 that can evaluate the appropriateness of driving (including automated driving) from predetermined viewpoints will be described. The driving evaluation device 10 is implemented by various information processing devices (computers), such as a PC (personal computer) or a general-purpose server.

[0011] Typical examples of vehicles that can be driven include four-wheeled vehicles such as passenger cars and trucks. However, vehicles are not limited to four-wheeled vehicles; they may also include, for example, two-wheeled vehicles such as motorcycles, material handling vehicles such as forklifts, construction machinery such as hydraulic excavators and bulldozers, aircraft such as drones, ships, and robots that operate and move autonomously.

[0012] Hereafter, to distinguish between automated driving and driving by a driver, driving by a driver will be referred to as "manual driving." Note that manual driving may include driving in which part of the operation is automated (e.g., driving using driver assistance functions).

[0013] <Example of hardware configuration for the operation evaluation device 10> Figure 1 shows an example of the hardware configuration of the operation evaluation device 10 according to this embodiment. As shown in Figure 1, the operation evaluation device 10 according to this embodiment includes an input device 101, a display device 102, an external I / F 103, a communication I / F 104, a RAM (Random Access Memory) 105, a ROM (Read Only Memory) 106, an auxiliary storage device 107, and a processor 108. Each of these hardware components is connected to each other via a bus 109 so as to be able to communicate.

[0014] The input device 101 is, for example, a keyboard, mouse, touch panel, or physical button. The display device 102 is, for example, a display or display panel. Note that the operation evaluation device 10 does not necessarily have to have at least one of the input device 101 and the display device 102.

[0015] The external I / F 103 is an interface with an external device such as a recording medium 103a. Examples of the recording medium 103a include a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), a USB (Universal Serial Bus) memory card, and the like.

[0016] The communication I / F 104 is an interface for connecting to a communication network. The RAM 105 is a volatile semiconductor memory (storage device) that temporarily holds programs and data. The ROM 106 is a non-volatile semiconductor memory (storage device) that can hold programs and data even when the power is turned off. The auxiliary storage device 107 is a non-volatile storage device such as, for example, a HDD (Hard Disk Drive), a SSD (Solid State Drive), or a flash memory. The processor 108 is an arithmetic device such as, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).

[0017] Note that the hardware configuration shown in FIG. 1 is an example, and the hardware configuration of the driving evaluation device 10 is not limited to this. For example, the driving evaluation device 10 may have a plurality of auxiliary storage devices 107 and a plurality of processors 108, may not have some of the illustrated hardware, or may have various hardware other than the illustrated hardware.

[0018] <Example of the functional configuration of the driving evaluation device 10> FIG. 2 is a diagram showing an example of the functional configuration of the driving evaluation device 10 according to the present embodiment. As shown in FIG. 2, the driving evaluation device 10 according to the present embodiment includes an acquisition unit 201, a text generation unit 202, a comparison unit 203, and a notification unit 204. Each of these units is realized, for example, by a process in which one or more programs installed in the driving evaluation device 10 are executed by a processor 108 or the like.

[0019] The acquisition unit 201 acquires the driving situation information 300 provided to the driving evaluation device 10. Note that the acquisition unit 201 acquires the driving situation information 300 each time the driving situation information 300 is provided to the driving evaluation device 10.

[0020] The driving situation information 300 is information representing the past, current, or future driving situation of the vehicle. When evaluating manual driving, specific examples of the driving situation information 300 include videos captured by the driving recorder of the vehicle that the user is currently driving or has driven in the past. In addition, other specific examples of the driving situation information 300 when evaluating manual driving include the accelerator opening, steering wheel cut angle and direction, and brake depression amount of the vehicle at each time. On the other hand, when evaluating autonomous driving, specific examples of the driving situation information 300 include simulation videos generated by the entity that controls the autonomous driving of the vehicle (hereinafter also referred to as the "autonomous driving control engine"). In addition, other specific examples of the driving situation information 300 when evaluating autonomous driving include the accelerator opening, steering wheel cut angle and direction, and brake depression amount of the vehicle at each time in the simulation video. The simulation video is, for example, a video generated by simulating the driving of the vehicle from a certain time t (e.g., the current time t) to t+Δ with a predetermined time width of Δ (e.g., a video virtually captured by the driving recorder of the vehicle during the simulation).

[0021] However, even when evaluating autonomous driving, it is also possible to use the same driving situation information 300 as in the case of evaluating manual driving. That is, even when evaluating autonomous driving, as specific examples of the driving situation information 300, it is possible to use a video captured by the driving recorder of the vehicle that the user is currently driving or has driven in the past, or to use the accelerator opening, steering wheel cut angle and direction, and brake depression amount of the vehicle at each time in the video.

[0022] The autonomous driving control engine, for example, generates a simulation video from a certain time t to t+Δ according to a predetermined scenario, and then controls the vehicle's autonomous driving according to that simulation video. The autonomous driving control engine is implemented using a machine learning model (e.g., a neural network) created using known machine learning techniques.

[0023] The text generation unit 202 takes the driving situation information 300 acquired by the acquisition unit 201 as input and generates a first text 410 that describes the driving situation represented by the driving situation information 300 in natural language. The text generation unit 202 can generate the first text 410 from the driving situation information 300 using a machine learning model, such as a generation AI (Artificial Intelligence). For example, if the driving situation information 300 is a video (including a simulation video), the text generation unit 202 can generate the first text 410 by generating a caption for the video using the generation AI. In this case, the text generation unit 202 may generate the first text 410 using a generation AI provided as an external service, or it may generate the first text 410 using a generation AI provided by the driving evaluation device 10. Furthermore, when generating the first text 410 using the generation AI, the driving situation information 300 is used as input data, and a prompt (instruction) such as "Please generate a sentence that explains the input data." is input to the generation AI.

[0024] Generative AI, for example, is a function that generates various types of content, such as text, using machine learning and probability / statistical models, including large-scale language models (LLMs) created using deep learning techniques, a subfield of machine learning. Large-scale language models are constructed, for example, by training machine learning and statistical models such as neural networks using large amounts of text data as training data.

[0025] The comparison unit 203 compares the semantic content of the first text 410 generated by the text generation unit 202 with the second text 420, which explains driving criteria from a certain predetermined viewpoint in natural language. That is, the comparison unit 203 determines whether the semantic content of the first text 410 is consistent with the semantic content of the second text 420. Based on this, the appropriateness of the driving situation expressed in the first text 410 is evaluated based on the criteria expressed in the second text 420.

[0026] The comparison unit 203 can compare the semantic content of the first text 410 and the second text 420, for example, using a generating AI. In this case, the comparison unit 203 may compare the semantic content of the first text 410 and the second text 420 using a generating AI provided as an external service, or it may compare the semantic content of the first text 410 and the second text 420 using a generating AI possessed by the driving evaluation device 10. Furthermore, when comparing the semantic content of the first text 410 and the second text 420 using a generating AI, the first text 410 is used as the first input data and the second text 420 as the second input data, and a prompt (instruction sentence) such as "Compare the semantic content of the first input data with the semantic content of the second input data to see if there is any contradiction, and generate a sentence with the comparison result and its basis." is input to the generating AI.

[0027] Specific examples of the second text, 420, include texts representing the Road Traffic Act, texts representing safe driving standards within delivery companies, and texts representing rules for vehicle operation within factory premises. Examples of safe driving standards within delivery companies include, for example, "Do not drive through school zones," "Do not park or stop in places that would inconvenience nearby residents, even if parking or stopping is permitted," "Do not drive on routes where there are railway tracks crossing the road," "Do not use back roads or shortcuts," and "Do not park anywhere other than designated parking areas for loading and unloading goods." Examples of rules for vehicle operation within factory premises (e.g., forklifts) include, for example, "Do not turn around anywhere other than designated turning areas," "Do not park or stop in pre-designated areas," "Do not pass large vehicles in narrow spaces," "Do not stop anywhere other than designated waiting areas for loading and unloading," "Do not travel on routes other than those designated for loading and unloading," and "Do not exceed pre-designated drivable areas." The second text 420 is stored in a memory area such as the auxiliary storage device 107.

[0028] The notification unit 204 notifies a predetermined recipient of the comparison results obtained by the comparison unit 203. For example, when evaluating manual driving, the notification unit 204 notifies the user of the comparison results. On the other hand, when evaluating automated driving, for example, the notification unit 204 notifies the automated driving control engine of the comparison results. By notifying the automated driving control engine of the comparison results, the automated driving control engine can achieve highly accurate automated driving based on those results.

[0029] <Example of operation of the operation evaluation device 10> Figure 3 is a flowchart showing an example of the operation of the operation evaluation device according to this embodiment. Steps S101 to S104 below are performed, for example, each time operation status information 300 is provided to the operation evaluation device 10.

[0030] The acquisition unit 201 acquires the given driving condition information 300 (step S101).

[0031] The text generation unit 202 takes the driving situation information 300 acquired in step S101 as input and generates a first text 410 that describes the driving situation represented by the driving situation information 300 in natural language (step S102). An example of the first text 410 is shown in Figure 4. The first text 410 shown in Figure 4 indicates that the vehicle is traveling straight at a speed of 60 km / h, a red triangular sign is visible 30 m ahead of the vehicle, the sign says "STOP", the accelerator is open at 30%, and the brake is pressed 0%.

[0032] The comparison unit 203 compares the semantic content of the first text 410 generated in step S102 with the second text 420, which explains driving criteria from a certain predetermined viewpoint in natural language (step S103). That is, the comparison unit 203 determines whether the semantic content of the first text 410 is consistent with the semantic content of the second text 420. Based on the criteria expressed in the second text 420, the appropriateness of the driving situation expressed in the first text 410 is evaluated.

[0033] An example of the second text 420 is shown in Figure 5. The second text 420 shown in Figure 5 indicates that in intersections where traffic control is not in place or immediately before such intersections, road signs may indicate that a stop should be made, that a stop must be made immediately before a stop line indicated by a road sign, or immediately before the intersection if no stop line is provided, and that in this case, unless the road being traveled is a priority road, the vehicle must not obstruct the progress of vehicles traveling on the intersecting road. Furthermore, Figure 6 shows an example of the results of comparing the meaning of the first text 410 shown in Figure 4 and the second text 420 shown in Figure 5. The comparison result 500 shown in Figure 6 is considered to be contradictory to the first text 410 shown in Figure 4 and the second text 420 shown in Figure 5, because the sign visible in front of the vehicle is considered to be a sign indicating that a stop should be made, while the vehicle is not performing a stopping action with the accelerator pressed and the brake not applied.

[0034] The notification unit 204 notifies a predetermined notification recipient of the comparison result obtained in step S103 (step S104). In this case, for example, when evaluating manual driving, the notification unit 204 notifies the user of the comparison result. On the other hand, for example, when evaluating automated driving, the notification unit 204 notifies the automated driving control engine of the comparison result. The notification unit 204 may notify the comparison result in natural language as is, or it may convert the comparison result in natural language into some kind of information (e.g., a score) and then notify the converted information. When converting the comparison result in natural language into a score, a generation AI may be used. For example, by inputting a prompt such as "On a scale of 1 to 4, how would you rate this comparison result?" into the generation AI, the comparison result in natural language may be converted into a score on a scale of 1 to 4.

[0035] <Variation> • Variation 1 In the above embodiment, the target was a vehicle, but it is also possible to target, for example, a pedestrian wearing a wearable camera. In this case, the driving situation information 300 would be a video taken by the wearable camera, and the second text 420 would be text that explains criteria regarding walking from a certain predetermined viewpoint in natural language. This would make it possible, for example, to evaluate whether a pedestrian is able to walk safely inside a factory or whether a pedestrian is able to walk safely on a school route.

[0036] • Variation 2 In the above embodiment, the semantic content of the first text 410 and the second text 420 were compared. However, for example, the appropriateness of the semantic content represented by the first text 410 may be determined using a pre-trained machine learning model. Specifically, a pre-trained machine learning model may be created using the second text 420 as training data, and then the comparison unit 203 may use this pre-trained machine learning model to determine the appropriateness of the semantic content represented by the first text 410 (i.e., the appropriateness of the driving situation represented by the first text 410).

[0037] <Summary> As described above, the driving evaluation device 10 according to this embodiment converts the given driving situation information 300 into a first text 410, and then compares the meaning of this first text 410 with a second text 420 that explains driving criteria from a certain predetermined viewpoint in natural language. Based on these criteria, the appropriateness of the driving situation represented by the given driving situation information 300 is evaluated.

[0038] Therefore, by using the driving evaluation device 10 according to this embodiment, it becomes possible to evaluate, for example, the appropriateness of the user's actual driving or the appropriateness of the automated driving realized by the automated driving control engine. In particular, by evaluating the appropriateness of the automated driving realized by the automated driving control engine, it becomes possible to realize more accurate automated driving (e.g., safer automated driving).

[0039] The present invention is not limited to the embodiments specifically disclosed above, and various modifications, changes, and combinations with known technologies are possible as long as they do not deviate from the spirit described in the claims. [Explanation of Symbols]

[0040] 10. Operation evaluation device 101 Input Device 102 Display device 103 External I / F 103a Recording medium 104 Communication I / F 105 RAM 106 ROM 107 Auxiliary storage 108 processors 109 Bus 201 Acquisition Department 202 Text Generation Unit 203 Comparison Section 204 Notification Department 300 Operating Status Information 410 First Text 420 Second Text

Claims

1. An acquisition unit that acquires information representing the situation of the target, A text generation unit generates a first text that describes the situation in natural language based on the aforementioned information, An evaluation unit that evaluates the appropriateness of the situation represented by the first text based on criteria for evaluating the appropriateness of the situation from a predetermined viewpoint, A notification unit that notifies a predetermined recipient of the results of the aforementioned evaluation, An information processing device having

2. The evaluation unit described above, The information processing apparatus according to claim 1, which evaluates the appropriateness of the situation represented by the first text by comparing the semantic content represented by the first text with the semantic content represented by the second text, based on a second text that explains the criteria in natural language and the first text.

3. The evaluation unit described above, The information processing apparatus according to claim 2, wherein a generating AI is used to compare the semantic content represented by the first text with the semantic content represented by the second text.

4. The evaluation unit described above, The information processing device according to claim 1, which evaluates the appropriateness of the situation represented by the first text based on a machine learning model that has been pre-trained using a second text that explains the aforementioned criteria in natural language as training data, and the first text.

5. The aforementioned information includes video footage captured by the camera equipment provided by the subject. The text generation unit, The information processing apparatus according to any one of claims 1 to 4, which generates the description of the video as the first text.

6. The subject is an autonomous vehicle, and the video is a simulation video of the autonomous vehicle in operation. The aforementioned notification unit, The information processing device according to claim 5, which notifies the results of the evaluation to the autonomous driving control engine that controls the autonomous driving of the autonomous vehicle.

7. The aforementioned subject is a vehicle driven by the user, The aforementioned notification unit, The information processing apparatus according to claim 5, which notifies the user of the results of the evaluation.

8. The information processing device according to claim 1, wherein the standard is the safe driving standard for the subject, or the operating rules for the subject within the factory.

9. Procedure for obtaining information that represents the situation of the target, A text generation procedure that generates a first text describing the situation in natural language based on the aforementioned information, An evaluation procedure for evaluating the appropriateness of the situation represented by the first text based on criteria for evaluating the appropriateness of the situation from a predetermined viewpoint, A notification procedure for notifying a predetermined recipient of the results of the aforementioned evaluation, A method of information processing performed by a computer.

10. Procedure for obtaining information that represents the situation of the target, A text generation procedure that generates a first text describing the situation in natural language based on the aforementioned information, An evaluation procedure for evaluating the appropriateness of the situation represented by the first text based on criteria for evaluating the appropriateness of the situation from a predetermined viewpoint, A notification procedure for notifying a predetermined recipient of the results of the aforementioned evaluation, A program that causes a computer to execute something.

Images