Scenario extraction method, scenario extraction device, and scenario extraction program

Abstract

To provide a scenario extraction method, a scenario extraction device, and a scenario extraction program for extracting various scenarios requested by a user.SOLUTION: A scenario extraction device 10 includes: an acquisition unit 12 for acquiring time-series data related to a vehicle, behavior of the vehicle input by a user, behavior of a front vehicle positioned in front of the vehicle, and an item related to a road on which the vehicle has traveled; a labeling unit 14 for generating classification of a vehicle label that encompasses each item, classification of a front vehicle label, and classification of a road label according to the items input by a user; an imparting unit 16 for imparting an item related to behavior of the vehicle included in classification of the vehicle label generated, an item related to behavior of the front vehicle included in classification of front vehicle label, and an item related to a road included in classification of a road label relative to the time-series data; and an extraction unit 18 for extracting data including an item input by a user from the time-series data to which the items are imparted.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a scenario extraction method, a scenario extraction device, and a scenario extraction program. [Background technology]

[0002] Conventionally, a system for performing intersection scenario acquisition as described in Patent Document 1 is known. In this system, a video stream associated with the surrounding environment is received. The video stream is analyzed and trimmed into a video clip of an intersection scene associated with the traveling of the host vehicle. The host vehicle, dynamic objects and their motion paths contained in the intersection scene are annotated, i.e., information-tagged, with action units describing the intersection scenario. Then, based on a query of an electronic dataset storing combinations of action units, at least one intersection scenario is acquired and a presentation of at least one intersection scenario video clip including the at least one intersection scenario is operatively controlled. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-136022 Summary of the Invention [Problem to be solved by the invention]

[0004] The system described in Patent Document 1 is capable of extracting one specific scenario. However, it is difficult to comprehensively extract various scenarios requested by users of the system, for example, scenarios other than intersection scenarios.

[0005] An object of the present disclosure is to provide a scenario extraction device and a scenario extraction program that extract various scenarios requested by a user. [Means for solving the problem]

[0006] The invention described in claim 1 is a scenario extraction method that acquires time series data on a vehicle (30), and items related to the behavior of the vehicle, items related to the behavior of a forward vehicle (35) located in front of the vehicle, and items related to the road (40) on which the vehicle has traveled, which are input by a user, generates a vehicle label classification including items related to the behavior of the vehicle, a forward vehicle label classification including items related to the behavior of the forward vehicle, and a road label classification including items related to the road, according to the items input by the user, assigns the generated items related to the vehicle behavior included in the vehicle label classification, the items related to the behavior of the forward vehicle included in the forward vehicle label classification, and the items related to the road included in the road label classification to the time series data, and extracts data including the items input by the user from the time series data to which items have been assigned.

[0007] The invention described in claim 12 is a scenario extraction device including an acquisition unit (12) that acquires time series data related to a vehicle (30), items related to the behavior of the vehicle, items related to the behavior of a forward vehicle (35) located in front of the vehicle, and items related to a road (40) on which the vehicle has traveled, which are input by a user; a labeling unit (14) that generates a vehicle label classification including items related to the behavior of the vehicle, a forward vehicle label classification including items related to the behavior of the forward vehicle, and a road label classification including items related to the road, according to the items input by the user; an assignment unit (16) that assigns to the time series data items related to the vehicle behavior included in the vehicle label classification, items related to the behavior of the forward vehicle included in the forward vehicle label classification, and items related to the road included in the road label classification, which were generated by the labeling unit; and an extraction unit (18) that extracts data including the items input by the user from the time series data to which items have been assigned by the assignment unit.

[0008] The invention described in claim 13 is a scenario extraction program that causes a scenario extraction device to function as an acquisition unit (12) that acquires time series data related to a vehicle (30), items related to the behavior of the vehicle, items related to the behavior of a forward vehicle (35) located in front of the vehicle, and items related to a road (40) on which the vehicle has traveled, which are input by a user; a labeling unit (14) that generates a vehicle label classification including items related to the behavior of the vehicle, a forward vehicle label classification including items related to the behavior of the forward vehicle, and a road label classification including items related to the road, according to the items input by the user; an assignment unit (16) that assigns to the time series data items related to the vehicle behavior included in the vehicle label classification, items related to the behavior of the forward vehicle included in the forward vehicle label classification, and items related to the road included in the road label classification, which are generated by the labeling unit; and an extraction unit (18) that extracts data including the items input by the user from the time series data to which items have been assigned by the assignment unit.

[0009] This allows data including items input by the user to be extracted, and therefore various scenarios requested by the user to be extracted.

[0010] Note that the reference symbols in parentheses attached to each component indicate an example of the correspondence between the component and specific components described in the embodiments described below. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a configuration diagram of a scenario extraction device according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing a vehicle used in the scenario extraction device. [Diagram 3] 1 is a table showing classifications and items of vehicle labels used for scenario extraction by the scenario extraction device. [Figure 4] 1 is a table showing classifications and items for forward vehicle labels. [Diagram 5] A table showing classifications and items for road labels. [Figure 6] A table showing classifications and items for front two-wheel labels. [Figure 7] A table showing classifications and items for front bicycle labels. [Figure 8] A table showing classifications and items for pedestrian ahead labels. [Figure 9] A table showing classifications and items for traffic labels. [Figure 10] A table showing classifications and items for time zone labels. [Figure 11] 4 is a flowchart showing a process of a scenario extraction device. [Figure 12] FIG. 13 is a diagram showing classification generation by the scenario extraction device. [Figure 13] FIG. 13 is a diagram showing item assignment by the scenario extraction device. [Figure 14] FIG. 13 is a diagram showing a request scenario extraction by the scenario extraction device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments will be described with reference to the drawings. In the following embodiments, the same or equivalent parts are denoted by the same reference numerals, and the description thereof will be omitted.

[0013] In a scenario extraction device that performs the scenario extraction method of this embodiment, various scenarios requested by a user are extracted. This scenario extraction device is used in a vehicle. Note that a scenario here refers to a collection of multiple correlations between the vehicle's running state and running environment and the behavior of traffic participants, such as cut-in and cut-out.

[0014] 1, the scenario extraction device 10 is mainly composed of a microcomputer and includes a CPU, ROM, flash memory, RAM, I / O, and bus lines connecting these components, etc. Furthermore, the scenario extraction device 10 includes an acquisition unit 12, a labeling unit 14, an assignment unit 16, an extraction unit 18, and an output unit 20 as functional blocks.

[0015] The acquisition unit 12 acquires time-series data on the vehicle 30 from the vehicle 30. The acquisition unit 12 also acquires items input by the user related to the behavior of the vehicle 30, items related to the behavior of a preceding vehicle 35, and items related to a road 40 on which the vehicle 30 has traveled, which are described below. The acquisition unit 12 also acquires items input by the user related to the behavior of a preceding two-wheeled vehicle 45, items related to the behavior of a preceding bicycle 50, items related to the behavior of a preceding pedestrian 55, items related to traffic scenes, and items related to the time period in which the vehicle 30 has traveled.

[0016] Here, the vehicle 30 is a four-wheeled vehicle as shown in FIG. 2. The vehicle in front 35 is a four-wheeled vehicle located in front of the vehicle 30. The two-wheeled vehicle in front 45 is a two-wheeled vehicle located in front of the vehicle 30. The bicycle in front 50 is a bicycle located in front of the vehicle 30. The pedestrian in front 55 is a pedestrian located in front of the vehicle 30. In FIG. 2, the vehicle 30 is shown with a dot pattern to distinguish it from the vehicle in front 35. A bicycle is a vehicle with two or more wheels that uses pedals or a hand crank and is driven by human power, and is other than wheelchairs for the physically disabled, walking aids, and cars for children. Furthermore, a traffic scene is a scene in which the vehicle 30 is traveling.

[0017] The vehicle 30 is also equipped with a group of sensors, a camera system, a Lidar system, a millimeter wave radar system, and the like, which are not shown. The vehicle 30 uses these to generate time series data on the state of the vehicle 30, such as the position, speed, acceleration, and direction of the vehicle 30. The vehicle 30 also uses these to generate time series data on the state of the vehicle 35 in front, such as the position, speed, acceleration, and direction of the vehicle 35 in front. The vehicle 30 also uses these to generate time series data on the road 40 on which the vehicle 30 has traveled. The vehicle 30 also uses these to generate time series data on the state of the two-wheeled vehicle 45 in front, such as the position, speed, acceleration, and direction of the two-wheeled vehicle 45 in front. The vehicle 30 also uses these to generate time series data on the state of the bicycle 50 in front, such as the position, speed, acceleration, and direction of the bicycle 50 in front. The vehicle 30 also uses these to generate time series data on the state of the pedestrian 55 in front, such as the position, speed, acceleration, and direction of the pedestrian 55 in front. The vehicle 30 also uses these to generate time series data on the traffic scene. Furthermore, the vehicle 30 uses these to generate time series data regarding the time periods during which the vehicle 30 traveled.

[0018] 1, the labeling unit 14 generates a vehicle label classification, a forward vehicle label classification, and a road label classification, which will be described later, in response to the items input by the user. The labeling unit 14 also generates a forward two-wheeled vehicle label classification, a forward bicycle label classification, a forward pedestrian label classification, a traffic label classification, and a time zone label classification, which will be described later, in response to the items input by the user.

[0019] Here, the vehicle label, the forward vehicle label, the road label, the forward two-wheeled vehicle label, the forward bicycle label, the forward pedestrian label, the traffic label, and the time zone label will be described with reference to Fig. 3 to Fig. 10. In Fig. 3 to Fig. 10, F1-X is a symbol indicating each label. F1-XY is a symbol indicating each classification. F1-XYZ is a symbol indicating each item. X, Y, and Z are natural numbers.

[0020] The vehicle label includes items related to the behavior of the vehicle 30. For example, as shown in FIG. 3, the vehicle label has four categories: vehicle operation, vehicle speed change, vehicle speed, and vehicle start / stop. The vehicle operation category includes lane keeping, lane change to the right, lane change to the left, right turn, left turn, and stop as items related to the behavior of the vehicle 30. The vehicle speed change category includes acceleration, deceleration, and constant speed as items related to the behavior of the vehicle 30. The vehicle speed category includes stop, low speed, medium speed, high speed, and reverse as items related to the behavior of the vehicle 30. The vehicle start / stop category includes start scenes and stop scenes as items related to the behavior of the vehicle 30.

[0021] The items in the classification of vehicle speed change, vehicle speed, and vehicle start / stop correspond to items related to the speed of the vehicle 30. Acceleration, for example, refers to a state in which the acceleration is a positive value and is greater than the first acceleration threshold. Deceleration, for example, refers to a state in which the acceleration is a negative value and is less than the second acceleration threshold. Constant speed, for example, refers to a state in which the acceleration is zero or near zero, and is less than the first acceleration threshold and greater than the second acceleration threshold. Stop, for example, refers to a state in which the speed is zero or near zero, and is less than the first speed threshold and greater than the second speed threshold. Low speed, for example, refers to a state in which the speed is a positive value, greater than the first speed threshold, and less than the third speed threshold. Medium speed, for example, refers to a state in which the speed is a positive value, greater than the third speed threshold, and less than the fourth speed threshold. High speed, for example, refers to a state in which the speed is a positive value and greater than the fourth speed threshold. Reversing, for example, refers to a state in which the speed is a negative value and less than the second speed threshold. Furthermore, the starting scene refers to a state in which the vehicle changes from stopped to accelerating. The stopping scene refers to a state in which the vehicle changes from deceleration to stopped. The first acceleration threshold and the second acceleration threshold are set by experiments, simulations, etc. so that each item is set. The first speed threshold, the second speed threshold, the third speed threshold, and the fourth speed threshold are set by experiments, simulations, etc. so that each item is set.

[0022] The forward vehicle label includes items related to the behavior of the forward vehicle 35. For example, as shown in FIG. 4, the forward vehicle label has seven classifications, including forward vehicle operation, forward vehicle speed change, forward vehicle speed, forward vehicle start / stop, orientation relationship with the vehicle 30, lateral position relationship with the vehicle 30, and vertical position relationship with the vehicle 30. The classification of forward vehicle operation includes cut-in from the right side, cut-in from the left side, cut-out to the right side, cut-out to the left side, lane keeping, right turn, and left turn as items related to the behavior of the forward vehicle 35. The classification of forward vehicle speed change includes acceleration, deceleration, and constant speed as items related to the behavior of the forward vehicle 35. The classification of forward vehicle speed includes stop, low speed, medium speed, high speed, and reverse as items related to the behavior of the forward vehicle 35. The classification of forward vehicle start / stop includes start and stop scenes as items related to the behavior of the forward vehicle 35. The classification of the directional relationship with vehicle 30 includes oncoming vehicle, same direction, outward crossing, and inward crossing as items related to the behavior of the preceding vehicle 35. The classification of the lateral positional relationship with vehicle 30 includes same lane, adjacent lane, and adjacent lane after that as items related to the behavior of the preceding vehicle 35. The classification of the longitudinal positional relationship with vehicle 30 includes short distance, medium distance, and long distance as items related to the behavior of the preceding vehicle 35.

[0023] The items in the classification of forward vehicle speed change, forward vehicle speed, and forward vehicle start / stop correspond to the items of the contents related to the speed of the forward vehicle 35. The outward crossing refers to a state in which the moving direction of the object in front of the vehicle 30, for example, the forward vehicle 35, is a direction away from and crosses the moving direction of the vehicle 30. The inward crossing refers to a state in which the moving direction of the object in front of the vehicle 30 is a direction approaching and crossing the moving direction of the vehicle 30. The short distance refers to a state in which the distance between the objects is less than the first distance threshold. The medium distance refers to a state in which the distance between the objects is equal to or greater than the first distance threshold and less than the second distance threshold. The long distance refers to a state in which the distance between the objects is equal to or greater than the second distance threshold. The first distance threshold and the second distance threshold are set by experiments, simulations, etc. so that each item is set. The lateral positional relationship with the vehicle 30 refers to the left-right positional relationship of the vehicle 30 with respect to the vehicle 30. The longitudinal positional relationship with vehicle 30 refers to the positional relationship with vehicle 30 in the forward direction of vehicle 30.

[0024] The road label includes items related to the road 40. For example, as shown in FIG. 5, the road label has seven classifications, including road type, motorway section, general road section, number of lanes, lane shape change, road gradient, and curve change. The classification of road type includes motorway, general road, and parking lot as items related to the road 40. The classification of motorway section includes main road, merging zone from the right, merging zone from the left, exit zone from the right, exit zone from the left, and ramp as items related to the road 40. The classification of general road section includes intersection, curved road, and straight road as items related to the road 40. The classification of number of lanes includes one lane, two lanes, and three or more lanes as items related to the road 40. The classification of lane shape change includes change from straight to curve, and continuous straight line as items related to the road 40. The classification of road gradient includes flat, uphill, and downhill as items related to the road 40. The classification of curve changes includes continuous curves and non-continuous curves as terms related to the road 40 .

[0025] In addition, a motorway is a road or a part of a road designated by a road administrator based on the law on roads, which is used for general traffic only by automobiles. A general road is a road other than a motorway that can be used by both vehicles and pedestrians. A ramp is a road that connects roads that intersect and connect with each other when the road is a grade separation, and is a part of the structure of an interchange or junction. In addition, a road gradient is a percentage value obtained by dividing the difference in elevation between two points by the horizontal distance. For example, a flat road refers to a state in which the road gradient is zero or close to zero, and the road gradient is equal to or less than a first gradient threshold and equal to or more than a second gradient threshold. For example, an uphill road refers to a state in which the road gradient is a positive value and is greater than the first gradient threshold. In addition, a downhill road refers to a state in which the road gradient is a negative value and is less than the second gradient threshold. In addition, the first gradient threshold and the second gradient threshold are set by experiments, simulations, etc. so that each item is set. In addition, a continuous curve refers to a road that has two or more consecutive curves. A non-continuous curve is one that is not a continuously curved road.

[0026] The front two-wheeled vehicle label includes items related to the behavior of the front two-wheeled vehicle 45. For example, as shown in FIG. 6, the front two-wheeled vehicle label has seven classifications, namely, front two-wheeled vehicle operation, front two-wheeled vehicle speed change, front two-wheeled vehicle speed, front two-wheeled vehicle start / stop, orientation relationship with the vehicle 30, lateral position relationship with the vehicle 30, and vertical position relationship with the vehicle 30. The classification of the front two-wheeled vehicle operation includes cut-in from the right side, cut-in from the left side, cut-out to the right side, cut-out to the left side, lane keeping, right turn, and left turn as items related to the behavior of the front two-wheeled vehicle 45. The classification of the front two-wheeled vehicle speed change includes acceleration, deceleration, and constant speed as items related to the behavior of the front two-wheeled vehicle 45. The classification of the front two-wheeled vehicle speed includes stop, low speed, medium speed, high speed, and reverse as items related to the behavior of the front two-wheeled vehicle 45. The classification of the front two-wheeled vehicle start / stop includes start scenes and stop scenes as items related to the behavior of the front two-wheeled vehicle 45. The classification of the directional relationship with the vehicle 30 includes oncoming vehicle, same direction, outward crossing, and inward crossing as items related to the behavior of the two-wheeled vehicle 45 in front. The classification of the lateral positional relationship with the vehicle 30 includes same lane, adjacent lane, and adjacent lane after that as items related to the behavior of the two-wheeled vehicle 45 in front. The classification of the longitudinal positional relationship with the vehicle 30 includes short distance, medium distance, and long distance as items related to the behavior of the two-wheeled vehicle 45 in front. The items in the classification of speed change of the two-wheeled vehicle in front, speed of the two-wheeled vehicle in front, and starting and stopping of the two-wheeled vehicle in front correspond to items related to the speed of the two-wheeled vehicle in front.

[0027] The bicycle-in-front label includes items related to the behavior of the bicycle in front 50. For example, as shown in FIG. 7, the bicycle-in-front label has five categories: bicycle-in-front motion, bicycle-in-front speed change, directional relationship with the vehicle 30, lateral positional relationship with the vehicle 30, and longitudinal positional relationship with the vehicle 30. The categories of bicycle-in-front motion include moving, stopping, cutting in, cutting out, lane keeping, right turn, and left turn as items related to the behavior of the bicycle in front 50. The categories of bicycle-in-front speed change include accelerating, decelerating, and constant speed as items related to the behavior of the bicycle in front 50. The categories of directional relationship with the vehicle 30 include oncoming vehicles, same direction, outward crossing, and inward crossing as items related to the behavior of the bicycle in front 50. The categories of lateral positional relationship with the vehicle 30 include same lane, middle distance to the right, long distance to the right, middle distance to the left, and long distance to the left as items related to the behavior of the bicycle in front 50. The classification of the longitudinal positional relationship with the vehicle 30 includes short distance, medium distance, and long distance as items related to the behavior of the bicycle in front 50. In addition to the above classifications, the bicycle in front label may also have a classification of the speed of the bicycle in front, which may include items related to the speed of the bicycle in front 50, such as stopped, slow, etc.

[0028] The pedestrian in front label includes items related to the behavior of the pedestrian in front 55. For example, as shown in FIG. 8, the pedestrian in front label has five classifications, namely, the action of the pedestrian in front, the speed change of the pedestrian in front, the orientation relationship with the vehicle 30, the lateral position relationship with the vehicle 30, and the longitudinal position relationship with the vehicle 30. The classification of the action of the pedestrian in front includes movement and stopping as items related to the behavior of the pedestrian in front 55. The classification of the speed change of the pedestrian in front includes acceleration, deceleration, and constant speed as items related to the behavior of the pedestrian in front 55. The classification of the orientation relationship with the vehicle 30 includes oncoming, same direction, outward crossing, and inward crossing as items related to the behavior of the pedestrian in front 55. The classification of the lateral position relationship with the vehicle 30 includes the same lane, middle distance on the right, long distance on the right, middle distance on the left, and long distance on the left as items related to the behavior of the pedestrian in front 55. The classification of the longitudinal position relationship with the vehicle 30 includes short distance, middle distance, and long distance as items related to the behavior of the pedestrian in front 55. In addition, the pedestrian ahead label may have a classification of the speed of the pedestrian ahead in addition to the above classification, and the classification of the speed of the pedestrian ahead may include items related to the speed of the pedestrian ahead 55, such as stopped, slow, etc.

[0029] The traffic label includes items related to a traffic scene. For example, as shown in FIG. 9, the traffic label has 11 classifications, including presence or absence of traffic signal, traffic signal color, presence or absence of construction area, presence or absence of intersection obstruction, presence or absence of intersection, presence or absence of traffic sign, presence or absence of stop sign, presence or absence of shoulder space, change in traffic signal color, presence or absence of road structure, and presence or absence of congestion. The classification of presence or absence of traffic signal includes presence or absence of traffic signal as items related to a traffic scene. The classification of traffic signal color includes red light, and yellow or green light as items related to a traffic scene. The classification of presence or absence of construction area includes presence or absence of construction area as items related to a traffic scene. The classification of presence or absence of intersection obstruction includes presence or absence of intersection obstruction as items related to a traffic scene. The classification of presence or absence of intersection includes presence or absence of intersection as items related to a traffic scene. The classification of presence or absence of traffic sign includes presence or absence of traffic sign as items related to a traffic scene. The classification of whether or not there is a stop sign includes whether there is a stop sign and whether or not there is a stop sign as items related to a traffic scene. The classification of whether or not there is shoulder space includes whether or not there is shoulder space and whether or not there is shoulder space as items related to a traffic scene. The classification of a change in traffic light color includes a change from a red light to a green light and a change from a green light to a red light as items related to a traffic scene. The classification of whether or not there is a road structure includes whether or not there is a road structure and whether or not there is a road structure as items related to a traffic scene. The classification of whether or not there is a traffic jam includes whether or not there is a traffic jam and whether or not there is a traffic jam as items related to a traffic scene.

[0030] The presence or absence of a traffic signal corresponds to the presence or absence of a traffic signal. The traffic signal color and traffic signal color change correspond to the color of a traffic signal. Examples of road structures include guard rails, median strips, and buildings on the road. An intersection obstruction refers to sunlight, buildings, and the like at an intersection that block the view of the driver of the vehicle 30. A traffic jam refers to a state in which the traffic density is equal to or greater than a density threshold. Traffic density is the number of vehicles per unit distance. The unit of traffic density is, for example, vehicles / km. The density threshold is set by experiments, simulations, and the like, so that each item is set.

[0031] The time period label includes items related to a time period. For example, the time period label has one classification of a time period, as shown in Fig. 10. The classification of the time period includes night and morning or afternoon as items related to a time period.

[0032] Returning to FIG. 1, the assignment unit 16 assigns items included in the classification of the labels generated by the labeling unit 14 to the time-series data acquired by the acquisition unit 12 using machine learning, teacher data, and the like.

[0033] Specifically, the assigning unit 16 assigns an item related to the behavior of the vehicle 30 included in the classification of the generated vehicle label to the time series data acquired by the acquisition unit 12 using machine learning, teacher data, etc. Furthermore, the assigning unit 16 assigns an item related to the behavior of the forward vehicle 35 included in the classification of the generated forward vehicle label to the time series data acquired by the acquisition unit 12 using machine learning, teacher data, etc. Furthermore, the assigning unit 16 assigns an item related to the road 40 included in the classification of the generated road label to the time series data acquired by the acquisition unit 12 using machine learning, teacher data, etc. Furthermore, the assigning unit 16 assigns an item related to the behavior of the forward two-wheeled vehicle 45 included in the classification of the generated forward two-wheeled label to the time series data acquired by the acquisition unit 12 using machine learning, teacher data, etc. Furthermore, the assigning unit 16 assigns an item related to the behavior of the forward bicycle 50 included in the classification of the generated forward bicycle label to the time series data acquired by the acquisition unit 12 using machine learning, teacher data, etc. Moreover, the assigning unit 16 assigns items related to the behavior of the pedestrian ahead 55 included in the classification of the generated pedestrian ahead label to the time series data acquired by the acquiring unit 12 using machine learning, teacher data, etc. Furthermore, the assigning unit 16 assigns items related to traffic scenes included in the classification of the generated traffic label to the time series data acquired by the acquiring unit 12 using machine learning, teacher data, etc. Furthermore, the assigning unit 16 assigns items related to time periods included in the classification of the generated time period label to the time series data acquired by the acquiring unit 12 using machine learning, teacher data, etc. Note that the teacher data is set by experiments, simulations, etc. so that each item is assigned to the time series data.

[0034] The extraction unit 18 extracts data including the items input by the user from the time-series data to which items have been assigned by the assignment unit 16. In this way, the extraction unit 18 extracts a scenario requested by the user.

[0035] The output unit 20 outputs the data extracted by the extraction unit 18 to an external device, for example, a database 60. The data extracted by the extraction unit 18 is also stored in the database 60. The user can then view the data extracted by the extraction unit 18, i.e., the requested scenario, from the database 60.

[0036] The scenario extraction device 10 is configured as described above. Next, as an example, scenario extraction by executing a program of the scenario extraction device 10 will be described with reference to the flowchart of Fig. 11 and Figs. 12, 13, and 14. Note that the program of the scenario extraction device 10 is executed, for example, by activation by a user.

[0037] Here, in one example, it is assumed that the user inputs deceleration as an item in the classification of vehicle speed change in the vehicle label. It is also assumed that the user inputs left turn as an item in the classification of forward vehicle operation in the forward vehicle label. It is also assumed that the user inputs inward intersection as an item in the classification of directional relationship with vehicle 30 in the forward vehicle label. It is also assumed that the user inputs same lane in the classification of lateral position relationship with vehicle 30 in the forward vehicle label. It is also assumed that the user inputs general road in the classification of road type in the road label. It is also assumed that the user inputs no intersection in the classification of intersection presence / absence in the traffic label.

[0038] 11, the acquisition unit 12 acquires various pieces of information. Specifically, the acquisition unit 12 acquires time-series data related to the vehicle 30 from the vehicle 30. Furthermore, the acquisition unit 12 acquires items input by the user.

[0039] Next, in step S102, the labeling unit 14 generates a classification of labels according to the items input by the user. In one example, the labeling unit 14 generates a classification of vehicle speed change for the vehicle label as shown in FIG. 12. The labeling unit 14 also generates a classification of forward vehicle operation, a classification of orientation relative to the vehicle 30, and a classification of lateral positional relationship relative to the vehicle 30 for the forward vehicle label. The labeling unit 14 also generates a classification of road type for the road label. The labeling unit 14 also generates a classification of presence or absence of intersection for the traffic label.

[0040] Next, in step S104, the assigning unit 16 assigns items included in the classification of the label generated in step S102 to the time series data acquired in step S100 using machine learning and teacher data, etc. In one example, the assigning unit 16 assigns constant speed, deceleration, and acceleration in the classification of vehicle speed change of the vehicle label to each time in the time series data, for example, as shown in FIG. 13. In addition, the assigning unit 16 assigns lane keeping, stop, and left turn in the classification of the forward vehicle operation of the forward vehicle label to each time in the time series data. Furthermore, the assigning unit 16 assigns inward crossing and same direction in the classification of the orientation relationship with the vehicle 30 of the forward vehicle label to each time in the time series data. In addition, the assigning unit 16 assigns adjacent lane and same lane in the classification of the lateral position relationship with the vehicle 30 of the forward vehicle label to each time in the time series data. Furthermore, the assigning unit 16 assigns general road in the classification of road type in the road label to each time in the time series data. In addition, the assignment unit 16 assigns "no intersection" in the classification of the intersection presence / absence in the traffic label to each time in the time-series data.

[0041] Next, in step S106, the extraction unit 18 extracts data including the items input by the user from the time-series data to which items have been assigned in step S104, as shown in Fig. 14. In this way, the extraction unit 18 extracts a scenario requested by the user.

[0042] 11 , subsequently, in step S108, the output unit 20 outputs the data extracted in step S106 to the database 60. The data extracted in step S106 is stored in the database 60. Then, the user can view the data extracted in step S106 from the database 60.

[0043] As described above, the scenario extraction device 10 extracts a scenario requested by the user. In the above example, the user inputs items included in the vehicle label, the forward vehicle label, the road label, and the traffic label, but this is not limiting. The user may input any item included in the vehicle label, the forward vehicle label, the road label, the forward two-wheeled vehicle label, the forward bicycle label, the forward pedestrian label, the traffic label, and the time zone label. Whatever item is input, the scenario extraction device 10 extracts a scenario requested by the user in the same manner as described above.

[0044] Next, it will be described how the scenario extraction device 10 extracts various scenarios requested by a user.

[0045] The scenario extraction device 10 includes an acquisition unit 12, a labeling unit 14, an assignment unit 16, and an extraction unit 18. The acquisition unit 12 acquires time-series data related to the vehicle 30. The acquisition unit 12 also acquires items related to the behavior of the vehicle 30, items related to the behavior of the forward vehicle 35, and items related to the road 40, which are input by the user. The labeling unit 14 generates a vehicle label classification, a forward vehicle label classification, and a road label classification according to the items input by the user. The assignment unit 16 assigns the items generated by the labeling unit 14 to the time-series data acquired by the acquisition unit 12. Here, the assignment unit 16 assigns items related to the behavior of the vehicle 30 included in the vehicle label classification, items related to the behavior of the forward vehicle 35 included in the forward vehicle label classification, and items related to the road 40 included in the road label classification, which are generated by the labeling unit 14. The extraction unit 18 extracts data including the items input by the user from the time-series data to which items are assigned by the assignment unit 16.

[0046] This allows data including items input by the user to be extracted, and therefore various scenarios requested by the user to be extracted.

[0047] In addition, the scenario extraction device 10 of this embodiment also provides the effects described below.

[0048] [1] The acquisition unit 12 acquires items related to the behavior of the two-wheeled vehicle 45 input by the user. The labeling unit 14 generates a classification of the two-wheeled vehicle in front according to the items input by the user. The assignment unit 16 assigns items related to the behavior of the two-wheeled vehicle in front, which are included in the classification of the two-wheeled vehicle in front generated by the labeling unit 14, to the time-series data acquired by the acquisition unit 12.

[0049] [2] The acquisition unit 12 acquires items related to the behavior of the bicycle 50 in front input by the user. The labeling unit 14 generates a classification of the bicycle in front label according to the items input by the user. The assignment unit 16 assigns items related to the behavior of the bicycle 50 in front included in the classification of the bicycle in front label generated by the labeling unit 14 to the time-series data acquired by the acquisition unit 12.

[0050] [3] The acquiring unit 12 acquires items related to the behavior of the pedestrian 55 ahead input by the user. The labeling unit 14 generates a classification of the pedestrian ahead label according to the items input by the user. The assigning unit 16 assigns items related to the behavior of the pedestrian 55 ahead included in the classification of the pedestrian ahead label generated by the labeling unit 14 to the time-series data acquired by the acquiring unit 12.

[0051] [4] The acquiring unit 12 acquires items related to traffic scenes input by a user. The labeling unit 14 generates a classification of traffic labels according to the items input by the user. The assigning unit 16 assigns items related to traffic scenes included in the classification of the traffic labels generated by the labeling unit 14 to the time-series data acquired by the acquiring unit 12.

[0052] [5] The acquisition unit 12 acquires items related to time periods input by a user. The labeling unit 14 generates a classification of time period labels according to the items input by the user. The assignment unit 16 assigns items related to time periods included in the classification of time period labels generated by the labeling unit 14 to the time series data acquired by the acquisition unit 12.

[0053] Any of the above [1]-[5] will allow you to subdivide the scenarios requested by users, making it easier to respond to scenarios requested by users.

[0054] (Other embodiments) The present disclosure is not limited to the above-described embodiments, and appropriate modifications can be made to the above-described embodiments. Furthermore, in each of the above-described embodiments, it goes without saying that the elements constituting the embodiments are not necessarily essential, except in cases where they are particularly clearly stated as essential or where they are clearly considered essential in principle.

[0055] The acquisition unit, labeling unit, assignment unit, extraction unit, output unit, and the method thereof described in the present disclosure may be realized by a dedicated computer provided by configuring a processor and memory programmed to execute one or more functions embodied in a computer program. Alternatively, the acquisition unit, labeling unit, assignment unit, extraction unit, output unit, and the method thereof described in the present disclosure may be realized by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the acquisition unit, labeling unit, assignment unit, extraction unit, output unit, and the method thereof described in the present disclosure may be realized by one or more dedicated computers configured by a combination of a processor and memory programmed to execute one or more functions and a processor configured with one or more hardware logic circuits. In addition, the computer program may be stored in a computer-readable non-transitory tangible recording medium as instructions executed by a computer.

[0056] In the above embodiment, the forward vehicle 35 is a four-wheeled vehicle, but is not limited to this and may be a two-wheeled vehicle, a bicycle, or the like.

[0057] In the above embodiment, the number of vehicles 35, two-wheeled vehicles 45, and bicycles 50 in front listed on each label is one, but may be more than one. Also, the number of pedestrians 55 listed on the pedestrian label in front is one, but may be more than one.

[0058] In the above embodiment, the classification of the time period in the time period label includes, but is not limited to, night and morning or afternoon as items related to the time period. The classification of the time period in the time period label may be indicated by a numerical value such as a time period from 6:00 to 12:00, a time period from 12:00 to 18:00, a time period from 18:00 to 24:00, etc. [Explanation of symbols]

[0059] 10 Scenario Extraction Device 12 Acquisition Department 14 Labeling Department 16 Granting Department 18 Extraction part 20 Output section 30 Vehicles 35 Front vehicle 40 Road

Claims

1. Acquiring time-series data on a vehicle (30), and items input by a user, including items related to the behavior of the vehicle, items related to the behavior of a forward vehicle (35) located in front of the vehicle, and items related to a road (40) on which the vehicle has traveled; generating a vehicle label classification including items related to the behavior of the vehicle, a forward vehicle label classification including items related to the behavior of the forward vehicle, and a road label classification including items related to the road, according to the items input by the user; assigning to the time-series data the generated items related to the behavior of the vehicle included in the vehicle label classification, the generated items related to the behavior of the forward vehicle included in the forward vehicle label classification, and the generated items related to the road included in the road label classification; A scenario extraction method comprising: extracting data including an item input by the user from the time-series data to which items have been assigned.

2. The items related to the behavior of the vehicle include contents related to the operation of the vehicle and the speed of the vehicle, The items related to the behavior of the forward vehicle include contents related to the operation of the forward vehicle, the speed of the forward vehicle, the directional relationship between the forward vehicle and the vehicle, and the positional relationship between the forward vehicle and the vehicle, 2. The scenario extraction method according to claim 1, wherein the road-related items include details related to road type, motorway portion, general road portion, number of lanes, lane shape changes, road gradient, and curve changes.

3. The vehicle and the forward vehicle are four-wheeled automobiles, Acquire items related to the behavior of a two-wheeled vehicle (45) located in front of the vehicle, the items being input by the user; generating a classification of a front two-wheeled vehicle label including items related to the behavior of the front two-wheeled vehicle according to the items input by the user; The scenario extraction method according to claim 1 or 2, further comprising the step of adding an item related to a behavior of the front two-wheeled vehicle included in a classification of the generated front two-wheeled vehicle label to the time-series data.

4. The scenario extraction method according to claim 3, wherein the items relating to the behavior of the two-wheeled vehicle in front include contents relating to the movement of the two-wheeled vehicle in front, the speed of the two-wheeled vehicle in front, the directional relationship between the two-wheeled vehicle in front and the vehicle, and the positional relationship between the two-wheeled vehicle in front and the vehicle.

5. The vehicle and the forward vehicle are four-wheeled automobiles, Acquire items related to the behavior of a front bicycle (50) located in front of the vehicle input by the user; generating a classification of a bicycle-in-front label that includes items related to the behavior of the bicycle in front according to the items input by the user; 3. The method of claim 1, further comprising the step of adding an item related to a behavior of the bicycle in front, which is included in the classification of the generated bicycle in front label, to the time-series data.

6. A scenario extraction method as described in claim 5, wherein the items related to the behavior of the bicycle in front include information related to the movement of the bicycle in front, the speed of the bicycle in front, the directional relationship between the bicycle in front and the vehicle, and the positional relationship between the bicycle in front and the vehicle.

7. Acquire items related to the behavior of a pedestrian (55) located in front of the vehicle, the items being input by the user; generating a classification of a pedestrian ahead label including items related to the behavior of the pedestrian ahead according to the items input by the user; The scenario extraction method according to claim 1 , further comprising the step of adding, to the time-series data, an item related to a behavior of the pedestrian ahead, which is included in a classification of the generated pedestrian ahead label.

8. The scenario extraction method according to claim 7 , wherein the items related to the behavior of the pedestrian in front include contents related to the movement of the pedestrian in front, the speed of the pedestrian in front, a directional relationship between the pedestrian in front and the vehicle, and a positional relationship between the pedestrian in front and the vehicle.

9. acquiring items related to a traffic scene, which is a scene in which the vehicle is traveling, input by the user; generating a classification of traffic labels that encompasses the items related to the traffic scene in response to the items entered by the user; The method of claim 1 , further comprising the step of adding an item related to the traffic scene included in the generated classification of the traffic label to the time-series data.

10. The scenario extraction method according to claim 9 , wherein the items relating to the traffic scene include information relating to the presence or absence and color of traffic lights.

11. Acquire items related to a time period during which the vehicle traveled, which are input by the user; generating a classification of time zone labels that includes items related to the time zone in accordance with the items input by the user; The method for extracting a scenario according to claim 1 , further comprising the step of adding an item related to the time period included in the classification of the generated time period label to the time series data.

12. an acquisition unit (12) that acquires time-series data on a vehicle (30), and items input by a user, the items relating to the behavior of the vehicle, the items relating to the behavior of a forward vehicle (35) located in front of the vehicle, and the items relating to a road (40) on which the vehicle has traveled; a labeling unit (14) that generates a vehicle label classification including items related to the behavior of the vehicle, a forward vehicle label classification including items related to the behavior of the forward vehicle, and a road label classification including items related to the road, according to the items input by the user; an assignment unit (16) that assigns to the time-series data an item related to the behavior of the vehicle included in the classification of the vehicle label, an item related to the behavior of the forward vehicle included in the classification of the forward vehicle label, and an item related to the road included in the classification of the road label, which are generated by the labeling unit; an extraction unit (18) that extracts data including the item input by the user from the time-series data to which the item is assigned by the assignment unit; A scenario extraction device comprising:

13. The scenario extraction device, an acquisition unit (12) that acquires time-series data on a vehicle (30), and items input by a user, the items relating to the behavior of the vehicle, the items relating to the behavior of a forward vehicle (35) located in front of the vehicle, and the items relating to a road (40) on which the vehicle has traveled; a labeling unit (14) for generating a vehicle label classification including items related to the behavior of the vehicle, a forward vehicle label classification including items related to the behavior of the forward vehicle, and a road label classification including items related to the road, in accordance with the items input by the user; an assignment unit (16) that assigns to the time-series data an item related to the behavior of the vehicle included in the classification of the vehicle label, an item related to the behavior of the forward vehicle included in the classification of the forward vehicle label, and an item related to the road included in the classification of the road label, which are generated by the labeling unit; and a scenario extraction program that causes the program to function as an extraction unit (18) that extracts data including the item input by the user from the time-series data to which the item has been assigned by the assignment unit;

Images