Method for outputting recognition information, program, recognition information output device, and deep learning model

Abstract

Obtain accurate recognition information using a simplified method. [Solution] The method for outputting recognition information is used in the evaluation of autonomous driving of a vehicle equipped with sensors and is executed by a computer. The method includes a learning step S20 in which object information, including the relative position and size of each of one or more moving objects located around the vehicle with respect to the sensor, and environmental conditions indicating the environment around the vehicle are input to train a deep learning model 40M that outputs recognition information relating to the recognition results of one or more objects by the sensor, an input step S30 in which object information and environmental conditions are input to the deep learning model, and an output step S40 in which the deep learning model outputs recognition information.

Description

Technical Field

[0001] The present invention relates to a method for outputting recognition information, a program, a recognition information output device, and a deep learning model.

Background Art

[0002] Currently, research on the automatic driving of moving bodies such as vehicles is actively underway. Simulation technologies for evaluating control methods and the like in automatic driving have been developed (see, for example, Patent Document 1). As a result, it is possible to safely evaluate control methods and the like of automatic driving without actually performing automatic driving of a vehicle on the road.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the above-described simulation, recognition information indicating the result of recognizing moving bodies such as other vehicles and pedestrians located around a vehicle performing automatic driving by a sensor is required. Such recognition information may differ from the result of actually recognizing a moving body by a sensor. For example, in a simulation, a moving body that cannot actually be recognized by a sensor may be recognized. Specifically, correct recognition information may not be obtained due to insufficient evaluation of the degree of shielding of a moving body from a sensor by a building or the like, weather, time, and the like. For example, although it is also possible to obtain highly accurate recognition information by calculation using a recognition algorithm corresponding to the recognition procedure of a sensor, it is difficult to perform a simulation in real time because the calculation takes time.

[0005] This invention has been made in view of the above problems, and aims to acquire accurate recognition information in a simplified manner. [Means for solving the problem]

[0006] To achieve the above objective, a recognition information output method according to one aspect of the present invention is used in the evaluation of autonomous driving of a vehicle equipped with sensors and is executed by a computer, and includes a learning step in which object information, including the relative position and size of each of one or more moving objects located around the vehicle with respect to the sensor, and environmental conditions indicating the environment around the vehicle are input to a deep learning model that outputs the recognition information relating to the recognition results of the one or more objects by the sensor, input step in which the object information and the environmental conditions are input to the deep learning model, and output step in which the deep learning model outputs the recognition information.

[0007] Furthermore, in order to achieve the above objective, a program according to one aspect of the present invention is a program that causes a computer to execute the above-mentioned method for outputting recognition information.

[0008] Furthermore, in order to achieve the above objective, a recognition information output device according to one aspect of the present invention is a recognition information output device that outputs recognition information used in the evaluation of autonomous driving of a vehicle equipped with sensors, comprising: an object information input unit into which object information including the relative position and size of each of one or more moving objects located around the vehicle with respect to the sensor is input; an environmental condition input unit into which environmental conditions indicating the environment around the vehicle are input; and a recognition information generation unit that generates the recognition information using a deep learning model into which the object information and the environmental conditions are input and which outputs the recognition information relating to the recognition results of the one or more objects by the sensor.

[0009] Furthermore, in order to achieve the above objective, a deep learning model according to one aspect of the present invention is used in the evaluation of autonomous driving of a vehicle equipped with sensors, and is a deep learning model executed by a computer that takes as input object information, including the relative position and size of each of one or more moving objects located around the vehicle with respect to the sensor, and environmental conditions indicating the environment around the vehicle, and outputs recognition information relating to the recognition results of the one or more objects by the sensor.

[0010] These comprehensive or specific embodiments may be implemented as a system, method, integrated circuit, computer program, or recording medium such as a non-temporary computer-readable CD-ROM, or as any combination of a system, method, integrated circuit, computer program, and recording medium. [Effects of the Invention]

[0011] According to the method for outputting recognition information of the present invention, accurate recognition information can be obtained in a simplified manner. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing an example of the configuration of a recognition information output device according to an embodiment. [Figure 2] This figure shows the input and output information of the deep learning model according to the embodiment. [Figure 3] This is a flowchart showing the method for outputting recognition information according to the embodiment. [Figure 4] This image corresponds to the object information and environmental conditions related to the embodiment. [Figure 5] This figure shows a portion of the recognition information related to the embodiment. [Figure 6] This graph shows the relationship between confidence level, which is an example of recognition information according to the embodiment, and the distance from the sensor to the object and the shielding rate. [Figure 7]A graph showing the relationship between the confidence level, which is an example of recognition information according to the embodiment, and the width and occlusion rate of a rectangular frame corresponding to the size of an object. [Figure 8] A flowchart showing an example of a simulation method for automatic driving of a vehicle according to the embodiment. [Figure 9] A schematic plan view showing the traffic environment in which a vehicle performing automatic driving travels. [Figure 10] A first plan view showing the results of simulation at each timing of automatic driving of a vehicle. [Figure 11] A second plan view showing the results of simulation at each timing of automatic driving of a vehicle.

Mode for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings.

[0014] Note that each of the embodiments described below shows comprehensive or specific examples. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, steps, order of steps, etc. shown in the following embodiments are examples and are not intended to limit the present invention. In addition, among the components in the following embodiments, components not described in the independent claims indicating the most general concept are described as optional components.

[0015] Also, each figure is a schematic diagram and is not necessarily drawn precisely. In each figure, the same reference numerals are given to the same components.

[0016] (Embodiment) An recognition information output device, a method for outputting recognition information, and a deep learning model according to the embodiment will be described.

[0017] [Recognition Information Output Device] The recognition information output device according to this embodiment will be described with reference to FIG. 1. FIG. 1 is a block diagram showing an example of the configuration of the recognition information output device 10 according to this embodiment.

[0018] The recognition information output device 10 is a device that outputs recognition information used in the evaluation of the automatic driving of a vehicle equipped with sensors. The recognition information output device 10 according to this embodiment outputs recognition information using the recognition information output method according to this embodiment. The recognition information output device 10 is used to simulate the recognition results of the sensors provided in the vehicle in the simulation of the automatic driving of the vehicle in the traffic environment. As the sensors, for example, LiDAR (Light Detection and Ranging), cameras, etc. can be used. Hereinafter, the vehicle performing automatic driving will also be referred to as the host vehicle.

[0019] The recognition information output device 10 outputs recognition information related to the recognition results of one or more objects, which are moving bodies around the sensor, by the sensor, based on information on the traffic environment around the sensor. The recognition information includes, for example, the relative position of each of the one or more objects recognized by the sensor with respect to the sensor, and the confidence level indicating the certainty of the recognition results of each of the one or more objects by the sensor. The recognition information may include error information regarding the size of the object. In this embodiment, as shown in FIG. 1, the recognition information output device 10 includes an object information input unit 20, an environmental condition input unit 30, a recognition information generation unit 40, and an output unit 50.

[0020] The object information input unit 20 is an input unit to which object information including the relative position and size of each of one or more objects, which are moving bodies located around the host vehicle, with respect to the sensor is input. The object information may include the type of each of the one or more objects. The types of the one or more objects are, for example, automobiles, motorcycles, pedestrians, trains, etc. The object information input unit 20 outputs the input object information to the recognition information generation unit 40.

[0021] The relative position of one or more objects with respect to each sensor may include, for example, the (absolute) position of one or more objects on the map corresponding to the area where the vehicle is located, and the (absolute) position of the vehicle on the map. The relative position of each object with respect to its sensor can be obtained from the absolute position of each object and the absolute position of the vehicle.

[0022] Furthermore, the relative position of each of the one or more objects with respect to the sensor may also be the relative position of each of the one or more objects with respect to the vehicle itself. By having the recognition information output device 10 possess information indicating the relative position of the vehicle with respect to the sensor, the relative position of an object with respect to the vehicle can be used as information indicating the relative position of an object with respect to the sensor.

[0023] The recognition information output device 10 may also infer the type of each of the one or more objects based on the size of each of the one or more objects.

[0024] The environmental condition input unit 30 is an input unit that receives environmental conditions indicating the environment around the vehicle. The environmental conditions include, for example, a shielding rate indicating the degree to which each of the one or more objects viewed from the sensor is shielded by structures located around the vehicle. Structures located around the vehicle include, for example, buildings and signs around the road on which the vehicle is traveling. Environmental conditions may also include, for example, weather and date and time. Depending on the weather and date and time, the difficulty of recognizing objects from the sensor will differ. The environmental condition input unit 30 outputs the input environmental conditions to the recognition information generation unit 40.

[0025] The recognition information generation unit 40 is a processing unit that generates recognition information using a deep learning model. In this embodiment, object information and environmental conditions are input to the recognition information generation unit 40, and the generated recognition information is output to the output unit 50. The deep learning model used by the recognition information generation unit 40 according to this embodiment will be explained with reference to Figure 2. Figure 2 is a diagram showing the input information and output information of the deep learning model 40M according to this embodiment. The deep learning model 40M is a regression model used in the evaluation of autonomous driving of a vehicle equipped with sensors and is executed by a computer. The recognition information generation unit 40 may have, for example, a non-temporary computer-readable recording medium on which a program that causes the computer to execute the deep learning model 40M is recorded.

[0026] As shown in Figure 2, the deep learning model 40M receives object information and environmental conditions as input and outputs recognition information. The deep learning model 40M used by the recognition information generation unit 40 is a pre-trained model. In training the deep learning model 40M, object information and environmental conditions, and training recognition information, which is information related to the recognition results of one or more objects, are used as training data.

[0027] Here, the teacher recognition information is recognition information obtained when one or more objects are recognized using a recognition method corresponding to the operation of the sensor when the sensor recognizes one or more objects. In this embodiment, the recognition method used to obtain the teacher recognition information is a method of recognizing one or more objects using a recognition algorithm corresponding to the operation of the sensor when it recognizes one or more objects. For example, if the sensor is a LiDAR, the teacher recognition information corresponding to the recognition result of one or more objects can be obtained from the input object information and environmental conditions by simulating with a recognition algorithm that corresponds to acquiring depth images by LiDAR and extracting objects from depth images.

[0028] Furthermore, the recognition method used to obtain teacher recognition information may be a method that actually recognizes one or more objects using sensors. For example, a vehicle may be driven in a predetermined traffic environment, and the recognition information corresponding to the recognition results of one or more objects recognized by the vehicle's sensors may be used as teacher recognition information.

[0029] The output unit 50 shown in Figure 1 outputs recognition information. The output unit 50 outputs the recognition information input from the recognition information generation unit 40. The output unit 50 outputs the recognition information to, for example, an autonomous driving evaluation device.

[0030] [How to output recognition information] The method for outputting recognition information according to this embodiment will be explained using Figure 3. Figure 3 is a flowchart showing the method for outputting recognition information according to this embodiment. The method for outputting recognition information according to this embodiment is used in the evaluation of autonomous driving of a vehicle equipped with sensors and is executed by a computer.

[0031] As shown in Figure 3, in the recognition information output method according to this embodiment, first, teacher recognition information to be used for training the deep learning model 40M is acquired (teacher recognition information acquisition step S10). As described above, teacher recognition information is recognition information relating to the recognition results of one or more objects. More specifically, teacher recognition information is recognition information obtained when one or more objects are recognized using a recognition method corresponding to the operation of the sensor when the sensor recognizes one or more objects. In this embodiment, it is recognition information obtained when one or more objects are recognized using a recognition algorithm corresponding to the operation of the sensor when it recognizes one or more objects, and a recognition method for recognizing one or more objects is used. Note that the recognition method used to obtain teacher recognition information may be a method that actually recognizes one or more objects using the sensor.

[0032] Next, the deep learning model 40M is trained (training step S20). The deep learning model 40M is a model that receives object information, including the relative position and size of each of one or more moving objects located around the vehicle, and environmental conditions indicating the environment around the vehicle, and outputs recognition information related to the recognition results of one or more objects by the sensors. In this embodiment, the deep learning model 40M is trained using object information and environmental conditions, and training recognition information, which is information related to the recognition results of one or more objects, as training data.

[0033] Next, object information and environmental conditions are input to the deep learning model 40M, which has been trained in the learning step S20 (input step S30).

[0034] Next, the deep learning model 40M outputs the recognition information (output step S40).

[0035] This allows recognition information to be output.

[0036] Note that Figure 3 shows the minimum configuration of the recognition information output method, but the configuration of the recognition information output method is not limited to this. For example, in the recognition information output method, the input step S30 and output step S40 may be executed repeatedly after the execution of the learning step S20. Also, the learning step S20 only needs to be executed at least once. In other words, the learning step S20 may be performed only once, or the learning step S20 may be executed using new training data after the execution of the input step S30 and output step S40.

[0037] [effect] This document will describe the effects of the recognition information output device 10, the recognition information output method, and the deep learning model 40M according to this embodiment.

[0038] As described above, the recognition information output device 10 according to this embodiment is a device that outputs recognition information used in the evaluation of autonomous driving of a vehicle equipped with sensors. The recognition information output device 10 comprises an object information input unit into which object information, including the relative position and size with respect to the sensor, of one or more moving objects located around the vehicle is input; an environmental condition input unit into which environmental conditions indicating the environment around the vehicle are input; and a recognition information generation unit that generates recognition information using a deep learning model 40M that receives the object information and environmental conditions and outputs recognition information relating to the recognition results of one or more objects by the sensor.

[0039] Thus, the recognition information output device 10 according to this embodiment can acquire highly accurate recognition information using a simplified method with a deep learning model 40M. Therefore, it becomes possible to acquire highly accurate recognition information in real time in applications such as autonomous driving simulations.

[0040] Furthermore, the recognition information output method according to this embodiment is used in the evaluation of autonomous driving of a vehicle equipped with sensors and is a method executed by a computer. The recognition information output method includes a learning step S20 in which object information, including the relative position and size of each of one or more moving objects located around the vehicle with respect to the sensor, and environmental conditions indicating the environment around the vehicle are input to train a deep learning model 40M that outputs recognition information relating to the recognition results of one or more objects by the sensor; an input step S30 in which object information and environmental conditions are input to the deep learning model 40M; and an output step S40 in which the deep learning model 40M outputs recognition information.

[0041] This method of outputting recognition information achieves the same effect as the recognition information output device 10.

[0042] Furthermore, the deep learning model 40M according to this embodiment is a regression model executed by a computer and is used in the evaluation of autonomous driving of a vehicle equipped with sensors. The deep learning model 40M receives object information, including the relative position and size of one or more moving objects located around the vehicle, and environmental conditions indicating the environment around the vehicle, and outputs recognition information related to the recognition results of one or more objects by the sensors.

[0043] This deep learning model 40M achieves the same effect as the recognition information output device 10.

[0044] Furthermore, the object information may include each of one or more types of objects.

[0045] Since the recognition performance characteristics of a sensor can differ depending on the type of object, including object information for each of the one or more types of objects allows for the acquisition of more accurate recognition information.

[0046] Furthermore, the recognition information may include a confidence level indicating the certainty of each of the recognition results for one or more objects by the sensor.

[0047] This allows for accurate determination of whether or not one or more objects can be recognized, based on the recognition information.

[0048] Furthermore, the environmental conditions may include a shielding rate indicating the degree to which each of the one or more objects viewed from the sensor is shielded by structures located around the vehicle.

[0049] This allows the influence of surrounding structures on the sensor to be accurately reflected in the recognition information when the sensor recognizes an object.

[0050] Furthermore, in the learning step S20 of the method for outputting recognition information, the deep learning model 40M may be trained using object information, environmental conditions, and training recognition information, which is information relating to the recognition results of one or more objects, as training data. Here, the training recognition information may be recognition information obtained when one or more objects are recognized using a recognition method corresponding to the operation of the sensor when the sensor recognizes one or more objects.

[0051] This makes it possible to acquire recognition information that faithfully reflects the characteristics of the sensor.

[0052] Furthermore, the recognition method used to acquire teacher recognition information may be a method that recognizes one or more objects using a recognition algorithm that corresponds to the operation of a sensor when it recognizes one or more objects, or it may be a method that actually recognizes one or more objects using a sensor.

[0053] These recognition methods allow us to obtain training recognition information that faithfully reflects the characteristics of the sensor. Therefore, we can realize a deep learning model 40M, which is a regression model that can obtain highly accurate recognition information.

[0054] [Example of recognition information output] An example of recognition information output by the deep learning model 40M used in the recognition information output device 10 and the recognition information output method according to this embodiment will be explained using Figures 4 to 7. Figure 4 is an image corresponding to the object information and environmental conditions according to this embodiment. Figure 4 shows an example of an image of the area in front of the vehicle as seen from the sensor. Figure 5 is a diagram showing a part of the recognition information according to this embodiment. Figure 6 is a graph showing the relationship between confidence level, which is an example of recognition information according to this embodiment, and the distance from the sensor to the object and the occlusion rate. Figure 7 is a graph showing the relationship between confidence level, which is an example of recognition information according to this embodiment, and the width of the rectangular frame corresponding to the size of the object and the occlusion rate. The horizontal axis in Figure 7 represents the width of the rectangular frame. The unit of the width of the rectangular frame is the number of pixels ([px]) of the sensor.

[0055] Based on the object information input to the deep learning model 40M according to this embodiment, information indicating the relative position between the sensor and one or more objects V0 is obtained, as shown in Figure 4. Note that in Figure 4, objects V0 that are obscured by obstacles such as buildings are not shown, but information indicating the occlusion rate of each of the one or more objects V0 is obtained based on the environmental conditions. The deep learning model 40M, to which the object information and environmental conditions have been input, obtains recognition information indicating the recognition result of the one or more objects V0 shown in Figure 4. For example, if the sensor is a LiDAR, the object V0 recognized by the sensor is surrounded by a rectangular frame, as shown in Figure 5. Thus, a rectangular frame surrounding the object V0 is obtained as recognition information.

[0056] Furthermore, the deep learning model 40M, through learning, generates a relationship between the confidence score indicating the certainty of object recognition and the relative position between the sensor and one or more objects V0, the rectangular frame width, and the occlusion rate of object V0, as shown in Figures 6 and 7. As shown in Figure 6, the confidence score decreases as the distance between the sensor and object V0 increases, and as the occlusion rate increases. As shown in Figure 7, the confidence score decreases as the rectangular frame width corresponding to object V0 decreases, and as the occlusion rate increases.

[0057] The deep learning model 40M outputs recognition information indicating the confidence level of sensor recognition for each object V0, based on the relationships shown in Figures 6 and 7, the distance from the sensor to each object V0, the width of the rectangular frame for each object, and the occlusion rate for each object.

[0058] [Methods for evaluating autonomous driving] An example of applying the recognition information obtained by the recognition information output device 10 and the recognition information output method according to this embodiment to a simulation for evaluating autonomous driving will be explained with reference to Figure 8. Figure 8 is a flowchart showing an example of a simulation method for autonomous driving of a vehicle according to this embodiment.

[0059] As shown in Figure 8, in the simulation of autonomous vehicle driving, the recognition information output device 10 first acquires information about objects and environmental conditions around the vehicle (first acquisition step S110). Prior to the simulation of autonomous vehicle driving, a learning step S20 for the method of outputting recognition information is performed, and the trained deep learning model 40M is used in the simulation of autonomous vehicle driving.

[0060] Next, the recognition information output from the recognition information output device 10 is acquired (second acquisition step S120). The second acquisition step S120 corresponds to the input step S30 and output step S40 of the recognition information output method described above. In other words, in the second acquisition step S120, the object information and environmental conditions acquired in the first acquisition step S110 are input to the trained deep learning model 40M, thereby acquiring the recognition information output from the trained deep learning model 40M.

[0061] Next, based on the recognition information acquired in the second acquisition step S120, the vehicle's driving control is performed (driving control step S130). Specifically, the vehicle's driving path, speed, etc., are controlled to avoid contact (and unnecessary proximity) with one or more objects V0 in the vicinity of the vehicle.

[0062] By repeating each of the above steps, it is possible to evaluate the vehicle's driving control.

[0063] [Examples of autonomous driving evaluations] Next, we will explain examples of autonomous driving evaluation using Figures 9 to 11. Figure 9 is a schematic plan view showing the traffic environment in which the autonomous vehicle V1 travels. Figures 10 and 11 are plan views showing the simulation results at various timings of autonomous driving for vehicle V1.

[0064] As shown in Figure 9, in the traffic environment on which vehicle V1 travels, a road R1 extending in a direction close to north-south intersects with a road R2 extending in an east-west direction. Road R2 is wider than road R1, and objects V0 such as vehicles traveling on road R2 have priority to enter the intersection with road R1. Vehicle V1 traveling on road R1 stops at the stop line before the intersection with road R2, and enters the intersection after recognizing, based on the sensor's recognition results, that there are no vehicles or other objects on road R2. There are obstacles such as buildings S1 and S2 around the intersection. Below, using Figure 10, the simulation results for when vehicle V1 travels north on road R1 will be explained, and using Figure 11, the simulation results for when vehicle V1 travels south on road R1 will be explained.

[0065] When vehicle V1 travels north on road R1, as shown in the plan view (a) of Figure 10, vehicle V1 stops temporarily before the intersection of road R1 and road R2. The sensor checks for the presence or absence of a recognizable object in the vicinity of vehicle V1, including road R2. Here, the presence or absence of a recognizable object is determined based on the recognition information output using the recognition information output device 10 and recognition information output method according to this embodiment. More specifically, the recognition information output device 10 and recognition information output method acquire recognition information based on object information and environmental conditions, and the presence or absence of a recognizable object is determined based on this recognition information. In the example shown in Figure 10, the object information includes the relative position of object V0 with respect to the sensor and the size of object V0. The environmental conditions include conditions such as the relative position of obstacles S1, S2, etc., with respect to the sensor.

[0066] Next, as shown in the plan view (b) of Figure 10, object V0 traveling on road R2 is recognized based on the recognition information. Object V0 approaches the intersection from the right side as seen from the sensor. Based on this recognition information, vehicle V1 waits for object V0 to pass the intersection without entering it. In the example shown in Figure 10, there is no obstruction between vehicle V1 and object V0, so object V0 does not enter the blind spot of an obstruction. For this reason, the confidence level of recognition of object V0 indicated by the recognition information is high.

[0067] Next, as shown in the plan view (c) of Figure 10, vehicle V1 proceeds through the intersection after confirming, based on the recognition information, that object V0 has passed through the intersection and that no other objects are traveling on road R2.

[0068] As described above, when vehicle V1 travels north on road R1, the recognition information can be used to simulate the autonomous driving of vehicle V1 without actually simulating the operation of the sensors.

[0069] When vehicle V1 travels south on road R1, as shown in the plan view (a) of Figure 11, vehicle V1 comes to a temporary stop before the intersection of road R1 and road R2. The sensor checks for the presence or absence of recognizable objects in the vicinity of vehicle V1, including road R2, as in the case of Figure 10.

[0070] Next, as shown in the plan view (b) of Figure 11, object V0 approaches the intersection from the left side of road R2. However, from the perspective of the sensors on vehicle V1, object V0 is completely obscured by the obstruction S1. In other words, the entire object V0 is in the blind spot of the obstruction S1. Therefore, the confidence level of recognition of object V0 in the recognition information is 0. That is, the sensor cannot recognize object V0.

[0071] Next, as shown in the plan view (c) of Figure 11, object V0 approaches the intersection further, and a portion of object V0 appears outside the blind spot of the occluder S1. However, because the occluding rate of object V0 is still high, the confidence in recognizing object V0 is low. Therefore, the sensor cannot recognize object V0.

[0072] Next, as shown in the plan view (d) of Figure 11, object V0 moves further closer to the intersection, and the proportion of object V0 located outside the blind spot of the occluder S1 increases. Consequently, the occluding rate of object V0 decreases, and the confidence in recognizing object V0 increases. Therefore, the sensor can recognize object V0.

[0073] Next, as shown in the plan view (e) of Figure 11, object V0 passes through the intersection. Since object V0 is not obstructed, the sensor can recognize that it has passed through the intersection. However, because there is an obstruction S1 near the intersection, the line of sight to vehicle V1, and therefore to the sensor, is poor, and vehicle V1 remains unable to start moving.

[0074] In this way, even when there are obstacles S1 or other objects around the vehicle V1, the recognition information can be used to simulate the autonomous driving of the vehicle V1 without actually simulating the operation of the sensors.

[0075] (Other variations, etc.) Although a recognition information output device, etc., according to one aspect of the present invention has been described above based on embodiments, the present invention is not limited to these embodiments. Various modifications to the embodiments that a person skilled in the art could conceive of may also be included within the scope of the present invention, as long as they do not depart from the spirit of the present invention.

[0076] Furthermore, the following forms may also be included within the scope of one or more aspects of this disclosure.

[0077] (1) At least some of the components included in the recognition information output device 10 described above may be a computer system or a network-connected server consisting of a microprocessor, ROM, RAM, hard disk unit, etc. A computer program is stored in the RAM or hard disk unit. The microprocessor achieves its function by operating in accordance with the computer program. Here, the computer program is composed of a combination of multiple instruction codes that indicate commands to the computer in order to achieve a predetermined function.

[0078] (2) At least some of the components included in the recognition information output device 10 described above may be made up of a single system LSI (Large Scale Integration). The system LSI is a multi-functional LSI manufactured by integrating multiple components onto a single chip, and specifically, it is a computer system that includes a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The system LSI achieves its function by operating the microprocessor in accordance with the computer program.

[0079] (3) At least some of the components included in the recognition information output device 10 described above may consist of an IC card or a standalone module that can be attached to or removed from each device. The IC card or module is a computer system consisting of a microprocessor, ROM, RAM, etc. The IC card or module may include the multi-functional LSI described above. The IC card or module achieves its function by operating the microprocessor in accordance with a computer program. The IC card or module may be tamper-resistant.

[0080] (4) Furthermore, at least a portion of the components included in the recognition information output device 10 described above may be a computer program or a digital signal recorded on a recording medium that can be read by a computer, such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray® Disc), semiconductor memory, etc. Alternatively, the digital signal may be recorded on one of these recording media.

[0081] Furthermore, at least some of the components included in the recognition information output device 10 described above may transmit the computer program or the digital signal via a telecommunications line, a wireless or wired communication line, a network such as the Internet, data broadcasting, etc.

[0082] (5) The disclosure may be a method for outputting recognition information as described above. Alternatively, it may be a computer program for implementing the above method for outputting recognition information using a computer, or it may be a digital signal consisting of the computer program. Furthermore, the disclosure may be implemented as a non-temporary computer-readable recording medium such as a CD-ROM on which the computer program is recorded.

[0083] (6) The Disclosure may also provide a computer system comprising a microprocessor and memory, wherein the memory stores the computer program, and the microprocessor operates in accordance with the computer program.

[0084] (7) Alternatively, the program or the digital signal may be carried out by another independent computer system by recording it on the recording medium and transferring it, or by transferring the program or the digital signal via the network or the like.

[0085] (8) The above embodiments and the above modified examples may be combined.

[0086] (Note) Furthermore, the following technologies are disclosed based on the above description.

[0087] (Technology 1) A method for outputting recognition information, which is used in the evaluation of autonomous driving of a vehicle equipped with sensors and is performed by a computer, comprising: a learning step in which object information, including the relative position and size of each of one or more moving objects located around the vehicle with respect to the sensor, and environmental conditions indicating the environment around the vehicle are input to a deep learning model that outputs the recognition information relating to the recognition results of the one or more objects by the sensor, is trained; an input step in which the object information and the environmental conditions are input to the deep learning model; and an output step in which the deep learning model outputs the recognition information.

[0088] (Technical 2) The method for outputting recognition information according to Technical 1, wherein the object information includes each of the one or more types of objects.

[0089] (Technical 3) The method for outputting recognition information according to Technical 1 or 2, wherein the recognition information includes a confidence level indicating the certainty of each of the one or more objects recognized by the sensor.

[0090] (Technology 4) A method for outputting recognition information according to any one of Techniques 1 to 3, wherein the environmental conditions include a shielding rate indicating the degree to which each of the one or more objects is shielded by a structure located around the vehicle when viewed from the sensor.

[0091] (Technical 5) A method for outputting recognition information according to any one of Technical 1 to 4, wherein in the learning step, the deep learning model is trained using the object information, the environmental conditions, and the training recognition information which is information relating to the recognition results of one or more objects as training data, and the training recognition information is the recognition information obtained when the sensor recognizes one or more objects using a recognition method corresponding to the operation of the sensor when the sensor recognizes one or more objects.

[0092] (Technical 6) The recognition method is a method for outputting recognition information according to Technical 5, wherein the recognition method is a method for recognizing one or more objects using a recognition algorithm that corresponds to the operation of the sensor when it recognizes one or more objects.

[0093] (Technical 7) The recognition method is a method of actually recognizing one or more objects using the sensor, as described in Technical 5, for outputting recognition information.

[0094] (Technology 8) A program that causes a computer to execute one of the recognition information output methods described in Technology 1 to 7.

[0095] (Technology 9) A recognition information output device that outputs recognition information used in the evaluation of autonomous driving of a vehicle equipped with sensors, comprising: an object information input unit that receives object information including the relative position and size of each of one or more moving objects located around the vehicle with respect to the sensor; an environmental condition input unit that receives environmental conditions indicating the environment around the vehicle; and a recognition information generation unit that receives the object information and the environmental conditions and generates the recognition information using a deep learning model that outputs the recognition information relating to the recognition results of the one or more objects by the sensor.

[0096] (Technology 10) A deep learning model used in the evaluation of autonomous driving of a vehicle equipped with sensors, which is executed by a computer, and which takes as input object information including the relative position and size of each of one or more moving objects located around the vehicle with respect to the sensor, and environmental conditions indicating the environment around the vehicle, and outputs recognition information relating to the recognition results of the one or more objects by the sensor. [Industrial applicability]

[0097] A method for outputting recognition information according to one aspect of the present invention can be applied, for example, to an evaluation device for the autonomous driving of a vehicle. [Explanation of symbols]

[0098] 10 Recognition Information Output Device 20. Object Information Input Unit 30 Environmental Condition Input Section 40 Recognition information generation unit 40M Deep Learning Model 50 Output section R1, R2 road S1, S2 shield V0 object V1 Vehicle

Claims

1. A method for outputting recognition information, which is used in the evaluation of autonomous driving of a vehicle equipped with sensors and is executed by a computer, A learning step in which object information, including the relative position and size of each of one or more moving objects located around the vehicle with respect to the sensor, and environmental conditions indicating the environment around the vehicle are input, and a deep learning model is trained to output recognition information relating to the recognition results of the one or more objects by the sensor, An input step in which the object information and the environmental conditions are input to the deep learning model, This includes an output step that causes the deep learning model to output the recognition information. Method for outputting recognition information.

2. The object information includes each of the one or more types of objects. The method for outputting recognition information according to claim 1.

3. The recognition information includes a confidence level indicating the certainty of each of the one or more objects recognized by the sensor. A method for outputting recognition information according to claim 1 or 2.

4. The aforementioned environmental conditions include a shielding ratio that indicates the degree to which each of the one or more objects is shielded by structures located around the vehicle when viewed from the sensor. A method for outputting recognition information according to claim 1 or 2.

5. In the learning step described above, the deep learning model is trained using the object information, the environmental conditions, and the training recognition information, which is information relating to the recognition results of one or more objects, as training data. The aforementioned teacher recognition information is the recognition information obtained when the sensor recognizes one or more objects using a recognition method corresponding to the operation of the sensor when the sensor recognizes one or more objects. A method for outputting recognition information according to claim 1 or 2.

6. The recognition method is a method for recognizing one or more objects using a recognition algorithm that corresponds to the operation of the sensor when it recognizes one or more objects. The method for outputting recognition information according to claim 5.

7. The recognition method is a method of actually recognizing one or more objects using the sensor. The method for outputting recognition information according to claim 5.

8. To cause a computer to execute the method for outputting recognition information described in claim 1 or 2. program.

9. A recognition information output device that outputs recognition information used in the evaluation of autonomous driving of a vehicle equipped with sensors, An object information input unit receives object information, including the relative position and size of each of one or more moving objects located around the vehicle, with respect to the sensor. An environmental condition input unit into which environmental conditions indicating the environment around the vehicle are input, The system includes a recognition information generation unit that receives the object information and environmental conditions as input and generates the recognition information using a deep learning model that outputs the recognition information relating to the recognition results of one or more objects by the sensor. Recognition information output device.

10. A deep learning model used in the evaluation of autonomous driving of vehicles equipped with sensors, and executed by a computer, Object information, including the relative position and size of each of the one or more moving objects located around the vehicle, and environmental conditions indicating the environment around the vehicle are input, and recognition information relating to the recognition results of the one or more objects by the sensor is output. Deep learning models.

Images