Information processing device, information processing method, and program

Abstract

To generate a high-accuracy surrounding map during vehicle travel.SOLUTION: An information processing device mounted on a vehicle comprises a control unit 110 configured to: acquire data via a sensor provided on the vehicle; input the data to a first model which is a trained estimation model for estimating information relating to an object for generating a road map as feature information, and estimate the feature information; input the data to a second model which is a trained estimation model for estimating topology information which is lane topology of a road or topology of an object or a lane, and estimate the topology information; and generate a road map of an area around the vehicle on the basis of the feature information and the topology information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates to map generation. [Background technology]

[0002] There is a technology for automatically generating a map. For example, Patent Document 1 discloses a mobile object that stores a preliminary map indicating the probability density of the presence of an object in each small area, generates a current map indicating the presence or absence of the object in each small area, and estimates its own position based on the preliminary map and the current map. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-173688 [Patent Document 2] Patent No. 7063310 [Patent Document 3] Japanese Patent Application Laid-Open No. 2019-168610 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure aims to generate a highly accurate surrounding map while a vehicle is traveling. [Means for solving the problem]

[0005] One aspect of the present disclosure is An information processing device mounted on a vehicle, the information processing device comprising a control unit that executes the following operations: acquiring data via a sensor provided on the vehicle; inputting the data into a first model, which is a trained estimation model that estimates information about objects for generating a road map as feature information, to estimate the feature information; inputting the data into a second model, which is a trained estimation model for estimating topology information, which is the topology of road lanes or the topology of the objects and the lanes, to estimate the topology information; and generating the road map around the vehicle based on the feature information and the topology information.

[0006] Another aspect of the present disclosure is An information processing method executed by an information processing device mounted on a vehicle, the information processing method including the steps of: acquiring data via a sensor provided on the vehicle; inputting the data into a first model, which is a trained estimation model that estimates information about objects for generating a road map as feature information, to estimate the feature information; inputting the data into a second model, which is a trained estimation model for estimating topology information, which is the topology of road lanes or the topology of the objects and the lanes, to estimate the topology information; and generating the road map around the vehicle based on the feature information and the topology information.

[0007] Another aspect is a program for causing a computer to execute the information processing method, or a computer-readable storage medium that non-temporarily stores the program. [Effects of the Invention]

[0008] According to the present disclosure, a highly accurate surrounding area map can be generated while the vehicle is traveling. [Brief explanation of the drawings]

[0009] [Figure 1]FIG. 2 is a diagram showing an outline of processing executed by an in-vehicle device. [Figure 2] FIG. 2 is a diagram illustrating components of the in-vehicle device according to the first embodiment. [Figure 3] 4 is a flowchart of a process executed by a control unit of the in-vehicle device according to the first embodiment. [Figure 4] FIG. 1 is a diagram illustrating feature information and topology information. [Figure 5] 10 is a flowchart of a process executed by a control unit of an in-vehicle device according to a second embodiment. [Figure 6] FIG. 10 is a diagram illustrating components of an in-vehicle device according to a third embodiment. [Figure 7] 10 is a flowchart of a process executed by a control unit of an in-vehicle device according to a third embodiment. [Figure 8] 5A and 5B are diagrams illustrating a case where a control unit of an in-vehicle device determines that an error has been detected. DETAILED DESCRIPTION OF THE INVENTION

[0010] (overview) In an autonomous vehicle, driving control is performed using a map of the surrounding area where the vehicle is traveling.

[0011] For example, in an autonomous vehicle, high-definition maps downloaded from an external device are used for autonomous driving control of the vehicle. In this case, it is necessary to download an appropriate high-definition map each time according to the area in which the vehicle is traveling.

[0012] However, frequent downloads of high-resolution maps in this way pose a problem in that the amount of communication between the external device and the vehicle becomes enormous.

[0013] Furthermore, even if high-resolution maps are downloaded in advance and used, depending on the timing of map updates, there may be cases where the current conditions of roads, etc. do not match the high-resolution maps created in advance.

[0014] In order to reduce the amount of communication between an external device and the vehicle and to reduce discrepancies between the map being used and the current road conditions, it is preferable for the on-board device to generate a map of the area around the vehicle's location in real time based on various sensor data acquired by the vehicle. This is because communication volume can be reduced by eliminating the need to download high-resolution maps with large amounts of data from an external device. Furthermore, using a map generated in real time rather than a pre-created map makes it easier for the current road conditions to be reflected in the map used for autonomous driving control.

[0015] An information processing device according to one aspect of the present disclosure includes: An information processing device mounted on a vehicle, comprising a control unit that executes the following operations: acquiring data via a sensor provided on the vehicle; inputting the data into a first model, which is a trained estimation model that estimates information about objects for generating a road map as feature information, to estimate the feature information; inputting the data into a second model, which is a trained estimation model for estimating topology information, which is the topology of road lanes or the topology of the objects and the lanes, to estimate the topology information; and generating the road map around the vehicle based on the feature information and the topology information.

[0016] Feature information is information about objects that exist on roads and have specific meanings in road maps. Specifically, feature information may be information about road boundaries, white lines on roads (including lane boundaries and stop lines), traffic lights, etc.

[0017] The first model is a trained estimation model for estimating feature information. A trained estimation model is a machine learning model that receives a certain amount of data and then The first model is a model that receives sensor data collected by a vehicle and outputs feature information estimated from the sensor data.

[0018] Topology information is information that indicates the topology between lanes, or the topology between lanes and objects on or near the road. Here, an object may be any of several types of objects that have a specific meaning in a road map. Topology is a mathematical structure that indicates the spatial relationship between objects. In other words, topology information is information that indicates how the lanes that make up a road are connected to each other, and how the lanes are connected to objects on or near the road.

[0019] The second model is a trained estimation model for estimating topology information. The second model is a model that inputs sensor data collected by a vehicle and outputs topology information estimated from the sensor data.

[0020] A road map is a map that displays the road itself around the vehicle, the lanes that make up the road, and objects on the road that have specific meanings (for example, white lines, stop lines, traffic lights, etc.).

[0021] An information processing device according to an aspect of the present disclosure inputs data collected by a vehicle while traveling into a plurality of trained estimation models, estimates feature information and topology information, and generates a road map of the area around the vehicle in real time while traveling based on the estimated feature information and topology information.

[0022] By estimating feature information and using it to generate a road map, an information processing device according to an embodiment of the present disclosure can grasp the general shape of the road and recognize the area in which the vehicle 10 can travel. Furthermore, by specifying topology information and using it to generate a road map, an information processing device according to an embodiment of the present disclosure can recognize the direction in which a vehicle can travel in the lanes of the road whose general shape has been grasped. In other words, an information processing device according to an embodiment of the present disclosure can recognize which lane is a lane in which a vehicle can travel, whether a left turn or a right turn is permitted, or whether a left turn or a right turn is prohibited, etc.

[0023] As a result, an information processing device according to an aspect of the present disclosure can generate a map that includes information such as the trajectory that the vehicle should travel, which cannot be determined solely from feature information based on sensor data.

[0024] The information processing device may further include a storage unit, and the control unit may acquire the first model and the second model from an external device and store the acquired first model and second model in the storage unit.

[0025] The estimation model may be updated (re-learned) by an external device as needed. By configuring the information processing device to be able to acquire the updated estimation model from an external device, the information processing device can use the latest estimation model as needed.

[0026] The data may also include information indicating the position of the vehicle, information indicating the attitude of the vehicle, and an image captured by a camera provided on the vehicle.

[0027] If the position of the vehicle on the road map and the vehicle's attitude relative to the road map coordinates are known, the position in space of the object recognized based on the image can be determined.

[0028] By providing such data as input data to the estimation model, the estimation model can accurately estimate the position of an object, etc. With this configuration, the accuracy of road maps generated by the information processing device can be improved.

[0029] The control unit may also generate control parameters for controlling the behavior of the vehicle in an autonomous driving mode based on the generated road map.

[0030] This configuration allows the vehicle to perform autonomous driving control based on maps generated in real time, which means that autonomous driving control of the vehicle can be achieved without downloading existing high-resolution maps.

[0031] In addition, when the control unit detects an error in the autonomous driving of the vehicle while it is traveling, the control unit may transmit the data including images captured by a camera installed in the vehicle to a specified device.

[0032] An error refers to any event that interferes with autonomous driving. For example, if the control unit detects any error in autonomous driving, it transmits image data that is likely to capture some event that caused the error to a central device that controls the vehicle.

[0033] According to this configuration, the event that caused the error can be notified to a central device that controls the vehicles, thereby assisting in the analysis of the cause of the error.

[0034] The control unit may also determine that the error has been detected if a user of the vehicle cancels the autonomous driving mode, if the sensor does not acquire a predetermined number of data a first number of times during a first period of time, or if a road map that differs from the most recently generated road map by a predetermined percentage or more is generated a second number of times during a second period of time.

[0035] In these cases, it can be assumed that the automated driving control using the automatically generated road map is not working properly, or that there may be a problem with the road map generation.

[0036] Furthermore, the control unit may determine that the error has been detected when a predetermined sensor provided in the vehicle does not acquire GPS (Global Positioning System) information within a predetermined time.

[0037] In other words, an information processing device according to one aspect of the present disclosure may treat a case where the vehicle's location information cannot be obtained at a predetermined timing as an error occurring in the autonomous driving control.

[0038] Furthermore, the road map that differs from the immediately preceding road map by a predetermined percentage or more may be a road map in which the estimation result of the white lines using the first model or the estimation result of the topology of the lane using the second model differs from the immediately preceding road map by a predetermined amount or more.

[0039] As a result, an information processing device according to one aspect of the present disclosure can determine that an error has occurred in the autonomous driving control if the generated road map contains an abnormality.

[0040] The control unit may acquire the re-learned first model and the re-learned second model from an external device when the control unit detects the error a predetermined number of times or more.

[0041] The re-learned first model and second model are, for example, the first model stored in the vehicle. This model is an additionally trained version of the first and second models. If the above-mentioned error occurs more than a predetermined number of times, the first and second models may not be appropriate. In such a case, it is preferable to obtain retrained first and second models from an external device in order to use more appropriate models.

[0042] Specific embodiments of the present disclosure will be described below with reference to the accompanying drawings. Unless otherwise specified, the hardware configuration, module configuration, functional configuration, etc. described in each embodiment are not intended to limit the technical scope of the disclosure to those configurations.

[0043] (First embodiment) [Outline of processing performed by the on-board device] An overview of an information processing device according to a first embodiment will be described with reference to FIG. 1. FIG. 1 is a diagram showing an overview of processing executed by an in-vehicle device 100. The information processing device according to this embodiment is realized as, for example, the in-vehicle device 100. The in-vehicle device 100 is mounted on a vehicle 10 and provides functions such as a car navigation system to a user. The vehicle 10 is typically an autonomous vehicle and can communicate with external devices via a wireless communication network (for example, a cellular communication network).

[0044] The vehicle 10 generates a road map in real time based on data sensed by the vehicle itself, and travels autonomously using the road map.

[0045] For example, the in-vehicle device 100 can acquire an estimation model used to generate a road map via a wireless communication network. The vehicle 10 is also equipped with various sensors and can detect objects around the vehicle 10 while traveling. For example, the in-vehicle device 100 can estimate information necessary for generating a road map by acquiring various data detected by the vehicle 10 and inputting the data into an estimation model.

[0046] The in-vehicle device 100 acquires data detected by various sensors mounted on the vehicle 10. The various sensors may be, for example, an in-vehicle camera or a GPS (Global Positioning System) device. For example, the in-vehicle device 100 acquires image data captured by the in-vehicle camera or location information (e.g., latitude and longitude information) acquired by the GPS device.

[0047] Next, the on-vehicle device 100 inputs the collected various data into an estimation model. There may be multiple estimation models, and the types of data input to each estimation model may be different. The on-vehicle device 100 may obtain one or more estimation models in advance from an external device.

[0048] Specifically, the on-board device 100 inputs data into a first model, which is an estimation model for estimating feature information, and a second model, which is an estimation model for estimating topology information. Here, feature information is information about objects that exist on a road and have a specific meaning on a road map. Examples of feature information include objects such as curbs that mark road boundaries, white lines on the road, stop lines, traffic lights, and crosswalks.

[0049] Furthermore, topology information is information that indicates the manner in which road lanes are connected to each other, or between road lanes and objects on or near the road. Topology information can also be said to be a network representation of roads in lane units (i.e., road network topology information). Specifically, topology information may be a representation of the connection relationships between road lanes, or between road lanes and objects on or near the road, using nodes and edges. Topology information that indicates the connection relationships between road lanes is used to represent the connection relationships between road lanes in a certain lane. The topology information, which indicates the connection relationship between road lanes and objects, is used by a vehicle traveling in a certain lane to plan which lane to select and travel in until it reaches its destination. In addition, the topology information, which indicates the connection relationship between road lanes and objects, is used by a vehicle traveling in a certain lane to determine which object (e.g., traffic light) to refer to in automated driving, etc.

[0050] Next, the on-vehicle device 100 acquires the feature information estimated by the first model and the topology information estimated by the second model, and then generates a road map of the area around the vehicle 10 based on the feature information and the topology information.

[0051] For example, the on-vehicle device 100 generates a map that maps road areas on which the vehicle can travel based on the feature information, and places other objects on the road areas. Then, the on-vehicle device 100 adds information indicating the trajectory on which the vehicle can travel based on the topology information to the generated map.

[0052] Then, the in-vehicle device 100 provides the generated road map to the automatic driving function of the vehicle 10. Therefore, the vehicle 10 can perform automatic driving control based on the generated road map.

[0053] As described above, the in-vehicle device 100 can generate a road map based on information estimated by a trained estimation model that receives as input various data detected by the vehicle 10. This allows the in-vehicle device 100 to automatically generate a surrounding map while the vehicle 10 is traveling.

[0054] [Configuration of on-board equipment] Next, a description will be given of the hardware and software configurations of the devices constituting the in-vehicle device 100. Fig. 2 is a diagram illustrating the components of the in-vehicle device 100 according to the first embodiment.

[0055] The in-vehicle device 100 can be configured as a computer having a processor (CPU, GPU, etc.), a main memory device (RAM, ROM, etc.), and an auxiliary memory device (EPROM, hard disk drive, removable media, etc.). The auxiliary memory device stores an operating system (OS), various programs, various tables, etc., and by executing the programs stored therein, various functions (software modules) that match predetermined purposes, as described below, can be realized. However, some or all of the functions may be realized as hardware modules using hardware circuits such as ASICs, FPGAs, etc.

[0056] The in-vehicle device 100 includes a control unit 110, a storage unit 120, a communication unit 130, and a display unit 140.

[0057] The control unit 110 is realized by a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) and a memory. The control unit 110 includes, as functional modules, an acquisition unit 111, an estimation unit 112, a generation unit 113, and an output unit 114. These functional modules may be realized by the control unit 110 executing a program.

[0058] The acquisition unit 111 communicates with an external device via the communication unit 130 and acquires one or more trained estimation models. Specifically, the acquisition unit 111 acquires a first model, which is a trained estimation model for estimating feature information, and a second model, which is a trained estimation model for estimating topology information. Here, the feature information refers to road information for generating a road map. Topology information is information about objects that have a specific meaning on a road map. Objects include, for example, curbs that mark road boundaries, white lines on the road, stop lines, traffic lights, and crosswalks. Topology information is information that shows how road lanes are connected to each other, or how road lanes are connected to objects on or near the road, and is information that mathematically shows the spatial relationships between lanes or between objects and lanes.

[0059] Furthermore, the acquisition unit 111 acquires data acquired by various sensors provided in the vehicle 10. The various sensors provided in the vehicle 10 include, for example, an on-board camera, a GPS device, a gyro sensor, a speed sensor, an acceleration sensor, and a LiDAR (Laser Imaging Detection and Ranging). Specifically, the data acquired by the acquisition unit 111 may include information indicating the position of the vehicle 10 (latitude and longitude information, etc.), information indicating the attitude of the vehicle 10 (tilt (orientation) from the coordinate axes of a road map), and an image captured by a camera provided in the vehicle 10.

[0060] The estimation unit 112 inputs various data acquired by the acquisition unit 111 into the trained estimation model acquired by the acquisition unit 111, and estimates information used to generate a road map. The estimation unit 112 periodically acquires data from the acquisition unit 111, and periodically estimates information used to generate a road map.

[0061] Specifically, the estimation unit 112 estimates feature information by inputting, into the first model, at least one of information indicating the position of the vehicle 10, information indicating the attitude of the vehicle 10, and an image captured by a camera provided on the vehicle 10. Specifically, the feature information may be information indicating white lines on a road, traffic lights, etc.

[0062] Furthermore, the estimation unit 112 inputs at least one of information indicating the position of the vehicle 10, information indicating the attitude of the vehicle 10, and an image captured by a camera provided on the vehicle 10 into the second model, and estimates topology information. Specifically, the topology information may be expressed as a matrix indicating the connection relationship between lanes.

[0063] The generation unit 113 generates a road map of the area around the vehicle 10 based on the feature information and topology information estimated by the estimation unit 112. For example, the generation unit 113 determines the general shape of the road by recognizing road boundaries and the like using the feature information, and then adds various objects to the determined general shape of the road to create a road map. Then, the generation unit 113 adds, to the road map, information such as the travelable directions of lanes of the road whose general shape has been identified, how they are connected to other lanes, and whether right or left turns are possible on each lane at an intersection, using the topology information. The generation unit 113 periodically acquires the feature information and topology information estimated by the estimation unit 112 and periodically generates a road map.

[0064] The output unit 114 outputs and provides the road map of the area around the vehicle 10 generated by the generation unit 113 to the automatic driving function of the vehicle 10. Alternatively, the output unit 114 may output the generated road map to the display unit 140 or the like. The output unit 114 may re-output an updated road map each time depending on the traveling position of the vehicle 10.

[0065] The storage unit 120 is a main storage device such as RAM or ROM, an EPROM, a hard disk drive, or an auxiliary storage device such as removable media. The auxiliary storage device stores an operating system (OS), various programs, various tables, etc., and by executing the programs stored therein, it is possible to realize functions that match the predetermined purpose of each unit of the control unit 110. However, some or all of the functions may be realized by hardware circuits such as ASICs or FPGAs.

[0066] The storage unit 120 stores data used or generated in the processing performed by the control unit 110. The storage unit 120 may also store sensor data obtained from the vehicle 10 and detected by various sensors of the vehicle 10.

[0067] The communication unit 130 is configured with a communication circuit that performs wireless communication. The communication unit 130 may be, for example, a communication circuit that performs wireless communication using 4G (4th Generation) or a communication circuit that performs wireless communication using 5G (5th Generation). The communication unit 130 may also be a communication circuit that performs wireless communication using LTE (Long Term Evolution) or a communication circuit that performs wireless communication using LPWA (Low Power Wide Area). The communication unit 130 may be a communication circuit that performs communication using the Wi-Fi (registered trademark) standard.

[0068] The display unit 140 is a display that displays images and the like to provide information to the user. The display unit 140 may be a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 140 may also be realized as a touch panel display. The display unit 140 displays the road map generated by the generation unit 113.

[0069] [Processing of on-board devices] Next, a description will be given of specific contents of the processing performed by the in-vehicle device 100. Fig. 3 is a flowchart of the processing performed by the control unit 110 of the in-vehicle device 100 according to the first embodiment.

[0070] For example, when a main switch of the vehicle is turned on, the in-vehicle device 100 starts the process illustrated in Fig. 3. Alternatively, the in-vehicle device 100 may start the process illustrated in Fig. 3 when a request is received from the user of the vehicle 10. The request may be an operation to cause the in-vehicle device 100 to start route guidance to the user's destination.

[0071] First, in step S10, the acquisition unit 111 acquires a first model and a second model, which are trained estimation models. The acquisition unit 111 communicates with an external device via the communication unit 130 and acquires the first model and the second model from the device. The number of estimation models acquired by the acquisition unit 111 is not limited to two. The number of estimation models acquired by the acquisition unit 111 may be three or more, or may be one estimation model as long as it realizes the functions of both the first model and the second model.

[0072] In step S11, the acquisition unit 111 acquires, from the vehicle 10, image data, attitude data, and position information of the vehicle 10 acquired by various sensors equipped in the vehicle 10. The image data is an image captured by an on-board camera equipped in the vehicle 10. The attitude data is an angle (e.g., an azimuth angle indicating the direction of travel) indicating how much the vehicle 10 is tilted from the coordinate axes of the road map generated by the on-board device 100. The position information of the vehicle 10 is latitude and longitude information indicating the position of the vehicle 10 acquired by a GPS device of the vehicle 10.

[0073] Next, in step S12, the estimation unit 112 inputs the image data, the attitude data, and the position information of the vehicle 10 to the first model and the second model. The estimation unit 112 may process each piece of data to match a data format required by each estimation model, and input the processed data to each estimation model.

[0074] Next, in step S13, the estimation unit 112 acquires feature information estimated by the first model. As described above, the feature information is information indicating objects that have specific meanings on the road. The feature information may include objects for determining road areas on which the host vehicle can travel.

[0075] FIG. 4 is a diagram illustrating feature information and topology information. As shown in FIG. 4(a), feature information is specifically information indicating traffic lights 200 installed on a road or white lines 210 painted on a road. The feature information may also include curbs 220 on the road or white lines 230 indicating roadside strips. By recognizing the curbs 220 on the road, the estimation unit 112 can determine the road area on which a vehicle can travel. Furthermore, by recognizing the white lines 210 and the white lines 230 indicating roadside strips, the estimation unit 112 can determine the general shape, number, etc. of the lanes that make up the road.

[0076] Furthermore, the estimation unit 112 can grasp information necessary for a road map to be generated later by recognizing objects, such as the traffic light 200, that have a specific meaning on the road other than the white lines 210 and 230. The traffic light 200 and the like are objects that have a specific meaning on the road map and need to be represented on the road map. The feature information is not limited to information indicating the traffic light 200 and the like, but is also not limited to road signs and other objects that have a specific meaning on the road map.

[0077] Next, in step S14, the estimation unit 112 acquires topology information estimated by the second model. As described above, the topology information is information that indicates the topology between road lanes or between road lanes and objects, that is, the manner in which road lanes are connected to each other or between road lanes and objects. As shown in (b) of FIG. 4, the topology information is information that indicates nodes (e.g., node 300) that are endpoints of sections obtained by dividing road lanes into segments, and edges (e.g., edge 310) that connect two nodes 300. The topology information shown in the figure indicates that edge 310 is connected to another edge via node 300. In this way, the topology information can indicate which other lanes a certain road lane is connected to.

[0078] Next, in step S15, the generation unit 113 generates a road map based on the feature information and topology information estimated by the estimation unit 112. For example, the generation unit 113 may obtain the general shape of the roads around the vehicle 10 based on the feature information.

[0079] Then, based on the feature information, the generation unit 113 may add objects such as structures or white lines on the road around the vehicle 10 to the outline of the road. In addition, based on the topology information, the generation unit 113 adds to the road map information such as the travelable directions of the lanes of the road whose outline has been identified, how they are connected to other lanes, and whether or not right or left turns are possible on each lane at an intersection.

[0080] Next, in step S16, the output unit 114 outputs the road map generated by the generation unit 113 to the automatic driving function of the vehicle 10. The automatic driving function of the vehicle 10 uses the road map data provided by the output unit 114 for automatic driving control of the vehicle 10.

[0081] After the process of step S16, the process returns to step S11. The control unit 110 periodically repeats the processes from step S11 to step S16. The control unit 110 stops the process when the main switch of the vehicle 10 is turned off or when a request is received from the user of the vehicle 10. The request may be an operation to cause the in-vehicle device 100 to end route guidance to the user's destination.

[0082] In the first embodiment, the estimation unit 112 inputs the acquired data into an estimation model to estimate feature information and topology information. Then, the generation unit 113 generates a road map of the area around the vehicle 10 based on the estimated feature information and topology information. This allows the on-board device 100 to determine information such as the trajectory the vehicle should travel, which could not be determined based on the feature information alone. It is possible to generate road maps with added information.

[0083] (Second embodiment) [Outline of processing performed by the on-board device] In the first embodiment, the in-vehicle device 100 outputs a road map of the area around the vehicle 10, which is generated in real time in accordance with the driving of the vehicle 10, to the automatic driving function of the vehicle 10. The vehicle 10 performs automatic driving control of the vehicle 10 based on the output road map. However, the road map does not necessarily have to be used only for automatic driving, and may also be used for other services. Therefore, in the second embodiment, the in-vehicle device 100 transmits the road map of the area around the vehicle 10, which is generated in real time, to the storage unit 120 of the in-vehicle device 100 or an external server device or the like.

[0084] 5 is a flowchart of the process executed by the control unit 110 of the in-vehicle device 100 according to the second embodiment. The in-vehicle device 100 starts step S20 in FIG. 5 after the process of step S15 in FIG.

[0085] First, in step S20, the output unit 114 acquires the road map of the area around the vehicle 10 generated by the generation unit 113. The output unit 114 may acquire the road map of the area around the vehicle 10 not as data in a format to be displayed as an image, but as data itself representing the lane topology and feature information of the road.

[0086] Next, in step S21, the output unit 114 stores the acquired road map in the storage unit 120. Alternatively, the output unit 114 transmits the acquired road map to an external server device.

[0087] In the second embodiment, the on-vehicle device 100 stores the road map generated in real time in accordance with the travel of the vehicle 10 in the storage unit 120 or transmits it to an external server device. This allows the on-vehicle device 100 to use the latest road map of the area around the vehicle 10 for other services.

[0088] (Third embodiment) In the second embodiment, the on-board device 100 generates control parameters for the autonomous driving of the vehicle 10 based on the generated road map, and supports the autonomous driving control of the vehicle 10. However, it is assumed that some error may occur in the autonomous driving control supported by the on-board device 100. An error is some event that hinders the continuation of the autonomous driving control. In such a case, it cannot be denied that there is a possibility that a problem has occurred in the road map generated by the on-board device 100. Therefore, in the third embodiment, when the on-board device 100 determines that an error has occurred in the autonomous driving of the vehicle 10, the on-board device 100 takes a predetermined action to improve the accuracy of the autonomous driving. In this embodiment, two examples of the predetermined action to improve the accuracy of the autonomous driving are (1) transmitting data acquired by the vehicle to an external device for verification, and (2) acquiring the latest estimation model from the external device.

[0089] 6 is a diagram illustrating components of an in-vehicle device 100A according to the third embodiment. The in-vehicle device 100A includes a control unit 110A, a storage unit 120, a communication unit 130, and a display unit 140. Descriptions of components of the in-vehicle device 100A that are the same as those of the in-vehicle device 100 will be omitted.

[0090] The control unit 110A includes an acquisition unit 111, an estimation unit 112, a generation unit 113, an output unit 114, and an error detection unit 115.

[0091] The error detection unit 115 determines whether or not an event has occurred that interferes with the autonomous driving of the vehicle 10. In this embodiment, the occurrence of an event that interferes with the autonomous driving is referred to as an error.

[0092] The error detection unit 115 determines that an error has been detected when the user of the vehicle 10 cancels the autonomous driving mode. Furthermore, the error detection unit 115 determines that an error has been detected when the sensor does not acquire a predetermined number of pieces of data a first number of times or more during a first period of time. Furthermore, the error detection unit 115 determines that an error has been detected when a road map that differs from the most recently generated road map by a predetermined percentage or more has been generated a second number of times or more during a second period of time. Although three events that can impede autonomous driving have been listed here, other events may also be detected as errors.

[0093] Fig. 7 is a flowchart of processing executed by the control unit 110A of the in-vehicle device 100A in the third embodiment. The in-vehicle device 100A starts the processing of Fig. 7 separately from the processing of Fig. 3 and Fig. 5. The processing of Fig. 7 is executed in parallel with the processing of Fig. 3 and Fig. 5.

[0094] First, in step S30, the error detection unit 115 determines whether or not an error has been detected during the autonomous driving of the vehicle 10. In this step, if the error detection unit 115 determines that an error has been detected, the determination is affirmative.

[0095] Here, the conditions under which the error detection unit 115 determines that an error has been detected will be described in detail. FIG. 8 is a diagram illustrating the conditions under which the control unit 110A of the in-vehicle device 100A determines that an error has been detected. As shown in (a) of FIG. 8, the error detection unit 115 determines that an error in the autonomous driving of the vehicle 10 has been detected when the autonomous driving mode of the vehicle 10 is canceled by the user. The cancellation of the autonomous driving mode may be a temporary cancellation, or may be an intervention in the driving operation by the user. This is because when the autonomous driving mode of the vehicle 10 is canceled by the user, there is a possibility that some kind of malfunction has occurred in the autonomous driving control.

[0096] Furthermore, as shown in (b) of FIG. 8, the error detection unit 115 determines that an error in the autonomous driving of the vehicle 10 has been detected if any of the various sensors mounted on the vehicle 10 fails to acquire data for a certain period of time a predetermined number of times or more. Here, the various sensors may be, for example, an in-vehicle camera or a GPS device. Furthermore, the data acquired by the various sensors may be image data, GPS information, or the like. This is because if the vehicle 10 fails to acquire data necessary for autonomous driving a predetermined number of times or more, there is a possibility that some kind of malfunction has occurred in the autonomous driving control.

[0097] 8(c), if a road map that differs from the most recently generated road map by a predetermined percentage or more is generated a predetermined number of times or more within a predetermined time period, the error detection unit 115 determines that an error has been detected in the autonomous driving of the vehicle 10. An example of a road map that differs from the most recently generated road map by a predetermined percentage or more is the following road map. (1) A road map in which the estimation result of the road area by the first model differs by a predetermined amount or more from the road map generated immediately before. (2) A road map in which the lane topology of the road estimated by the second model differs by a predetermined amount or more from the previously generated road map. If a road map is generated that differs from the previous map by a predetermined percentage or more, it is assumed that there is a problem with either the previous road map or the road map that has just been generated. If such a situation occurs a predetermined number of times or more within a predetermined period of time, there is a possibility that some kind of problem is occurring in the automatic driving control.

[0098] Returning to FIG. 7, the description will be continued.

[0099] If the determination in this step is affirmative, the process proceeds to step S31.

[0100] If the determination in this step is negative, the process ends.

[0101] When the process transitions to step S31, the output unit 114 transmits data including an image from the onboard camera to a predetermined external device. The output unit 114 may transmit the image data and at least one of the attitude data and the position information of the vehicle 10 to the predetermined external device. After the transmission, the acquisition unit 111 may acquire the results of an analysis of the data transmitted by the output unit 114 by the predetermined external device. Then, the output unit 114 may reflect the analysis results acquired by the acquisition unit 111 in the generation of control parameters for autonomous driving in the process of step S21 in FIG. 5.

[0102] Next, in step S32, the error detection unit 115 determines whether or not an error has been detected a predetermined number of times or more in the autonomous driving of the vehicle 10. If the error detection unit 115 determines that an error has been detected a predetermined number of times or more in the autonomous driving of the vehicle 10, this step is determined as positive.

[0103] If the determination in this step is affirmative, the process proceeds to step S33.

[0104] If the determination in this step is negative, the process proceeds to step S30.

[0105] When the process proceeds to step S33, the acquisition unit 111 acquires the latest first model and second model from an external device. The first model and second model to be acquired may be models that have been re-learned (additionally learned) from the first model and second model possessed by the host vehicle. The acquisition unit 111 re-acquires a re-learned estimation model for the trained estimation model acquired in step S10 of FIG. 3. The estimation model may be one that has been re-learned based on the data transmitted in step S31.

[0106] After step S33, the estimation model acquired in this step may be reflected in the processing from step S11 onward in FIG.

[0107] In the third embodiment, when the on-vehicle device 100A detects an error in the autonomous driving of the vehicle 10, the on-vehicle device 100A transmits data acquired by various sensors of the vehicle 10 to a predetermined external device. Furthermore, when the on-vehicle device 100A detects an error a predetermined number of times or more, the on-vehicle device 100A reacquires a re-learned estimation model from the predetermined external device. In this way, when an error occurs in the autonomous driving of the vehicle 10, the on-vehicle device 100A can correct the road map that it generates, thereby assisting in the execution of normal autonomous driving.

[0108] (Other variations) The above-described embodiment is merely an example, and the present disclosure may be modified as appropriate within the scope of the present disclosure. For example, the processes and means described in the present disclosure may be freely combined as long as no technical contradiction occurs.

[0109] In the above embodiment, the trained estimation model is acquired from an external device, but the in-vehicle device 100 may store the estimation model in advance, or may learn or re-learn the estimation model by itself.

[0110] The present disclosure can also be realized by supplying a computer program that implements the functions described in the above embodiments to a computer, and having one or more processors in the computer read and execute the program. Such a computer program can be stored in a non-transitory computer-readable storage medium that can be connected to the system bus of the computer. The non-transitory computer-readable storage medium may be provided to a computer via a network, etc. Examples of non-transitory computer-readable storage media include any type of disk, such as a magnetic disk (e.g., a floppy disk, a hard disk drive (HDD), etc.), an optical disk (e.g., a CD-ROM, a DVD disk, a Blu-ray disk), a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic card, a flash memory, an optical card, and any type of medium suitable for storing electronic instructions. [Explanation of symbols]

[0111] 100,100A...In-vehicle equipment 110,110A···Control unit 111...Acquisition part 112... Estimation section 113...Generation section 114... Output section 120...Storage section 130 Communications Department 140...Display section

Claims

1. An information processing device mounted on a vehicle, acquiring data via sensors mounted on the vehicle; inputting the data into a first model, which is a trained estimation model that estimates information about an object for generating a road map as feature information, to estimate the feature information; inputting the data into a second model, which is a trained estimation model for estimating topology information, which is a topology of road lanes or a topology of the object and the lanes, to estimate the topology information; generating the road map around the vehicle based on the feature information and the topology information; A control unit that executes Information processing device.

2. Further, a storage unit is provided, The control unit acquiring the first model and the second model from an external device; storing the acquired first model and second model in the storage unit; The information processing device according to claim 1 .

3. The data is The information includes information indicating the position of the vehicle, information indicating the attitude of the vehicle, and an image captured by a camera provided on the vehicle. The information processing device according to claim 1 .

4. The control unit generating control parameters for controlling the behavior of the vehicle in an automated driving manner based on the generated road map; The information processing device according to claim 1 .

5. The control unit When an error in the autonomous driving of the vehicle is detected while the vehicle is traveling, the data including an image captured by a camera provided in the vehicle is transmitted to a predetermined device. The information processing device according to claim 4 .

6. The control unit The error is determined to have been detected when a user of the vehicle cancels the autonomous driving mode, when the sensor does not acquire a predetermined number of data a first number of times during a first period of time, or when a road map that differs from the most recently generated road map by a predetermined percentage or more is generated a second number of times during a second period of time. The information processing device according to claim 5 .

7. The control unit If a predetermined sensor provided in the vehicle does not acquire GPS (Global Positioning System) information within a predetermined time, it is determined that the error has been detected. The information processing device according to claim 5 .

8. The road map that is different from the immediately preceding road map by a predetermined percentage or more is the road map that is different from the immediately preceding road map by the first model. the estimation result, or the estimation result of the lane topology by the second model, is a road map that differs from the immediately preceding road map by a predetermined amount or more. The information processing device according to claim 6 .

9. The control unit When the error is detected a predetermined number of times or more, the re-trained first model and the re-trained second model are acquired from an external device. The information processing device according to claim 5 .

10. An information processing method executed by an information processing device mounted on a vehicle, acquiring data via sensors mounted on the vehicle; a step of inputting the data into a first model, which is a trained estimation model that estimates information about objects for generating a road map as feature information, and estimating the feature information; a step of inputting the data into a second model, which is a trained estimation model for estimating topology information, which is a topology of road lanes or a topology of the object and the lanes, to estimate the topology information; generating the road map around the vehicle based on the feature information and the topology information; Including, Information processing methods.

11. acquiring the first model and the second model from an external device; storing the acquired first model and second model in a storage unit; further comprising: The information processing method according to claim 10.

12. The data is The information includes information indicating the position of the vehicle, information indicating the attitude of the vehicle, and an image captured by a camera provided on the vehicle. The information processing method according to claim 10.

13. generating control parameters for controlling the behavior of the vehicle in an automated driving manner based on the generated road map; further comprising: The information processing method according to claim 10.

14. When an error in the automated driving of the vehicle is detected while the vehicle is traveling, transmitting the data including an image captured by a camera provided in the vehicle to a predetermined device. further comprising: The information processing method according to claim 13.

15. a step of determining that the error has been detected when a user of the vehicle cancels the autonomous driving mode, when the sensor does not acquire a predetermined number of pieces of data a first number of times during a first period of time, or when a road map that differs from the immediately preceding road map by a predetermined percentage or more is generated a second number of times during a second period of time. further comprising: The information processing method according to claim 14.

16. The predetermined sensor provided in the vehicle is a GPS (Global Positioning System If the system information is not acquired within a predetermined time, it is determined that the error has been detected. The information processing method according to claim 14.

17. The road map that differs from the immediately preceding road map by a predetermined percentage or more is a road map in which the estimation result of the white lines by the first model or the estimation result of the topology of the lanes by the second model differs from the immediately preceding road map by a predetermined amount or more. The information processing method according to claim 15.

18. acquiring the re-trained first model and the re-trained second model from an external device when the error is detected a predetermined number of times or more; further comprising: The information processing method according to claim 14.

19. A program for causing a computer to execute the information processing method according to any one of claims 10 to 18.

Images