Vehicle interior / exterior linkage device and method

Abstract

A vehicle interior / exterior linking device for use in a vehicle provided with an in-vehicle network for data transfer, a wireless communication device for performing data communication wirelessly with the exterior of the vehicle, and a plurality of sensors, wherein said vehicle interior / exterior linking device includes: a data reception unit that receives sensor data from the plurality of sensors via the in-vehicle network; an end-to-end delay time estimation unit that estimates the end-to-end transmission delay time of sensor data from each of the plurality of sensors to a predetermined device capable of communicating by the wireless transmission device; a value determination unit that determines the value of the sensor data output by the plurality of sensors on the basis of the state of the vehicle and the states of the plurality of sensors; and a selection unit that uses the transmission delay time estimated by the delay time estimation unit and the values determined by the value determination unit as a basis to select some of the sensor data and transmit a copy of the selected sensor data to the predetermined device via the wireless communication device.

Description

[Technical field]

[0001] This disclosure relates to an in-vehicle / out-of-vehicle cooperation device and method. This application claims priority to Japanese Application No. 2020-031119, filed on February 27, 2020, and the entire contents of said Japanese application are incorporated herein by reference. [Background technology]

[0002] When driving a vehicle, it is necessary to pay close attention not only to the movement of the vehicle itself but also to the movement of other vehicles. Particular attention is required when pedestrians are present in addition to vehicles. As shown in FIG. 1, a conventional technology detects moving objects (hereinafter referred to as "moving objects") that exist in such a real space 50 using a number of sensors such as LiDAR (Light Detection And Ranging) and cameras, estimates their attributes (adult, child, vehicle, motorcycle, etc.), and creates a traffic situation bird's-eye view map 52 using high-resolution road map data (hereinafter referred to as "high-resolution map") that is prepared in advance in a virtual space.

[0003] In order to create such a traffic condition bird's-eye view map 52, it is necessary to collect sensor data, which is the output of a large number of sensors, from vehicles equipped with those sensors and from infrastructure sensors such as cameras installed on the roadside. For this purpose, it is considered to use a fifth generation mobile communication system (so-called "5G"). One example of such technology is disclosed in Patent Document 1, which will be described later.

[0004] As an example of 5G, FIG. 2 shows a communication system 70 including a first slice to a fourth slice.

[0005] Referring to Figure 2, the first slice includes vehicles 82 and 84 equipped with on-board devices 92 and 94 capable of 5G wireless communication, an infrastructure camera 88 equipped with a wireless communication device 98, a traffic signal controller 90 equipped with a wireless communication device 100 installed at a traffic light, a mobile phone 96 held by a pedestrian 86, etc., and is a slice defined to enable direct communication between them.

[0006] The second slice includes multiple base stations 110, 112, and 114, and is defined so that the communication terminals included in the first slice communicate with these base stations.

[0007] The third slice includes metro NWs (Networks) 120 and 122 and edge servers 126 and 128 provided in distributed DCs (Data Centers) 124, 130, etc. connected thereto, and is defined so that each communication terminal can communicate with them via base stations 110, 112, 114, etc. Note that the metro NW refers to a communication network constructed to connect buildings, social facilities, houses, etc. in a limited range, such as a city. The base stations 110, 112, 114, etc. are each connected to one of the edge servers 126, 128, etc. The edge servers 126, 128, etc. are so named because they are located on the outermost side of the system connected by the network and connect the network system with the rest.

[0008] The fourth slice includes a core NW 140 capable of communicating with multiple metro networks, and a core server 144 provided in a core DC 142 connected to this core NW 140 is capable of communicating with communication devices connected to each metro network, such as edge servers 126 and 128.

[0009] Typically, the above-mentioned traffic condition bird's-eye view map is constructed and maintained for a specific range in the edge server 128 or the like. Each communication terminal belonging to the first slice transmits sensor data detected by the sensor equipped therein to, for example, the edge server 126. The edge server 126 integrates the sensor data to reconstruct the actual road conditions in a virtual space, and creates and maintains the traffic condition bird's-eye view map 52. Based on the traffic condition bird's-eye view map 52 thus maintained, the edge server 126 transmits information to each communication terminal to assist the communication terminal in driving, etc. [Prior art documents] [Patent documents]

[0010] [Patent Document 1] JP 2018-018284 A [Patent Document 2] JP 2008-263580 A [Patent Document 3] JP 2018-073389 A Summary of the Invention

[0011] An in-vehicle / outdoor collaboration device according to a first aspect of the present disclosure is an in-vehicle / outdoor collaboration device used in a vehicle equipped with an in-vehicle network for data transfer, a wireless communication device that wirelessly communicates data with the outside of the vehicle, and a plurality of sensors, and includes a data receiving unit that receives sensor data from the plurality of sensors via the in-vehicle network, an end-to-end delay time estimation unit that estimates an end-to-end transmission delay time of the sensor data from each of the plurality of sensors to a predetermined device with which communication is possible via the wireless communication device, a value determination unit that determines a value of the sensor data output by the plurality of sensors based on a state of the vehicle and states of the plurality of sensors, and a selection unit that selects a portion of the sensor data based on the transmission delay time estimated by the delay time estimation unit and the value determined by the value determination unit, and transmits a copy of the selected sensor data to the predetermined device via the wireless communication device.

[0012] A vehicle-exterior collaboration method according to a second aspect of the present disclosure is a vehicle-exterior collaboration method for linking the inside and outside of the vehicle in a vehicle equipped with an in-vehicle network for data transfer, a wireless communication device that wirelessly communicates data with the outside of the vehicle, and a plurality of sensors by transmitting sensor data from the plurality of sensors to a specified external device, the method including the steps of: a computer receiving sensor data from the plurality of sensors mounted in the vehicle via the in-vehicle network; a computer estimating a transmission delay time of the sensor data between the end of the sensor and the specified device with which communication can be performed via the wireless communication device; a computer determining a value of the sensor data output by the plurality of sensors based on a state of the vehicle and a state of the plurality of sensors; and a computer selecting a portion of the sensor data based on the transmission delay time estimated in the step of estimating the transmission delay time and the value determined in the step of determining the value, and transmitting a copy of the selected sensor data to the specified device via the wireless communication device.

[0013] The scope and content of this disclosure will become apparent from a consideration of the detailed description of the disclosure in conjunction with the accompanying drawings. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing a schematic relationship between actual road conditions and a road condition bird's-eye view map. [Diagram 2] FIG. 2 is a diagram showing a schematic configuration of the communication system disclosed in Patent Document 1. As shown in FIG. [Diagram 3] FIG. 3 is a diagram showing a schematic configuration of a traffic assistance system according to a first embodiment of the present disclosure. [Figure 4] FIG. 4 is a block diagram showing a configuration of elements related to a traffic assistance system in a vehicle according to the first embodiment of this disclosure. [Diagram 5] FIG. 5 is a block diagram showing a configuration of an edge server that is a traffic assistance server that communicates with vehicles according to the first embodiment of the present disclosure. [Figure 6]FIG. 6 is a block diagram illustrating a schematic configuration for communicating the sensor data of the vehicle according to the first embodiment. [Figure 7] FIG. 7 is a block diagram showing a schematic hardware configuration of the vehicle interior / exterior linking unit used in the vehicle according to the first embodiment shown in FIG. [Figure 8] FIG. 8 is a diagram showing data required for in-vehicle / out-of-vehicle cooperation, which is stored in a memory in the in-vehicle / out-of-vehicle cooperation unit shown in FIG. [Figure 9] FIG. 9 is a diagram illustrating an example of the contents stored in the sensor data set table illustrated in FIG. [Figure 10] FIG. 10 is a diagram showing an example of the sensor priority policy table shown in FIG. [Figure 11] FIG. 11 is a diagram showing, in a table format, the correspondence relationship between the sensor types and the sensor type numbers in the vehicle according to the first embodiment. [Figure 12] FIG. 12 is a diagram showing, in a table format, the correspondence between the sensor position type indicating the position of a sensor mounted on a vehicle and the sensor position type number. [Figure 13] FIG. 13 is a schematic diagram showing an example of a mounting position of a camera in a vehicle. [Figure 14] FIG. 14 is a diagram for explaining the priority of sensors based on the value of the sensor data. [Figure 15] FIG. 15 is a flowchart showing a control structure of a program for controlling communication from a vehicle, executed by the transportation assistance server according to the first embodiment of this disclosure. [Figure 16] FIG. 16 is a flowchart showing a control structure of a program for controlling transmission of sensor data, which is executed by an on-board device mounted on a vehicle in the transportation assistance system in accordance with the first embodiment. [Figure 17] FIG. 17 is a flowchart showing a control structure of a program for estimating a transmission delay, executed by an on-board device mounted in a vehicle in the traffic assistance system in accordance with the first embodiment. [Figure 18]FIG. 18 is a flowchart showing a control structure of a program for updating the priority policy of a sensor, executed by an on-board device mounted on a vehicle in the traffic assistance system in accordance with the first embodiment. [Figure 19] FIG. 19 is a flowchart showing a control structure of a program for implementing sensor data selection and switch setting processing, executed by an in-vehicle device mounted on a vehicle in the traffic assistance system in accordance with the first embodiment. [Figure 20] FIG. 20 is a block diagram showing a schematic configuration of a traffic assistance system according to the second embodiment of the present disclosure. [Figure 21] FIG. 21 is a block diagram showing a configuration of elements related to a traffic assistance system in a vehicle according to a second embodiment of this disclosure. [Figure 22] FIG. 22 is a flowchart showing a control structure of a program for controlling transmission of sensor data, which is executed by an on-board device mounted on a vehicle in the transportation assistance system in accordance with the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] [Problem this disclosure aims to solve] A communication system such as that disclosed in Patent Document 1 can construct and maintain a traffic condition bird's-eye view map 52. Dynamic information and caution information related to traffic obtained from the traffic condition bird's-eye view map 52 are transmitted to each communication terminal. As a result, for example, a vehicle equipped with such a communication terminal can know the road conditions when necessary, which can be useful for traffic safety.

[0016] However, the above-mentioned conventional technologies still have problems to be solved. The first problem is how to deal with dynamic fluctuations in the transmission capacity from the vehicle to a traffic support server such as an edge server. Large fluctuations in the transmission capacity may occur due to switching between 4G / 5G areas and shadowing. In addition, large fluctuations in the transmission capacity may occur due to the internal state of the vehicle (load state of the in-vehicle network and CPU (Central Processing Unit) resources) and state fluctuations of the traffic support server. The traffic support system needs to appropriately deal with such dynamic fluctuations in the transmission capacity. The second problem is the selection of sensor data. If it is not possible to transmit all the sensor data from the vehicle to the traffic support server due to fluctuations in the transmission capacity, it becomes necessary to select and transmit part of the sensor data. To do this, it is necessary to determine what kind of sensor data should be transmitted from the vehicle to the traffic support server most efficiently.

[0017] There is no prior art that can solve both the first and second problems. For example, the above-mentioned Patent Document 2 proposes a technology that switches the type, resolution, and frame rate of an image of a camera mounted on a vehicle according to changes in transmission capacity. However, the technology described in Patent Document 2 does not take into consideration the state inside the vehicle-mounted device when performing such control. Furthermore, the technology described in Patent Document 2 does not take into consideration what conditions a sensor should be selected according to various situations in which the vehicle is placed, that is, the value of sensor data.

[0018] Meanwhile, the above-mentioned Patent Document 3 discloses a technique for determining a priority when transmitting a data sample from a vehicle to a server based on the value of the data sample. In Patent Document 3, the value of the data sample is calculated based on a data value map in a table called a data value table. Based on the calculated value, a priority is determined for transmitting the data sample to the server. The data value map is determined, for example, based on the purpose for which the data sample is to be used in the server, and is transmitted to the vehicle as needed. When the vehicle receives the data value map, the vehicle updates the existing data value map with the received data. Therefore, from the viewpoint of the server, data can be collected from the vehicle with a priority according to the purpose of the data.

[0019] However, Patent Document 3 does not disclose a technology that changes the method of calculating the value of data in real time based on the situation in which the vehicle is placed. In particular, in a traffic assistance system, the value of data does not change depending on the use, but the priority of data to be transmitted to the server should always be processed in real time according to the situation of the vehicle and its surroundings. Patent Document 3 does not disclose such a technology.

[0020] Therefore, an object of this disclosure is to provide an in-vehicle / out-of-vehicle collaboration device and method that can determine the priority of data transmission according to changes in transmission capacity and the conditions of the vehicle and its surroundings, so that traffic assistance can be effectively performed by a specified device such as a traffic assistance server.

[0021] [Description of the embodiments of the present disclosure] In the following description and drawings, the same parts are denoted by the same reference numerals. Therefore, detailed description thereof will not be repeated. Note that at least some of the following embodiments may be combined in any desired manner.

[0022] (1) An in-vehicle / external communication device according to a first aspect of the present disclosure is an in-vehicle / external communication device used in a vehicle equipped with an in-vehicle network for data transfer, a wireless communication device for wirelessly communicating data with the outside of the vehicle, and a plurality of sensors, and includes: a data receiving unit that receives sensor data from the plurality of sensors via the in-vehicle network; an end-to-end delay time estimation unit that estimates an end-to-end transmission delay time of the sensor data from each of the plurality of sensors to a predetermined device with which communication is possible via the wireless communication device; a value determination unit that determines a value of the sensor data output by the plurality of sensors based on a state of the vehicle and states of the plurality of sensors; and a selection unit that selects a portion of the sensor data based on the transmission delay time estimated by the delay time estimation unit and the value determined by the value determination unit, and transmits a copy of the selected sensor data to the predetermined device via the wireless communication device.

[0023] The end-to-end delay time estimation unit estimates a transmission delay time of the sensor data from each of the multiple sensors mounted on the vehicle to a specified device. The value determination unit determines the value of the sensor data output by the multiple sensors based on the state of the vehicle and the state of the multiple sensors. The selection unit selects a part of the sensor data based on the transmission delay time estimated by the delay time estimation unit and the value determined by the value determination unit, and transmits a copy of the selected sensor data to the specified device via the wireless communication device. The end-to-end transmission delay time from the sensor to a specific device, including the delay time in the in-vehicle network, is estimated, and the sensor data to be transmitted to the specified device is selected based on the estimated transmission delay time and the value determined for each of the sensor data. The in-vehicle / external linking device selects and transmits sensor data that can be effectively used by the specified device, taking into account the end-to-end transmission delay time from the sensor to the specified device while following the fluctuation of the transmission capacity of the wireless communication device, so that it is possible to provide an in-vehicle / external linking device that can maximize the transmission capacity and demonstrate the function of the specified device as much as possible.

[0024] (2) The end-to-end delay time estimation unit may include a network transmission time estimation unit that estimates, for each sensor, a network delay time required for sensor data from the multiple sensors to reach the data receiving unit via the in-vehicle network; a processing time estimation unit that estimates a processing time required for processing the sensor data that has reached the data receiving unit until it is input into the wireless communication device; a wireless communication delay time estimation unit that estimates a wireless communication delay time required for the sensor data to be transmitted from the wireless communication device to a specified device; and a delay time calculation unit that calculates a transmission delay time based on the data amount, network delay time, processing time, and wireless communication delay time of each of the multiple sensors.

[0025] In addition to the processing time required for the sensor data that has arrived at the data receiving unit to be input to the wireless communication device and the wireless communication delay time required for the sensor data to be transmitted from the wireless communication device to a specified device, the network delay time required for the sensor data from multiple sensors to reach the data receiving unit via the in-vehicle network is estimated for each sensor, and the end-to-end transmission delay time is calculated based on these. Since the end-to-end transmission delay time is calculated taking into account the transmission delay time of the in-vehicle network, the amount of data that can be transmitted to a specified device within the allowable delay time can be more accurately estimated, and the sensor data to be transmitted can be more accurately selected.

[0026] (3) The value determination unit may include an infrastructure detection range estimation unit that estimates the detection range of an infrastructure sensor present in the vicinity of the vehicle based on the vehicle's position, and a value setting unit that estimates the detection ranges of a plurality of sensors based on the vehicle's position and attitude, and sets a value of the sensor data from each sensor so that the smaller the area of ​​overlap with the detection range of the infrastructure sensor estimated by the infrastructure detection range estimation unit, the greater the value of the sensor data from the sensor.

[0027] Among the multiple sensors mounted on a vehicle, sensor data from a sensor with a large detection range that overlaps with the detection range of an infrastructure sensor has low utility value for a specified device. The smaller the detection range of a sensor that overlaps with the detection range of an infrastructure sensor, the more valuable the sensor data output by that sensor is for a specified device. Therefore, by estimating the detection ranges of infrastructure sensors present in the vicinity of the vehicle and calculating the value of sensors whose detection ranges overlap with the detection ranges of infrastructure sensors to be small, the sensor data from those sensors is preferentially selected and transmitted to the specified device. Since high-value sensor data is transmitted to the specified device within a limited transmission capacity, stable processing based on the sensor data can be performed in the specified device regardless of fluctuations in transmission capacity.

[0028] (4) The infrastructure detection range estimation unit may include an infrastructure detection range update unit that repeatedly updates the estimated detection range of an infrastructure sensor present in the vicinity of the vehicle based on the position of the vehicle at a predetermined time interval, and the value setting unit may include a value update unit that repeatedly updates the value of the sensor data from each sensor based on the position and attitude of the vehicle at a predetermined time interval so that the value of a sensor is greater for a sensor that has a smaller area of ​​overlap with the detection range of the infrastructure sensor updated by the infrastructure detection range update unit.

[0029] The detection range of the infrastructure sensor is updated at a predetermined time interval. The value of the sensor data is also updated at a predetermined time interval based on the position and attitude of the vehicle and the updated detection range of the infrastructure sensor. Therefore, the value of each sensor of the vehicle is updated at least at a predetermined time interval in accordance with the movement of the vehicle and the communication environment around the vehicle. As a result, regardless of the movement of the vehicle and changes in the communication environment around the vehicle, highly valuable sensor data is transmitted to a specified device. Therefore, regardless of changes in the environment, the specified device can perform stable processing.

[0030] (5) Each of the multiple sensors may be classified into one of multiple sensor types, and the infrastructure detection range estimation unit may include a sensor type detection range estimation unit that estimates the detection range of infrastructure sensors present in the vicinity of the vehicle for each sensor type of the infrastructure sensor based on the position of the vehicle, and the value setting unit may include a sensor type sensor data value setting unit that estimates, for each of the multiple sensor types, the detection range of each sensor belonging to that sensor type among the multiple sensors based on the position and attitude of the vehicle, and sets the value of sensor data from sensors belonging to that sensor type among the multiple sensors so that the smaller the area of ​​overlap with the detection range of the infrastructure sensor estimated for that sensor type by the sensor type detection range estimation unit, the higher the value.

[0031] Since the value setting unit sets the value of the sensor data for each type of sensor, it is possible to more effectively utilize transmission capacity and select effective sensor data for processing by a specific device.

[0032] (6) The in-vehicle / external integration device may further include a sensor dataset storage unit that stores information specifying multiple sensor datasets together with an identifier of each sensor dataset, and the multiple sensor datasets may include information specifying, for each of multiple sensor types, the number of sensors to select from the multiple sensors and a data format related to the amount of sensor data from the sensors. The in-vehicle / external integration device may further include a sensor dataset reading unit that, in response to receiving one of the identifiers from a specific device, reads information of the sensor dataset corresponding to the identifier from the sensor dataset storage unit, a sensor data selection unit that selects, for each of the multiple sensor types, sensor data from a number of sensors specified by the information of the sensor dataset read by the sensor dataset reading unit in order of decreasing value based on the information of the sensor dataset read by the sensor dataset reading unit and the value set by the sensor type sensor data value setting unit for the sensor data from each sensor, and a network setting unit that sets the in-vehicle network to transfer the sensor data selected by the sensor data selection unit in priority to other sensor data.

[0033] When an identifier specifying one of the sensor data sets is provided to the vehicle interior / exterior integration device, the number of sensors specified by the sensor data set is selected for each sensor type in order of the sensor data with the highest value. For all types of sensors, a predetermined number of valuable sensor data items are transmitted to a predetermined device based on the designation by the sensor data set. As a result, the predetermined device can effectively use the sensor data to perform stable processing.

[0034] (7) The vehicle may further include a vehicle control device that controls the vehicle using sensor data from the multiple sensors, and the vehicle control device may include an autonomous driving ECU (Electronic Control Unit) that performs autonomous driving of the vehicle using sensor data received from the multiple sensors via a network.

[0035] The autonomous driving ECU operates based on sensor data from multiple sensors installed in the vehicle. One of these sensor data is selected by the vehicle interior / exterior linkage device and sent to a specific device, so that the autonomous driving ECU and the specific device can share the sensor data, and the autonomous driving of the vehicle and the processing of the specific device can be efficiently performed.

[0036] (8) The vehicle control device may include a remote control ECU that controls the vehicle in accordance with specific remote control commands, a sensor data transmission unit that transmits sensor data received from a plurality of sensors via a network to an external remote control device via a wireless communication device, and an input device that inputs remote control commands received from the remote control device via the wireless communication device to the remote control ECU.

[0037] The remote control device operates based on sensor data received from multiple sensors mounted on the vehicle, and transmits remote control commands to the remote control ECU. Any of the sensor data transmitted to the remote control device is selected by the vehicle interior / exterior linkage device and transmitted to a specific device. Therefore, the sensor data can be shared between the remote control device and the specific device, and the vehicle can be remotely controlled and the specific device can execute processing efficiently.

[0038] (9) A vehicle-exterior collaboration method according to a second aspect of the present disclosure is a vehicle-exterior collaboration method for linking the inside and outside of the vehicle in a vehicle equipped with an in-vehicle network for data transfer, a wireless communication device for wirelessly communicating data with the outside of the vehicle, and a plurality of sensors by transmitting sensor data from the plurality of sensors to a specified external device, the method including the steps of: a computer receiving sensor data from the plurality of sensors mounted in the vehicle via the in-vehicle network; a computer estimating a transmission delay time of the sensor data between the end of the sensor and the specified device with which communication can be performed via the wireless communication device; a computer determining a value of the sensor data output by the plurality of sensors based on a state of the vehicle and a state of the plurality of sensors; and a computer selecting a portion of the sensor data based on the transmission delay time estimated in the step of estimating the transmission delay time and the value determined in the step of determining the value, and transmitting a copy of the selected sensor data to the specified device via the wireless communication device.

[0039] In the step of estimating the end-to-end transmission delay time of the sensor data from each of the multiple sensors mounted on the vehicle to a predetermined device, the end-to-end transmission delay time of the sensor data is estimated. Based on the state of the vehicle and the state of the multiple sensors, the value of the sensor data output by the multiple sensors is determined. Based on the estimated transmission delay time and the value determined in the step of determining the value, a part of the sensor data is selected, and a copy of the sensor data is transmitted to the predetermined device via the wireless communication device. The end-to-end transmission delay time from the sensor to the specific device, including the delay time in the in-vehicle network, is estimated, and the sensor data to be transmitted to the predetermined device is selected based on the estimated transmission delay time and the value determined for each of the sensor data. While following the fluctuation of the transmission capacity due to wireless communication, the sensor data that can be effectively used by the predetermined device is selected and transmitted, taking into account the end-to-end transmission delay time from the sensor to the predetermined device. Therefore, it is possible to provide a vehicle-external cooperation method that can maximize the transmission capacity while exerting the function of the predetermined device as much as possible.

[0040] [Effect of this disclosure] As described above, according to this disclosure, it is possible to provide an in-vehicle / out-of-vehicle connection device and method that can determine the priority of data transmission according to changes in transmission capacity and the conditions of the vehicle and its surroundings, so that traffic assistance can be effectively provided by a specified device such as a traffic assistance server.

[0041] <First embodiment> <composition> Overall composition A schematic configuration diagram of the traffic support system according to this disclosure is shown in Fig. 3. Referring to Fig. 3, this traffic support system includes vehicles 82 and 84, infrastructure sensors (not shown), mobile phones carried by pedestrians, and an edge server 128 that is a traffic support server that communicates with these via a base station 110 and creates and maintains a communication status map 152 and a traffic status bird's-eye view map 150, similar to the system described in Patent Document 1.

[0042] The communication status map 152 is, for example, a high-definition map prepared in advance for traffic support divided into a grid of multiple sections with sides of a certain length, and the communication status in each section is recorded. Typically, the communication status for each section is represented by the center position of the section and a representative value of the transmission speed of data transmitted from vehicles in the section to the traffic support server during a certain period of the past. In order to calculate the representative value of the transmission speed for the certain period of the past, data on the transmission speed of data received by the traffic support server during the certain period of the past is also accumulated. As the representative value of the transmission speed, typically, the average transmission speed, the most frequent value of the transmission speed, the median value of the transmission speed, a representative value calculated from data excluding a certain number of the top and bottom speeds, etc. can be used.

[0043] A traffic condition overhead map is a map for a certain area that stores the positions, speeds, attributes, etc. of moving objects such as vehicles and pedestrians and fixed objects such as buildings in the area in association with identification numbers on a high-definition map prepared in advance in a virtual space. The traffic condition overhead map is constructed based on data transmitted from so-called infrastructure sensors installed in the area and on-board sensors installed in vehicles, and information obtained from specified sources regarding road construction, accidents, the installation locations of infrastructure sensors and their attributes, etc.

[0044] In the following, a first embodiment will be described taking as an example how to transmit sensor data from the vehicle 82 to the edge server 128.

[0045] 《In-vehicle device》 Referring to FIG. 4, the vehicle 82 includes various sensors 170 and an autonomous driving ECU (Electronic Control Unit) 172 for controlling the autonomous driving of the vehicle, an in-vehicle device 92 for collecting sensor data from these sensors 170 and controlling the autonomous driving ECU 172 while communicating with the autonomous driving ECU 172, and an exterior communication device 162 used by the in-vehicle device 92 to communicate with elements outside the vehicle, such as an edge server 128 and other vehicles.

[0046] The in-vehicle device 92 includes a network switch (NSW) 160 for in-vehicle communication to which the sensor 170, the automatic driving ECU 172, etc. are connected, and an in-vehicle / out-of-vehicle cooperation unit 164 that is connected to the sensor 170 and the automatic driving ECU 172 via the NSW 160, transmits sensor data from the sensor 170 to the outside via the exterior communication device 162, and controls the operating state of the automatic driving ECU 172, etc., in cooperation with information collected from the in-vehicle sensors, etc. and data received from the outside via the exterior communication device 162. As described later, the in-vehicle / out-of-vehicle cooperation unit 164 controls processing when collecting sensor data and processing when transmitting sensor data to the outside in response to an instruction from the edge server 128 and a change in the situation of the vehicle 82. Note that the in-vehicle device 92 shown in FIG. 4 includes the NSW 160 and the in-vehicle / out-of-vehicle cooperation unit 164, but may be a single in-vehicle device including the automatic driving ECU 172, the exterior communication device 162, or both. The in-vehicle device 92 may be realized as a semiconductor integrated circuit such as an Application-Specific Integrated Circuit (ASIC) or a system Large Scale Integration (LSI), or as a device in which a plurality of semiconductor integrated circuits are mounted on a substrate.

[0047] Normally, the sensor data from the sensor 170 is transmitted to the autonomous driving ECU 172, but as indicated by the arrow 174, a portion of the sensor data is selected by the in-vehicle / out-of-vehicle cooperation unit 164 and transmitted to the edge server 128. In the following embodiment, a description will be given of how the in-vehicle / out-of-vehicle cooperation unit 164 selects sensor data that enables the edge server 128 to most efficiently construct the traffic condition bird's-eye view map 150, depending on the transmission capacity between the exterior communication device 162 and the edge server 128, the processing time in the network in the vehicle 82 and the in-vehicle / out-of-vehicle cooperation unit 164 and the like, and the situation in which the vehicle 82 is placed.

[0048] The number of sensors mounted on vehicles is now very large, and is expected to continue to increase in the future, as well as the amount of data output from each sensor. For this reason, it is believed that the transmission speed of conventionally used in-vehicle networks will no longer be sufficient, and the use of networks with gigabit-class transmission speeds for in-vehicle networks is being considered. As will be described later with reference to FIG. 6, in the following embodiment, the in-vehicle devices of the vehicle 82 communicate with each other through such a network. Existing technology related to networks connecting computers can be applied to such networks, and it is expected that the cost of constructing an in-vehicle network will be reduced. In this example, TCP-IP (Transmission Control Protocol-Internet Protocol), which is currently the most popular communication protocol for in-vehicle networks, is adopted.

[0049] <<Transportation Support Server>> 5, the edge server 128 includes a reception processing unit 210 for receiving signals from a plurality of infrastructure sensor equipment 180 (including any combination of cameras, millimeter wave radar, and LiDAR) and signals from a sensor (any combination of cameras, LiDAR, and millimeter wave radar) 170 mounted on the vehicle 82 as described above. Each of the infrastructure sensor equipment 180 includes an infrastructure sensor 190 including any combination of cameras, millimeter wave radar, and LiDAR, and a communication device 192 for transmitting a signal output from the infrastructure sensor 190 to the reception processing unit 210 of the edge server 128. Similarly, the vehicle 82 includes a sensor 170 including a camera, LiDAR, or millimeter wave radar, and an exterior communication device 162 for transmitting at least a part of a signal output from the sensor 170 to the reception processing unit 210 of the edge server 128.

[0050] The edge server 128 further includes a moving object tracking unit 212 that determines and tracks the position of each moving object in a predetermined first period by analyzing ranging signals from LiDAR, millimeter wave radar, etc., among the signals received by the receiving processing unit 210, and outputs moving object tracking results 213, a moving object tracking result memory unit 214 for storing the moving object tracking results 213, and an attribute detection unit 216 that determines the attributes and positions of moving objects such as vehicles, people, etc. in the image in a predetermined second period longer than the first period by performing image analysis on image data from the camera among the signals received by the receiving processing unit 210. The edge server 128 further includes an attribute memory unit 218 for storing attributes 217 output by the attribute detection unit 216, an integration processing unit 224 for repeatedly integrating the moving object tracking results 213 stored in the moving object tracking result memory unit 214 and the attributes 217 stored in the attribute memory unit 218 at a period shorter than the second period and outputting a traffic condition overhead map 225 which is the analysis result after the integration, and a traffic condition overhead map memory unit 226 for accumulating and storing the traffic condition overhead map 225 output by the integration processing unit 224.

[0051] The moving object tracking results 213, attributes 217, and traffic condition overhead map 225 are calculated at predetermined time intervals, but analysis results calculated at a certain time in the past are also accumulated and stored as history in the moving object tracking result storage unit 214, attribute storage unit 218, and traffic condition overhead map storage unit 226. When the integration processing unit 224 performs integration processing, the history of the traffic condition overhead map 225, which is the past integrated analysis results accumulated in the traffic condition overhead map storage unit 226, may be referred to.

[0052] The edge server 128 further includes a vehicle tracking unit 220 for obtaining vehicle information consisting of identification information of the managed vehicle, on-board sensor information, and vehicle attributes such as position, speed, and vehicle size based on the signals received by the receiving processing unit 210 from each vehicle, and a vehicle information memory unit 222 for storing vehicle information 221 of each vehicle analyzed by the vehicle tracking unit 220.

[0053] The edge server 128 further includes an information transmission unit 228 that compares the moving object information of the traffic condition overhead map 225 with the vehicle information 221 stored in the vehicle information storage unit 222, and performs processing such as reporting traffic support information, such as information about moving objects with attributes that are considered to be dangerous, such as children and pedestrians walking while looking at their smartphones, accident vehicles on the road, broken-down vehicles, fallen objects, etc., in the integrated moving object information, to vehicles located within a specified range of the object, and a transmission processing unit 230 for transmitting a signal for the information report by the information transmission unit 228 to the target vehicle.

[0054] The edge server 128 further includes a high-resolution map storage unit 262 for storing a high-resolution map, and a high-priority area extraction unit 240 for extracting coordinates that specify a high-priority area based on the traffic condition bird's-eye view map stored in the traffic condition bird's-eye view map storage unit 226 and the high-resolution map stored in the high-resolution map storage unit 262. Here, the high-priority area is an area for which sensor data related to the area needs to be collected preferentially. Examples of high-priority areas include an area in which a number of moving objects exceeding a predetermined threshold value are present, and an area in which a moving object with an attribute that may behave dangerously, such as a child, is detected. The high-priority area extraction unit 240 extracts high-priority areas by dividing a road map into predetermined sections, for example, and determining whether each section meets the above-mentioned conditions. For the detection of an area where moving objects exceeding a threshold value are gathered, a plurality of areas (e.g., intersections) to be noted on the road map may be determined in advance, and only those areas may be examined to determine whether the conditions are met.

[0055] The edge server 128 further includes a sensor priority policy storage unit 242 that stores a plurality of sensor priority policies described later, a policy determination unit 244 that determines which of the sensor priority policies stored in the sensor priority policy storage unit 242 should be applied to each of the high priority areas extracted by the high priority area extraction unit 240 according to the situation of the extracted area, and a candidate vehicle selection unit 246 that selects, for each of the areas extracted by the high priority area extraction unit 240, vehicles that include the area in the detection range of the sensor as candidates for target vehicles for sensor data collection. That is, in the high priority area, the edge server 128 collects sensor data from only some of the vehicles. These some of the vehicles are vehicles equipped with high performance sensors and transmission devices. By selecting such vehicles and collecting sensor data, the edge server 128 can collect necessary sensor data with a margin even when there are many vehicles in the high priority area and the communication situation may deteriorate.

[0056] The sensor priority policy indicates a guideline for which type of sensor data from which sensor data is to be prioritized among sensor data. For example, assume that the presence of a child is detected in a certain area. Such an area is referred to as a child detection area here. Unlike adults, children may suddenly make unexpected movements. Therefore, in a child detection area, it is necessary to obtain information on the position of the detected moving object at a high frequency. Therefore, it is necessary to transmit sensor data from a sensor that can detect the position of a moving object at high speed, such as LiDAR, to the edge server 128 in preference to sensor data from a sensor that is difficult or time-consuming to detect the position of a moving object, such as a camera. On the other hand, for example, consider an area where an accident vehicle exists on the road. Here, such an area is referred to as an accident area. Since an accident vehicle does not usually move, there is no need to transmit the position coordinates to the edge server 128 at a high frequency. Also, in order to know the situation of the accident, it is easier for the driver of each vehicle to understand the situation if the information is distributed in the form of an image, such as a camera. Therefore, in such an area, it is better to prioritize a camera over a LiDAR. In this way, the sensor priority policy provides a basic guideline for which sensor is desirable to be preferentially transmitted to the edge server 128 depending on the situation of each area.

[0057] In this embodiment, it is assumed that each vehicle transmits sensor data to the edge server 128 in areas other than the high priority area, and at that time, transmits coordinates (relative coordinates with respect to the base position of the vehicle) that specify the detection range of the vehicle's sensor as vehicle information. Using these coordinates and the coordinates of the base position of the vehicle, the absolute coordinates of the range that the sensor of each vehicle can detect are calculated, and candidate vehicles can be selected by comparing these coordinates with the coordinates of each area.

[0058] The edge server 128 further includes a dataset query unit 250 for transmitting the sensor priority policy determined by the policy determination unit 244 for each of the candidate vehicles selected by the candidate vehicle selection unit 246, and querying a sensor dataset table consisting of the sensor dataset that each vehicle will transmit to the edge server 128 in accordance with the sensor priority policy; a sensor dataset table storage unit 248 for storing, for each vehicle, the sensor dataset table obtained from each candidate vehicle in response to the query by the dataset query unit 250; a vehicle selection unit 252 for selecting a vehicle that can transmit the most efficient sensor data for the edge server 128 to create a traffic condition overview map based on these sensor dataset tables, and transmitting an instruction to the selected vehicle to request the transmission of the sensor data; and a query transmission / reception unit 254 for communicating with each vehicle via wireless communication by the dataset query unit 250 and the vehicle selection unit 252.

[0059] Regarding the transmission of the most effective sensor data for creating a traffic situation overview map, various standards can be considered from the viewpoint of the designer. For example, it can be considered that the most important data from the viewpoint of driving support for traffic participants, that is, valuable data leading to accident prevention, is transmitted to the edge server 128 at a transmission speed sufficient for that purpose. From another viewpoint, it can be considered that the sensor data is transmitted to the edge server 128 so that the contents of the traffic situation overview map are as accurate as possible and can follow actual changes at a sufficient speed. In that case, it can be considered that it is important to divide the traffic situation overview map into several areas, change the importance of each area, and make it possible to reflect changes in the traffic situation overview map at a higher following speed for the important areas than for the other areas. In short, it corresponds to transmitting "the most effective sensor data most efficiently" to the edge server 128 to the edge server 128 from the viewpoint that the system designer most values ​​in maintaining and managing the traffic situation overview map, and to satisfy the standard that is set so that the load on the edge server 128 can be prevented from becoming excessive.

[0060] 5, the edge server 128 further includes a line speed information storage unit 256 that extracts line speed information when transmitting data from each section to the edge server 128 based on the sensor data and vehicle information received by the reception processing unit 210 from the vehicles in each section of the high-definition map and stores the extracted line speed information for a predetermined time, a communication status map management unit 258 that creates and manages a communication status map indicating a communication status for each section of the high-definition map stored in the high-definition map storage unit 262 based on the line speed information stored in the line speed information storage unit 256, and a communication status map storage unit 260 that stores the communication status map created by the communication status map management unit 258. When the data set inquiry unit 250 requests a transmission data set table set from each vehicle, it refers to this communication status map.

[0061] In FIG. 5, the inquiry transmission / reception unit 254 may use the same hardware as the reception processing unit 210 and the transmission processing unit 230. Whether or not the sensor priority policy is met can be determined quantitatively, but in this embodiment, it is determined in advance using a sensor data set table corresponding to each sensor priority policy as described later. The sensor data set table can be created based on subjective criteria, or can be created by formulating a certain formula based on various factors such as transmission capacity such as line speed, the number of high priority areas that the traffic situation bird's-eye map focuses on, the position of the vehicle in the area, the capacity of the transmission equipment of each vehicle, the number of moving objects existing in the area, and the like, and comparing the values ​​obtained by applying these factors to the formula. However, in practice, it is more realistic to use a method using the sensor data set table described below.

[0062] In this embodiment, it is assumed that a sensor data set table is prepared in advance for each vehicle. In response to an inquiry from the data set inquiry unit 250, each vehicle transmits the sensor data set table to the data set inquiry unit 250. The sensor data set table will be described later with reference to FIG. 9.

[0063] The sensor data set table to be used may be switched depending on the sensor priority policy adopted. In addition, each vehicle or infrastructure sensor may have at least a sensor for detecting the position of an object such as LiDAR or millimeter wave radar, and a sensor for acquiring an image such as a camera. In this embodiment, intersections with many moving objects and areas where children are present are both extracted as high priority areas of the same level. However, this disclosure is not limited to such an embodiment. Intersections are rather wide areas, and areas where children are detected are narrow areas, and the characteristics of the two are different. Therefore, only one of them may be extracted. In addition, a wide area may be extracted first, and then a narrow area may be extracted from that.

[0064] In addition, for both image sensors such as cameras and distance measurement sensors such as LiDAR and millimeter wave radar, the resolution and amount of sensor data vary depending on the product specifications. Therefore, it is necessary to prepare a sensor data set table that is compatible with each sensor priority policy, taking into full consideration the type of sensor to be targeted, the resolution of the sensor data, and the transmission interval.

[0065] 6 exemplarily illustrates the sensor arrangement and network configuration of a vehicle 82. Referring to Fig. 6, the vehicle 82 includes an in-vehicle network 288 having a gigabit-class transmission speed to which the above-described in-vehicle / outdoor cooperation unit 164 and the autonomous driving ECU 172 are connected, and sensor units 280, 282, 284, and 286 mounted on the right front, left front, right rear, and left rear of the vehicle 82, respectively.

[0066] Each of the sensor units 280, 282, 284, and 286 includes a millimeter wave radar, a camera, and a LiDAR.

[0067] In this embodiment, the in-vehicle network 288 includes four gigabit class network switches 292, 294, 296, and 298 to which sensors belonging to the same sensor unit are respectively connected, a first multi-gigabit switch 300 bridging between the two network switches 292 and 294 at the front of the vehicle, and a second multi-gigabit switch 302 bridging between the two network switches 296 and 298 at the rear of the vehicle and connected to the first multi-gigabit switch 300. The in-vehicle / external linking unit 164 is connected to the network switch 292, and the autonomous driving ECU 172 is connected to the network switch 294. A TCU (Telematics Control Unit) 290 corresponding to the external communication device 162 shown in FIG. 4 is connected to the network switch 292 together with the in-vehicle / external linking unit 164.

[0068] In this way, the sensor units 280, 282, 284, and 286 are disposed in different positions on the vehicle, so that the value of the sensor data from these sensor units may differ depending on the situation of the vehicle, as will be described later.

[0069] Fig. 7 shows a hardware configuration of the vehicle interior / exterior cooperation unit 164. Referring to Fig. 7, the vehicle interior / exterior cooperation unit 164 includes a microprocessor 320. The microprocessor 320 includes a bus 342, a CPU 340, a ROM (Read-Only Memory) 344, a RAM (Random Access Memory) 346, a DMAC (Direct Media Access Controller) 348, an input / output I / F (Interface) 352, a timer 350, all of which are connected to the bus 342, and an interrupt controller 354 that is connected to the DMAC 348, the timer 350, and the input / output I / F 352 and causes an interrupt to the CPU 340 in response to signals from these.

[0070] The interrupt controller 354 periodically generates a timer interrupt to the CPU 340 based on the time counted by the timer 350. The interrupt controller 354 also generates an input / output interrupt to the CPU 340 when there is an input / output from the outside of the input / output I / F 352. The interrupt controller 354 further generates an interrupt to the CPU 340 in response to a signal from the DMAC 348 when the data transfer by the DMAC 348 is completed.

[0071] Fig. 8 shows the RAM 346 in which the in-vehicle / out-of-vehicle cooperation unit 164 shown in Fig. 4 stores data for in-vehicle / out-of-vehicle cooperation. Among these data, fixed data is stored in the ROM 344 ​​shown in Fig. 7 and is loaded into the RAM 346 when the CPU 340 executes the program. Other data is dynamically generated by the CPU 340 when the program is executed and stored in the RAM 346.

[0072] 8, the RAM 346 stores mounted sensor information 400 which is information on sensors mounted on the vehicle 82, a sensor data set table 402 which stores a plurality of sensor data sets which are to be transmitted to the outside by the vehicle 82, a sensor priority policy table 404 which stores a plurality of sensor priority policies which indicate how to determine the priority of each of the types of sensors according to the vehicle situation etc. when a sensor data set for transmission to the outside is specified, high-definition map data 406 for a navigation system (not shown) mounted on the vehicle 82, and data from various sensors mounted on the vehicle 82. The sensor data storage area 408 temporarily stores sensor data transmitted to the vehicle interior / exterior link unit 164 for transmission to the edge server 128, and a packet priority policy table 410 holds information for setting packet transfer priority (packet priority policy) in table format so that when a sensor data set to be transmitted is determined and a sensor priority is determined, the network switches 292, 294, 296, 298, the first multi-gigabit switch 300, and the second multi-gigabit switch 302 shown in FIG. 6 transfer packets of sensor data with the priority according to the determined sensor data set. The packet priority policy is information for specifying packet priority that can be set in each switch, the number of queues for each packet priority, packet transfer frequency by priority, etc. Using this, a packet transfer policy by priority is set for each packet, and information specifying the priority is stored in the packet header of the sensor data, so that each switch executes transfer processing according to the priority.

[0073] Fig. 9 shows an example of the sensor data set table 402 stored in the RAM 346. Referring to Fig. 9, in this example, the sensor data set table 402 stores eight sensor data sets indicated by identification numbers (ID) 0 to 7 in the first column. In this embodiment, the identification numbers are determined so that when each sensor data set is selected, the smaller the data amount transmitted, the smaller the identification number, and the larger the data amount transmitted, the larger the identification number.

[0074] In the example shown in FIG. 9, LiDAR, millimeter wave radar, and camera are assumed as sensors. The second column in FIG. 9 shows the nature of the data transmitted in each sensor data set. In the example shown in FIG. 9, the resolution or compression rate at the time of transmission of the sensor data of LiDAR, millimeter wave radar, and camera are shown in order from the left. For example, in the sensor data set with identification number 2, LiDAR is uncompressed, millimeter wave radar is also uncompressed, and the image data of the camera is an SD (Standard Definition) image. An SD image is an image consisting of 720×480 pixels or 720×576 pixels. In addition, in the case of ID=5, LiDAR is uncompressed, millimeter wave radar is also uncompressed, but the camera image is a full HD (High Definition) image. A full HD image is an image consisting of 1900×1080 pixels. The "HD images" of identification numbers 3, 6, etc. are images consisting of 1280×780 pixels. These images are images obtained from the same camera, and the center part is common to images of any resolution. In other words, the HD image is an image of the central part of the full HD image, and the SD image is an image of the central part of the further HD image. By selecting data in this way, an image in the direction the camera is always facing is transmitted to the edge server 128.

[0075] The third column is the interval at which data is transmitted (data interval). In this example, this data interval is also determined for each ID and each sensor. In FIG. 9, for example, for identification number 1, the LiDAR and millimeter wave radar are both transmitted 10 times per second, and camera images are not transmitted. For identification number 5, the LiDAR and millimeter wave radar are both transmitted 10 times per second, and camera images are transmitted three times, full HD images, to the edge server 128.

[0076] The rightmost column indicates how many sensors' data will be transmitted from each type of sensor installed in the vehicle. For example, identification number 1 indicates that data from four sensors will be transmitted for both LiDAR and millimeter wave radar, and that no cameras will be used. For identification number 4, data from four sensors will be transmitted for both LiDAR and millimeter wave radar, but there will also be four cameras. In contrast, for identification number 6, the same four sensors will be used for LiDAR and millimeter wave radar, but only two cameras will be used.

[0077] In this way, the total amount of data to be transmitted can be controlled by specifying the resolution or compression rate of each sensor, the transmission interval, and the number of sensors to be used for each identification number. In other words, the content and total amount of sensor data transmitted from the vehicle to the edge server 128 can be controlled by specifying the sensor data set ID.

[0078] In the above description, the characteristics of the area are exemplified as a child detection area and an accident area, but the characteristics of the area are not limited to these. For example, adults who operate a smartphone should be given the same attention as children. The same applies when a bicycle with two or three people on it is detected. Furthermore, examples of areas where camera images are prioritized include areas where there are many parked vehicles, areas where traffic jams have existed for a long time, and areas where a broken-down vehicle is parked in a place where vehicles do not usually stop (broken-down vehicle area).

[0079] In this embodiment, as described above, each vehicle transmits a sensor data set table held by each vehicle to the edge server 128. After determining the high priority area, the edge server 128 refers to the sensor data set table of each vehicle, determines which vehicle transmits the sensor data set corresponding to which identification number, and requests the vehicle to transmit the sensor data by specifying the identification number of the sensor data set. When the vehicle receives this request, the vehicle determines whether or not all of the sensor data sets corresponding to the identification number transmitted from the edge server 128 can be transmitted within the allowable delay time based on the available transmission capacity, the delay time of data communication by the network in the vehicle, the processing time by the CPU mounted on the in-vehicle device, and the transmission delay to the edge server 128. If the determination is positive, the transmission of the specified sensor data set is started. If the determination is negative, the edge server 128 is requested to decrease the identification number by 1, and the transmission of the sensor data is not started. When the edge server 128 receives a response from the vehicle to decrease the identification number by 1, the edge server 128 selects another vehicle and requests the vehicle to transmit the specific sensor data set. If any vehicle starts transmitting a sensor data set, communication continues, and if all vehicles request that the identification number be decreased by 1, the amount of sensor data to be transferred is decreased, a value that is 1 lower than the previously specified number is specified as the identification number of the sensor data set, and the vehicle selection and the sensor data transmission request are repeated again. Details of this procedure will be described later with reference to FIG.

[0080] FIG. 10 shows an example of a sensor priority policy table 404 determined for the identification number of a sensor data set. The example shown in FIG. 10 corresponds to identification number 3 or 6 in FIG. 9. The number of sensors in the sensor data set with identification number 3 in FIG. 9 is "4, 4, 2". That is, there are four millimeter wave radars and four LiDARs, and two cameras. In the example shown in FIG. 10, if there are four millimeter wave radars, millimeter wave radars #0, 1, 2, and 3, they are adopted in this order. The same is true for LiDAR, and if there are LiDARs #0, #1, #2, and 3, they are adopted in this order. As for cameras, there are cameras #0, #1, #2, and 3, and among them, cameras #2 and 3 are adopted. In this way, which camera is adopted from among the four cameras is determined based on the value of the sensor data obtained from each camera.

[0081] Below, we will explain one example of how to determine which camera to use (how to create the sensor priority policy table 404) when only two of four cameras are to be used, as in the case of the camera described above.

[0082] Referring to FIG. 11, in this example, sensor type numbers 0, 1, and 2 are assigned to the millimeter wave radar, LiDAR, and camera, respectively, according to the type of the sensor. Furthermore, as shown in FIG. 12, sensor position type numbers are assigned according to the position type of the sensor (front / rear, left / right). In the example shown in FIG. 12, 0 is assigned when the sensor position is forward, 1 is assigned when the sensor position is backward, 0 is assigned when the sensor position is left, and 1 is assigned when the sensor position is right. In this example, the sensor position is a combination of front / rear and left / right. That is, in the case of cameras, as shown in FIG. 13, when the above-mentioned sensor type numbers are combined with the front / rear and left / right sensor position types for the front left camera 440, the front right camera 442, the rear left camera 444, and the rear right camera 446 of the vehicle 82, the following codes are assigned to each camera. Camera 440 200 Camera 442 201 Camera 444 210 Camera 446 211

[0083] By using this code, it is possible to determine which of the four cameras to select when only two are to be used. The method is explained below. The following explanation concerns a method in which the value of the image data obtained from each camera is determined, and a code according to the value is selected, thereby selecting the camera corresponding to that code.

[0084] With reference to FIG. 14, consider a target area 500 managed by the real space 50, and assume that an intersection 512 is located within this area. Assume that a building 514 exists near the intersection 512, and that the building 514 has a base station for high-speed data communication. Therefore, while high-speed communication is possible in a high-speed communication area 510 centered around the intersection 512, only low-speed communication is available in other areas. This information can be obtained from the communication status map 152 shown in FIG. 3. It is also known that an infrastructure sensor is installed at the intersection 512. This information can also be obtained from the traffic condition bird's-eye view map 150 shown in FIG. 3.

[0085] In this state, as shown by the vehicle 520 in FIG. 14, the vehicle is traveling in the high-speed communication area 510 toward the intersection 512. As described above, the intersection 512 is provided with an infrastructure sensor, and the information is transmitted to the edge server 128 at high speed. Therefore, even if the vehicle 520 transmits image data of the vicinity of the intersection 512 to the edge server 128, the overlap between the detection range of the camera of the vehicle 520 and the detection range of the infrastructure camera is large, and the data is duplicated and wasted. On the other hand, there is no infrastructure sensor behind the vehicle 520. Therefore, there is little or no overlap between the detection range of the camera of the vehicle 520 and the detection range of the surrounding infrastructure cameras, and the image data obtained by the camera behind the vehicle 520 is highly useful if it is transmitted to the edge server 128. Furthermore, since the vehicle 520 is in an area where high-speed communication is possible, HD images or full HD images can be used as the images.

[0086] Consider also vehicle 522 traveling in a direction away from intersection 512 in high-speed communication area 510. In this case, for the same reason as vehicle 520, an image of the rear of vehicle 522 is of low value to edge server 128. On the other hand, an image of the front of vehicle 522 is of high value to edge server 128. In the case of vehicle 522, since it is also in an area where high-speed communication is possible, HD images or full HD images can be used.

[0087] Furthermore, consider a vehicle 524 traveling on a road outside the high-speed communication area 510. Since the vehicle 524 is in a low-speed communication area, SD images should be used as the images. In addition, since there is no infrastructure camera near the vehicle 524, both the front image and the rear image of the vehicle 524 are useful to the edge server 128.

[0088] As described above, in this embodiment, the value of images from cameras that obtain images that do not overlap with images obtained by infrastructure cameras is set high, and the value of images from other cameras is set low, and a camera that obtains images with high value is selected. In other words, in this embodiment, the value of images from cameras that obtain images that complement the detection range of the infrastructure camera is set high, and the value of images that overlap with the detection range of the infrastructure camera is set low.

[0089] For example, assuming that each of the vehicles 520, 522, and 524 is equipped with four LiDARs, four millimeter wave radars, and four cameras as shown in FIG. 6, the following describes how to determine the sensor priority policy table 404 shown in FIG. 10.

[0090] The sensor type is represented by the symbols shown in Fig. 11, and the position of each sensor (front / back, left / right) is represented by the symbols shown in Fig. 12. These are coded in the order of (sensor type, front / back, left / right). The same applies to sensors other than cameras.

[0091] For example, in the case of vehicle 520, the rear camera is used regardless of whether it is the left or right camera. In other words, two of the four cameras are used. Since vehicle 520 is in an area where high-speed communication is possible, the sensor data set in this case is identification number "6" shown in FIG. 9. In this case, if the codes of the 12 sensors are sorted in descending order, these 12 sensors are arranged in the following order:

[0092] 211 210 201 200 111 110 101 100 011 010 001 000

[0093] Of these, the bottom four are millimeter wave radars, and all of these are adopted in the sensor data set with identification number 6. The middle four are LiDARs, and all of these are also adopted. Of the top four, the first two, "211" and "210", refer to the rear cameras of the vehicle 520. Since only two cameras are adopted in the sensor data set with identification number 6, these top two are adopted and the other cameras are not adopted.

[0094] Similarly, for vehicle 522, as a result of sorting the codes in ascending order, only the top code values ​​"200" and "201" are used. For vehicle 524, all cameras are used without the need for sorting.

[0095] By coding each sensor in this way and sorting the codes in ascending order if the front sensors have a higher value, or in descending order if the rear sensors have a higher value, the desired sensor can be selected.

[0096] In this embodiment, the entire code is sorted in ascending or descending order. However, this disclosure is not limited to such an embodiment. By using the second and third digits of the code as separate keys and sorting them in ascending or descending order, respectively, it is possible to select a sensor based on the values ​​defined for both the front and rear and the left and right.

[0097] "Program to realize edge server 128" 15 is a flowchart showing a control structure of a program for implementing the process of requesting specific vehicles in each area to transmit sensor data and collecting the sensor data by a computer in edge server 128. This program runs repeatedly at a predetermined time interval in parallel with the program for creating a traffic condition bird's-eye view map and the program for creating a communication condition map.

[0098] Referring to FIG. 15, this program includes step 550 for extracting high priority areas from among the areas managed by edge server 128 in accordance with the above-mentioned criteria, step 552 for executing the following process 554 for each area extracted in step 550, and step 556 for instructing vehicles that are not in any of the high priority areas, after completion of step 552, to transmit sensor data at a longer period (including the case where transmission is stopped) than vehicles transmitting sensor data from within the high priority areas, and terminating execution of the program.

[0099] The process 554 includes a step 570 of determining a sensor priority policy according to the characteristics of the area to be processed; a step 572 of calculating the detection range of the sensor of a vehicle present in the area to be processed from the data transmitted from the vehicle, and selecting as vehicle candidates a vehicle that includes the area to be processed in its detection range or that is likely to enter the detection range of the sensor of the area to be processed in the near future based on the moving speed and direction of the vehicle, and that is equipped with a predetermined sensor; a step 574 of determining, for each vehicle selected in step 572, an identification number of a sensor dataset based on the sensor dataset table received from the vehicle in accordance with the sensor priority policy determined in step 570; a step 576 of executing a process 578 of attempting to receive sensor data for each vehicle; a step 580 of determining whether or not reception of a sensor dataset has started from the target vehicle as a result of executing step 576, and branching the flow of control according to the determination; and a step 582 of decrementing the value of the identifier of the sensor dataset to be transmitted to each vehicle by 1 in response to the determination in step 580 being negative, and returning control to step 576. If the determination in step 580 is positive, this means that sensor data for the target area has been received, so processing 554 for that area ends and processing moves to the next area.

[0100] Process 578 includes step 590 of transmitting the identifier of the sensor data set determined in step 574 for the target vehicle and requesting the target vehicle to transmit the corresponding sensor data set, step 592 of judging whether transmission has started as a result of the processing of step 590 and branching the flow of control according to the judgment, step 594 of transmitting an instruction to other vehicles present in the high-priority area being processed to stop transmitting sensor data in response to the judgment of step 592 being positive, and step 596 of exiting the loop processing of process 578 for the high-priority area being processed. When the judgment of step 592 is negative (a request to decrease the identifier of the sensor data set by 1 has been received from the target vehicle), process 578 is terminated and process 578 is initiated for the next vehicle among the candidate vehicles in the high-priority area being processed.

[0101] It should be noted that although process 554 is performed for each area, there may be cases where multiple candidate vehicles exist within an area. In such cases, it is desirable to execute process 578 in order from vehicles that are considered to be most efficient at collecting sensor data. For example, the candidate vehicles are sorted using criteria such as the order of the number of types of sensors installed, the order of the largest transmission capacity, and the order of the shortest RTT (Round Trip Time: the time from sending a signal to receiving a response), and process 578 is executed from the vehicle with the highest ranking. Of course, this disclosure is not limited to such a method, and it is also possible to select vehicles in a random order, or to select vehicles in order of the length of time that they are expected to be in a high-priority area.

[0102] When reception of sensor data has started for all high priority areas, control proceeds to step 556, where vehicles present outside the high priority areas are instructed to transmit sensor data at a long interval, and the process ends. The identifier of the sensor data set transmitted from each vehicle at this time may be a predetermined one, or may be selected by the vehicle depending on the communication speed of the area in which the vehicle is located.

[0103] "Program to realize the in-vehicle and out-of-vehicle linkage unit 164" 16, a program executed by a computer to realize the in-vehicle / out-of-vehicle cooperation unit 164 shown in FIG. 6 is executed when some event is received by the in-vehicle / out-of-vehicle cooperation unit 164. Here, the event includes an interrupt to the CPU.

[0104] This program includes step 640 for determining whether the received event is an instruction to transmit a sensor data set and branching the control flow according to the determination. The instruction to transmit the sensor data set received in step 640 includes an identifier of one of the sensor data sets in the sensor data set table of the vehicle equipped with the on-board device executing this program.

[0105] The program further includes a step 642 of estimating a transmission delay required for transmitting the designated sensor data set to the edge server 128 based on the available wireless transmission capacity, the network delay in the vehicle, processing by the CPU, etc., in response to the determination in step 640 being affirmative; a step 644 of determining whether or not the designated sensor data set can be transmitted to the edge server 128 based on the transmission delay estimated in step 642, i.e., whether or not the transmission delay calculated in step 642 is equal to or less than the allowable delay, and branching the flow of control according to the determination; and a step 645 of determining whether or not the designated sensor data set can be transmitted to the edge server 128 based on the transmission delay estimated in step 642, i.e., whether or not the transmission delay calculated in step 642 is equal to or less than the allowable delay, and branching the flow of control according to the determination. In response to the determination in step 644, the method includes a step 650 of selecting a sensor that constitutes the sensor data determined by the specified identifier, setting network switches 292, 294, 296, 298, first multi-gigabit switch 300, and second multi-gigabit switch 302 in in-vehicle network 288 to transmit sensor data from the selected sensor preferentially, and terminating execution of the program; and in response to the determination in step 644 being negative, requesting edge server 128 to decrease the value of the identifier of the data set to be transmitted by a predetermined number (e.g., 1), and terminating execution of the program.

[0106] This program further includes step 660, in response to the determination in step 640 being negative, determining whether the received content is an instruction to stop data transmission from the edge server 128 and branching the control flow, and step 664, in response to the determination in step 660 being negative, resetting each switch in the in-vehicle network 288 (see FIG. 6) from the state set in step 650 to a default state and terminating execution of this program.

[0107] Furthermore, this program includes step 670, in response to the determination in step 660 being negative, determining whether the received data is an instruction from the edge server 128 indicating to transmit sensor data at a low speed, and branching the flow of control according to the determination, and step 674, in response to the determination in step 670 being positive, reselecting the sensor data to be transmitted at a low speed, and setting each switch in the in-vehicle network 288 (see FIG. 6) to preferentially forward the transmission of sensor data from the selected sensor, and terminating execution of this program.

[0108] This program further includes step 680, in response to the negative determination in step 670, determining whether the received data is sensor data from a sensor mounted on the vehicle and branching the control flow according to the determination, step 682, in response to the positive determination in step 680, transferring the received sensor data to the autonomous driving ECU 172 (see FIG. 6), step 683, following step 682, selecting a sensor and setting a switch according to a priority policy, step 684, following step 683, determining whether the sensor data received in step 680 is to be transferred to the edge server 128 and branching the control flow according to the determination result, step 686, in response to the positive determination in step 684, transferring the sensor data received in step 680 to the edge server 128 and terminating the execution of the program, and step 688, in response to the negative determination in step 680, performing processing according to the received data and terminating the execution of the program. When the negative determination in step 684 is made, nothing is done and the execution of the program is terminated.

[0109] In this embodiment, in steps 650, 664, and 674, before the sensor data is transferred, the presence or absence of transfer of the sensor data to the edge server 128 and the transfer speed are confirmed, and if necessary, the priority of the switch is set according to the confirmation result and the sensor data is transferred. If the transfer is not necessary, the sensor data is not transferred. Specifically, in this embodiment, the data output unit from each sensor is instructed to add information indicating that the sensor data packet from a predetermined sensor should be added to one of the priority queues of each switch set according to the priority policy. As a result, whether or not the sensor data should be transmitted, and if so, which sensor data should be transmitted, are controlled in almost real time.

[0110] Fig. 17 shows, in the form of a flowchart, a control structure of the program executed in step 642 of Fig. 16. Referring to Fig. 17, this program includes step 700 of executing process 702 of estimating end-to-end delay time from a sensor that is an output source of sensor data mounted on a vehicle to a destination (i.e., edge server 128) of the sensor data, and step 704 of determining, as a result of step 700, whether or not the estimated value of transmission delay for all sensor data is smaller than an allowable delay time, and branching the control flow in accordance with the determination result.

[0111] This program further includes step 706, in response to the determination in step 704 being affirmative, of setting a flag indicating whether or not the sensor data can be transmitted to the edge server 128 within the tolerable delay time to 0 and terminating execution of this program, and step 708, in response to the determination in step 704 being negative, of setting the flag to 9 and terminating execution of this program. In step 644 in Fig. 16, if the flag is 0, it is determined that transmission of the sensor data is possible, and if it is 9, it is determined that transmission of the sensor data is not possible.

[0112] Process 702 in FIG. 17 includes step 730 of confirming (estimating) the transmission capacity (wireless transmission delay in this case) from the in-vehicle / external communication unit 164 to the edge server 128, step 732 of confirming (estimating) the transmission delay of data from the sensor to be processed to the TCU 290 in the in-vehicle network 288 shown in FIG. 6, step 734 of confirming (estimating) the system processing time required by the CPU 340 (see FIG. 7) and the TCU 290 to transmit the sensor data, and step 736 of estimating the delay time that occurs in the transmission of the sensor data to be processed from the sensor to the edge server 128 by summing up steps 730, 732, and 734, and terminating process 702.

[0113] The wireless transmission delay that is a problem in step 730 is often caused by fluctuations in the transmission capacity of the wireless communication used by the TCU 290 in Fig. 6. The transmission capacity of the wireless communication may vary significantly due to switching between 4G communication and 5G communication, shadowing by a large vehicle, etc. In step 730, for example, the observation result when the most recent data communication was actually performed between the vehicle and the edge server 128 may be used as the transmission delay.

[0114] The transmission delay is calculated using the following formula.

[0115] Wireless transmission delay = sensor data volume / line speed +In-car network transmission delay + System Processing Time +Margin

[0116] The in-vehicle network transmission delay and the system processing time may be tabulated based on the most recent observation results. Alternatively, a margin may be set including the in-vehicle network transmission delay and the system processing time.

[0117] When calculating the delay time in this manner, it is necessary to synchronize the timer maintained in the vehicle 82 with an external reference timer. For example, when determining the line speed based on the time required to transmit data from the edge server 128 to the vehicle, an accurate transmission delay cannot be calculated unless the timer in the vehicle and the timer in the edge server 128 are synchronized. Therefore, the timer 350 shown in Fig. 7 is synchronized with an external time based on a protocol such as NTP (Network Time Protocol), AVB (Audio-Video Bridging), or TSN (Time-Sensitive Networking). In addition, by synchronizing the timers in each part of the vehicle with the timer 350, it is possible to prevent problems from occurring in the calculation of the transmission delay, etc.

[0118] The transmission delay of the in-vehicle network refers to the sum of the time taken for the sensor data from each sensor to reach the in-vehicle / external linking unit 164 through the network and the time taken for the sensor data to reach the TCU 290 from the in-vehicle / external linking unit 164 in FIG. 6. The processing time of the CPU refers to the processing time required for the CPU 340 shown in FIG. 7 to perform processing for transmitting the sensor data to the edge server 128. Since the in-vehicle / external linking unit 164 controls the vehicle, the in-vehicle network and the CPU resources need to have a margin for in-vehicle control. It is easy to check the status of the transmission capacity and the in-vehicle network, and it is easy to estimate the influence on the transmission of the sensor data. However, the CPU resources are easily affected by other processing, and are not clearly known until the transmission of the actual sensor data is started, so it is more difficult to estimate them than the other two elements. Therefore, the processing for transmitting the sensor data under various conditions may be performed, and the CPU resources consumed as a result may be measured and stored in the form of a table or the like.

[0119] Fig. 18 is a flowchart showing a control structure of a program for implementing a priority policy update process executed by CPU 340 of in-vehicle / out-of-vehicle cooperation unit 164 shown in Fig. 7 in response to an interrupt signal generated by timer 350 at predetermined time intervals. Referring to Fig. 18, this program includes step 800 of detecting the position of the host vehicle from an output of a GPS (Global Positioning System) or the like, step 802 of searching for infrastructure sensor equipment present around the host vehicle from map information based on step 800, and step 804 of determining whether the infrastructure sensor searched for in step 802 is within a predetermined distance ahead of the host vehicle and branching the control flow in response to the determination result.

[0120] This program further includes step 806, which, when the determination in step 804 is positive, determines whether or not there is an infrastructure sensor within a specified distance rear of the vehicle, and branches the control flow based on the determination result; step 808, which is executed when the determination in step 806 is positive, which compares the distance to the forward infrastructure sensor with the distance to the rear infrastructure sensor, determines whether the forward infrastructure sensor is farther away, and branches the control flow in response to the determination; step 812, which, when the determination in step 808 is positive, decides to give priority to the forward sensors among the sensors mounted on the vehicle and terminates execution of the program; and step 810, which, when the determination in step 808 is negative, decides to give priority to the rear sensors among the sensors mounted on the vehicle and stops execution of the program.

[0121] The program further includes step 814, in response to the determination in step 806 being negative, deciding to give priority to the rear sensor and terminating execution of the program; step 816, in response to the determination in step 804 being negative, deciding whether or not an infrastructure sensor is present within a predetermined distance behind the host vehicle and branching the control flow according to the determination; step 818, in response to the determination in step 816 being positive, deciding to give priority to the front sensor among the sensors mounted on the host vehicle and terminating execution of the program; and step 820, which is executed when the determination in step 816 is negative, randomly setting the priority order of the sensors mounted on the host vehicle and terminating execution of the program.

[0122] By executing this program, sorting the sensor codes in ascending or descending order according to the priority as described above, and further selecting the number of sensors with high priority for each sensor type determined by the sensor data set, a sensor priority policy table 404 for determining the sensor data set to be transmitted to the edge server 128 is obtained. This program is repeated at predetermined time intervals, and the sensor priority policy table 404 is updated each time. Therefore, if the predetermined time is set sufficiently short, an appropriate sensor priority policy table 404 can always be obtained in almost real time according to the vehicle position and surrounding conditions.

[0123] Fig. 19 is a flowchart showing a control structure of a program for achieving step 650 in Fig. 16. Referring to Fig. 19, this program includes step 850 of selecting a sensor that outputs sensor data to be transmitted to edge server 128 for each sensor type by performing the following process 852 on each sensor data in a sensor data set identified by an identifier received from edge server 128, and step 854 of setting a priority policy for packet forwarding corresponding to the priority determined in step 850 in each switch (or the output unit of each sensor) and terminating execution of this program.

[0124] The process 852 includes a step 870 of determining whether the number of sensors defined in the specified sensor data set is less than the number of sensors of the sensor type to be processed that are installed in the vehicle, and branching the control flow depending on the determination, a step 872 of determining the value of each sensor using the sensor priority policy table 404 when the determination in step 870 is positive, a step 874 of selecting, following step 872, a specified number of sensors for the sensor type to be processed by the specified sensor data set starting with the sensors with the highest value, and a step 878 of selecting all sensors of the sensor type to be processed and terminating the process 852 in response to the determination in step 870 being negative.

[0125] <Operation> With reference to FIG. 6, each sensor included in the sensor units 280, 282, 284, and 286 outputs sensor data according to the surrounding situation. These sensor data are transmitted to the vehicle interior / exterior cooperation unit 164 via the network switches 292, 294, 296, and 298, the first multi-gigabit switch 300, and the second multi-gigabit switch 302, etc. Among the sensors, the LiDAR measures the distance to the lattice point of the part of the moving object facing the LiDAR that exists within a predetermined range around, and outputs a distance measurement signal as a point cloud that is a collection of three-dimensional position coordinates of each point. The millimeter wave radar measures the distance to the moving object existing in the surroundings, and outputs the signal. The camera captures an image of the imaging range at a predetermined frame rate. Of the images, an image (SD image, HD image, full HD image) of the range specified by the sensor data set that includes the center of the image is output. These sensor data are transmitted to the autonomous driving ECU 172 shown in FIG. 6, and among them, the data of the sensor specified by the sensor data set specified by the edge server 128 is also transmitted to the edge server 128. When determining the sensor data set, the edge server 128 and the in-vehicle / out-of-vehicle cooperation unit 164 operate as follows.

[0126] A timer 350 shown in Fig. 7 generates a timer interrupt at predetermined time intervals and provides an interrupt signal to the CPU 340. In response to this interrupt signal, the CPU 340 executes a program whose control structure is shown in Fig. 18. By periodically executing this program, a priority policy for determining the priority when selecting a sensor according to a sensor data set from among a plurality of sensors possessed by the vehicle, in accordance with the relationship between the vehicle and surrounding infrastructure sensors, is updated in real time.

[0127] As described above, the reception processing unit 210 of the edge server 128 receives signals from the multiple infrastructure sensor equipment 180 and signals from the sensor 170 mounted on the vehicle 82, provides signals from the LiDAR and millimeter wave radar to the moving object tracking unit 212, and provides image data from the camera to the attribute detection unit 216. Meanwhile, the reception processing unit 210 provides information indicating the position, speed, and traveling direction of the vehicle from the information received from the sensor 170 to the vehicle tracking unit 220. The vehicle tracking unit 220 maintains data indicating the current position and moving direction of each vehicle based on the received information. Note that each vehicle transmits coordinates indicating the detection range of the sensor mounted on the vehicle to the edge server 128. The vehicle information storage unit 222 stores such information together with the coordinates of each vehicle.

[0128] The moving object tracking unit 212 of the edge server 128 analyzes the distance measurement signal from the LiDAR or the like received from the reception processing unit 210 to determine the position of each moving object in a first cycle. The time required for this analysis is short, and the moving object tracking result 213 is updated in the first cycle. The attribute detection unit 216 performs image analysis on the image data received from the reception processing unit 210 to determine the attributes of moving objects such as vehicles and people in the image. Since image processing takes time, the attribute detection unit 216 detects attributes in a second cycle, which is longer than the update cycle of the moving object tracking result 213 by the moving object tracking unit 212. The attributes 217 of the attribute detection unit 216 are stored in the attribute storage unit 218.

[0129] Referring to FIG. 5, the reception processing unit 210 receives sensor data and vehicle data from the infrastructure sensor equipment 180 and the vehicle 82 in the target area. The moving object tracking unit 212 detects the position of the moving object in the target area from the data and tracks the movement of the moving object. The tracking results are stored in the moving object tracking result storage unit 214. Similarly, the attribute detection unit 216 detects the attributes of each moving object. The detected attributes of the moving object are stored in the attribute storage unit 218. The integration processing unit 224 integrates the tracking results of the moving objects and the attributes of the moving objects, creates a traffic situation overhead map 225 using a high-resolution map stored in the traffic situation overhead map storage unit 226, and stores the data in the traffic situation overhead map storage unit 226. The integration processing unit 224 constantly updates the traffic situation overhead map 225 stored in the traffic situation overhead map storage unit 226 based on newly received data.

[0130] The vehicle tracking unit 220 stores the vehicle information 221 received from each vehicle in the vehicle information storage unit 222. The vehicle tracking unit 220 further tracks each vehicle in the area based on the vehicle information 221 stored in the vehicle information storage unit 222 and the newly received vehicle information 221. The vehicle tracking unit 220 further requests a vehicle that has newly entered the area to transmit a transmission data set table, and receives the received transmission data set table in the sensor data set table storage unit 248.

[0131] When information to be transmitted to each vehicle occurs in the traffic condition overhead map 225, the information transmission unit 228 generates the information to be transmitted to each vehicle from the data stored in the traffic condition overhead map memory unit 226, and transmits it to each vehicle via the transmission processing unit 230.

[0132] The line speed information storage unit 256 extracts and stores a set of the transmission time held in the data received by the reception processing unit 210 from each vehicle in the area, the reception time at the edge server 128, and the position of each vehicle stored in the vehicle information storage unit 222. The communication status map management unit 258 creates a communication status map relating to the transmission time between the vehicle and the edge server 128 for each section obtained by dividing the map into a grid based on the data stored in the line speed information storage unit 256 and the high-resolution map stored in the high-resolution map storage unit 262, and stores the communication status map in the communication status map storage unit 260. The communication status map management unit 258 always keeps the communication status map storage unit 260 in an up-to-date state based on the most recent line speed information stored in the line speed information storage unit 256.

[0133] The high priority area extraction unit 240, the policy decision unit 244, the candidate vehicle selection unit 246, the data set inquiry unit 250, and the vehicle selection unit 252 operate as follows: Referring to Fig. 15, the following process is repeated at a predetermined time.

[0134] First, the edge server 128 extracts high-priority areas from among the areas covered by the edge server 128 according to the above-mentioned criteria (step 550). Then, for each area extracted in step 550, a process 554 is executed in step 552.

[0135] In process 554, a sensor priority policy according to the characteristics of the area to be processed is determined (step 570). Next, in step 572, the detection range of the sensor of the vehicle present in the area to be processed is calculated from the data transmitted from the vehicle, and it is determined whether the vehicle includes the area to be processed in the detection range of the sensor, and whether the vehicle is likely to enter the area to be processed in the near future and is equipped with a predetermined sensor, and a vehicle that satisfies the conditions is selected as a vehicle candidate. Next, in step 574, for each vehicle selected in step 572, a sensor data set to be collected from the vehicle is determined and stored according to the sensor priority policy determined in step 570, the communication status map, and the vehicle information related to the sensor equipped in each vehicle.

[0136] In step 576, process 578 is executed for each candidate vehicle selected in step 572. In this embodiment, process 578 is executed in order from the candidate vehicles having the most desirable attributes (type, number and performance of sensors installed, data transmission performance, etc.).

[0137] Process 578 sends a transmission of the sensor data along with the identifier of the sensor data set determined in step 574 to the target vehicle.

[0138] 16, in the vehicle receiving the sensor data, the determination in step 640 becomes positive, step 642 is executed, and the transmission delay when transmitting the specified sensor data set to the edge server 128 is estimated. This process is as shown in FIG. 17. That is, for each sensor data, the wireless transmission delay (step 730), the in-vehicle network transmission delay (step 732), and the system processing time (step 734) are confirmed (estimated), and based on them, the total end-to-end delay time from the sensor to be processed to the edge server 128 is estimated. If the estimated value of the transmission delay for all the sensor data is smaller than the allowable delay, the flag is set to 0 in step 706. If the estimated value of the transmission delay for even one of the sensor data is equal to or greater than the allowable delay time, the flag is set to 9 in step 708.

[0139] If it is determined in step 644 that the transmission delay calculated in step 642 is less than the allowable delay and therefore the specified sensor data set can be transmitted to the edge server 128, then in step 650 a sensor for transmitting sensor data to the edge server 128 is selected according to the sensor data set, and each switch is configured to preferentially forward the sensor data from that sensor.

[0140] That is, referring to Fig. 19, the process 852 is executed for each sensor data in the sensor data set corresponding to the identifier specified by the edge server 128. In the process 852, it is first determined whether the number of sensors specified by the sensor data set for a certain type of sensor is less than the total number of sensors equipped in the vehicle. If the determination is negative, the sensors of the target type mounted on the vehicle are selected as sensors that transmit sensor data to all edge servers 128 (step 878). If the determination in step 870 is positive, in step 872, the value of each sensor is determined according to the sensor priority policy at that time, and in step 874, sensors starting from the sensor with the highest value are selected up to the number of sensors determined by the specified sensor data set. When sensors are selected for all sensor types in this way, in step 854, each switch is set to transfer sensor data from the center selected with priority.

[0141] As a result, the selected sensor starts outputting sensor data addressed to the edge server 128. When the CPU 340 (FIG. 7) receives this sensor data, the sensor data is transferred to the autonomous driving ECU 172 in step 682 via a route of step 640 → step 660 → step 670 → step 680 in FIG. 16. Furthermore, if the sensor data is to be transferred to the edge server 128, the sensor data is transferred to the edge server 128 via a route of step 684 → step 686.

[0142] 15 again, in this case, the determination in step 592 becomes positive (flag=0), the processes in steps 594 and 596 are executed, and the process in step 576 ends. In the following step 580, it is determined that the transmission of sensor data from the target vehicle is being executed, so the process for the target area ends, and the process 554 is executed for the next high priority area.

[0143] If the transmission delay in the vehicle that requested the transfer of the sensor data set in step 590 is equal to or greater than the allowable delay time, the following process is executed.

[0144] 16, in this case, the process is performed in the order of step 640 → step 642 → step 644. Since the determination in step 644 is negative, the process in step 648 is executed. That is, the vehicle transmits information to the edge server 128 requesting that the identifier of the sensor data set be decreased by 1.

[0145] 15 again, in the edge server 128 that receives this request, the determination in step 592 is negative (flag ≠ 0). Execution of process 578 for the target vehicle is terminated, and process 578 is executed for the next vehicle. When sensor data cannot be transferred for all of the candidate vehicles within the allowable delay time, the determination in step 580 is negative, and in step 582, 1 is subtracted from the identifier value of the sensor data set for each vehicle. Then, process 578 is repeated for each vehicle using the new identifier. This process is repeated until a vehicle capable of transmitting sensor data is found. Although not shown in the figure, when transmission is not possible for all of the candidate vehicles, the process can be restarted from step 570.

[0146] Once sensor data transmission has commenced from the selected vehicles for all of the targeted high priority areas, step 556 is executed, in which instructions are sent to vehicles outside the area indicating that they should transmit sensor data at longer intervals.

[0147] 16, when each vehicle receives this instruction, step 674 is executed via the path of step 640 → step 660 → step 670 in FIG. 16. In step 674, the sensors are selected, data is output from the sensors, and the switches in the network are reconfigured (reset) so that a data set different from that in step 650 (in most cases, all sensor data) is sent to the edge server 128 at long intervals. As a result, sensor data is sent to the edge server 128 at long intervals from vehicles other than the selected vehicle.

[0148] For vehicles other than those selected in the high-priority area, an instruction to stop transmitting sensor data is sent from the edge server 128. In this case, for those vehicles, the process of step 664 is executed via the route of step 640 → step 660 in Fig. 16. That is, step 664 is reset, and all sensor data is excluded from being transferred to the edge server 128.

[0149] The operation of the program executed in each vehicle when a data set transmission instruction, a data transmission stop instruction, and a low-speed data transmission instruction are received has been described above with reference to Fig. 16. When CPU 340 receives sensor data from a sensor, the sensor data is transferred to autonomous driving ECU 172 (see Fig. 6) in step 682 via a route of step 640 → step 660 → step 670 → step 680, and in subsequent steps 684 and 686, if the sensor data is to be transferred to edge server 128, it is transferred to edge server 128, and if not, nothing is done and execution of this program is terminated.

[0150] If it is determined in step 680 that the received data is not sensor data, control proceeds to step 688, where processing is performed according to the received data, and processing ends.

[0151] As described above, according to this embodiment, in an area where a predetermined condition is satisfied (high priority area), only a vehicle that can transmit the most effective sensor data for creating a traffic situation overview map to the edge server 128 most efficiently within a predetermined allowable delay according to a sensor priority policy corresponding to the condition of the area transmits the sensor data, and other vehicles do not transmit the sensor data. The amount of data transmitted to the edge server 128 is not excessive, and the traffic situation overview map can be created and maintained in real time in a timely manner. In this case, the transmission delay is estimated by estimating the transmission capacity (line speed) from the transmitter of the vehicle to the edge server 128, as well as the transmission delay of the in-vehicle network and the processing time in the CPU, and then adding them up, and comparing them with the allowable delay by adding a margin. As a result, the edge server 128 can collect information required for creating and maintaining the traffic situation overview map from the vehicle selected in consideration of the line speed with the edge server 128 and the factors of the transmission delay in the vehicle with a high probability. As a result, the risk of communication congestion between the vehicle and the edge server 128 in the target area can be reduced. For the target area, a sensor priority policy according to the reason for the target area is adopted, and vehicles that can transmit sensor data in a form that conforms to the sensor priority policy are selected. Therefore, the most necessary data according to the traffic situation is transmitted to the edge server 128, and a traffic situation overview map that fully reflects the necessary information can be created.

[0152] 17, an end-to-end transmission delay time of the sensor data from each of the multiple sensors mounted on the vehicle 82 to the edge server 128 is estimated. The process shown in FIG. 18 and the process of step 872 shown in FIG. 19 determine the value of the sensor data output by the multiple sensors mounted on the vehicle 82 based on the vehicle state including the position and attitude of the vehicle 82, the installation positions of the multiple sensors mounted on the vehicle 82 on the vehicle 82, and the overlap state of the detection ranges of the multiple sensors and the detection ranges of the infrastructure sensors. The process of step 874 in FIG. 19 selects a part of the sensor data based on the end-to-end transmission delay time estimated in the process 702 in FIG. 17, and the value determined in the process of FIG. 18 and step 872 in FIG. 19 according to the position, traveling direction, etc. of the vehicle 82, and transmits a copy of the sensor data to the edge server 128 via the TCU 290 in FIG. 6. The end-to-end transmission delay time from the sensor to the edge server 128 is estimated, including the delay time in the in-vehicle network, and sensor data to be transmitted to the edge server 128 is selected based on the estimated transmission delay time and the value determined for each piece of sensor data by the process in Fig. 18 and step 874 in Fig. 19. The in-vehicle / out-of-vehicle cooperation unit 164 selects and transmits sensor data that can be effectively used by the edge server 128, taking into account the end-to-end transmission delay time from the sensor to the edge server 128 while following fluctuations in transmission capacity by the TCU 290, so that the edge server 128 can function as well as possible while making the most of the available transmission capacity.

[0153] In addition to the processing time in the CPU 340 until the sensor data that has arrived at the input / output I / F 352 shown in Fig. 7 is input to the TCU 290 in Fig. 6, and the communication delay time required for the sensor data to be transmitted from the TCU 290 to the edge server 128, the network delay time until the sensor data from multiple sensors reaches the input / output I / F 352 in Fig. 7 via the in-vehicle network 288 shown in Fig. 6 is estimated for each sensor, and the end-to-end transmission delay time from each sensor to the edge server 128 is calculated based on these. Since the end-to-end transmission delay time is calculated taking into account the transmission delay time of the in-vehicle network 288, the amount of data that can be transmitted to the edge server 128 within the allowable delay time can be more accurately estimated, and the sensor data to be transmitted can be more accurately selected.

[0154] Among the sensors mounted on the vehicle 82, sensor data from a sensor having a large overlapping detection range with the detection range of an infrastructure sensor has low utility value in the edge server 128. The smaller the overlapping detection range of a sensor with the detection range of an infrastructure sensor, the more valuable the sensor data output by that sensor is in the edge server 128. Therefore, by estimating the detection range of an infrastructure sensor present in the vicinity of the vehicle 82 using an algorithm such as that illustrated in FIG. 18 described with reference to FIG. 14, the value of a sensor whose detection range overlaps with the detection range of an infrastructure sensor is calculated to be high and that of a sensor whose detection range overlaps with the detection range of an infrastructure sensor is calculated to be low, and the value of a sensor whose detection range overlaps with the detection range of an infrastructure sensor is calculated to be low, and the sensor data from those sensors is preferentially selected and transmitted to the edge server 128. Since high-value sensor data is transmitted to the edge server 128 within a limited transmission capacity, the edge server 128 can perform stable processing based on the sensor data regardless of fluctuations in the transmission capacity. Note that the algorithm illustrated in FIG. 18 does not check the details of the detection range of the infrastructure sensor, but it satisfies the above-mentioned condition in that it selects at least a sensor whose detection range is unlikely or almost unlikely to overlap with the infrastructure sensor among the sensors mounted on the vehicle 82.

[0155] As shown by the algorithm in Fig. 18, the detection range of the infrastructure sensor is updated at a predetermined time interval. The value of the sensor data is also updated at a predetermined time interval based on the position and attitude of the vehicle 82 and the updated detection range of the infrastructure sensor as shown in Fig. 18. Therefore, the value of each sensor of the vehicle 82 is updated at least at a predetermined time interval in accordance with the movement of the vehicle 82 and the environment around the vehicle 82. As a result, regardless of the movement of the vehicle 82 and changes in the environment around the vehicle, high-value sensor data is transmitted to the edge server 128. Therefore, regardless of changes in the distribution of vehicles and the environment, the edge server 128 can perform stable processing.

[0156] 19, the value of sensor data is set for each type of sensor, so that effective sensor data for processing by the edge server 128 can be selected by making more effective use of transmission capacity.

[0157] As can be seen from the algorithm shown in Fig. 19, when an identifier specifying one of the sensor data sets is provided to the in-vehicle / out-of-vehicle linking unit 164, the sensor data set corresponding to the identifier is determined as shown in Fig. 9. The number of sensors specified by the sensor data set are selected for each sensor type in descending order of the value of the sensor data. For all types of sensors, a predetermined number of sensor data items having value based on the designation by the sensor data set are transmitted to the edge server 128. As a result, the edge server 128 can effectively use the sensor data to perform stable processing.

[0158] 6 operates based on sensor data from multiple sensors mounted on the vehicle 82. Among the sensor data, sensor data selected by the vehicle interior / exterior cooperation unit 164 is transmitted to the edge server 128, so that the sensor data can be shared between the autonomous driving ECU 172 and the edge server 128, and autonomous driving of the vehicle and processing of the edge server 128 can be efficiently performed.

[0159] <Second embodiment> <composition> In the first embodiment, a part of the sensor data transmitted to the automatic driving ECU 172 is transmitted to the edge server 128. However, this disclosure is not limited to such an embodiment. Referring to FIG. 20, a traffic assistance system 900 according to the second embodiment includes a vehicle 914 having a configuration similar to that of the vehicle 82 according to the first embodiment, a vehicle 916 having a configuration similar to that of the vehicle 82 instead of the vehicle 84 in FIG. 4 but equipped with a remote control ECU (not shown) instead of the automatic driving ECU 172 shown in FIG. 4 and FIG. 6, and a remote control server 910 for receiving sensor data from the vehicle 916 via the Internet 912, generating a signal (remote control command) for controlling the remote control ECU of the vehicle 916 based on the sensor data, and transmitting the signal to the vehicle 916 via the Internet 912.

[0160] 21, a vehicle 916 includes an in-vehicle device 932 instead of the in-vehicle device 92 shown in Fig. 4, and a remote control ECU 930 instead of the automatic driving ECU 172 shown in Fig. 4. The in-vehicle device 932 includes an NSW 950, an in-vehicle / external cooperation unit 952, and an external communication unit 954 instead of the NSW 160, the in-vehicle / external cooperation unit 164, and the external communication unit 162 of the in-vehicle device 92. The remote control ECU 930 does not have an automatic driving function like the automatic driving ECU 172, and in this embodiment, it simply operates according to remote driving commands transmitted from the remote control server 910.

[0161] 21, the in-vehicle device 932 does not transmit sensor data from the sensor 170 to the remote control ECU 930, but transmits it to the remote control server 910 via the NSW 950, the in-vehicle / outdoor cooperation unit 952, and the exterior communication device 954, as shown by the arrow 960. The remote operation command transmitted from the remote control server 910 is received by the in-vehicle / outdoor cooperation unit 952 via the reverse path of the sensor data transmission path, and is provided to the remote control ECU 930 via the NSW 950.

[0162] In such an embodiment, basically, all of the sensor 170 is transmitted to the remote control server 910, while a part of it is transmitted to the edge server 128. For this purpose, the control structure of a program executed by the CPU of the vehicle interior / exterior cooperation unit 952 is shown in FIG.

[0163] Referring to Figure 22, this program differs from the program shown in Figure 16 in that, instead of step 682 in Figure 16, it includes step 972 for transferring sensor data to the remote control server 910 when the sensor data is received, step 980 for determining whether the received data is a remote control command from the remote control server 910 and branching the control flow in response to a negative determination in step 680, and step 982 for transferring the received remote control command to the remote control ECU 930 shown in Figure 21 and terminating execution of this program when the determination in step 980 is positive.

[0164] As can be easily understood by comparing Figures 4 and 21, and Figures 16 and 22, respectively, the vehicle 916 of this embodiment operates in the same manner as the vehicle 82 of the first embodiment and can achieve the same effects, except that it operates by remote control rather than autonomous driving.

[0165] In the second embodiment, the hardware may be the same as that shown in the first embodiment except for the remote control ECU 930, and the difference from the first embodiment is in the control structure of the program executed by the vehicle 916, etc. The edge server 128 may be exactly the same as that in the first embodiment.

[0166] As described above, in the second embodiment, vehicles that use sensor output for remote control rather than autonomous driving can also be used as targets for collecting sensor data. This makes it possible to broaden the targets for collecting sensor data, and while minimizing the impact on vehicles with autonomous driving functions, it is also possible to maintain high quality of the traffic situation overview map maintained by the edge server.

[0167] 20 and 21 operates based on sensor data received from a sensor 170 mounted on a vehicle 916, and transmits a remote control command to a remote control ECU 930. Any of the sensor data transmitted to the remote control server 910 is selected by an in-vehicle / out-of-vehicle cooperation unit 952 and transmitted to the edge server 128. Therefore, the sensor data can be shared between the remote control server 910 and the edge server 128, and the vehicle 916 can be remotely controlled and the edge server 128 can perform processing efficiently.

[0168] In the above embodiment, the line speed is measured by the vehicle interior / exterior linkage device to determine whether the sensor data set can be transmitted. However, this disclosure is not limited to such an embodiment. For example, the edge server 128 may transmit the line speed between each vehicle and the edge server 128, which is estimated from the communication status map 152 (see FIG. 3) in the area managed by the edge server 128 and the position of each vehicle, to the vehicle interior / exterior linkage device. In this case, the vehicle interior / exterior linkage device uses the line speed to determine whether the sensor data of the sensor data set can be transmitted.

[0169] In the above embodiment, the edge server 128 further collects sensor data set tables from each vehicle, and requests the candidate vehicles to transmit sensor data by specifying the identifiers of the sensor data sets based on the tables. However, this disclosure is not limited to such an embodiment. For example, it is possible to inquire each vehicle about a data set that can transmit the most sensor data from a specified sensor to the edge server 128 according to conditions such as communication speed, without the edge server 128 collecting the sensor data set tables. The edge server 128 may select the vehicle that can transmit the most data among the vehicles that responded.

[0170] Furthermore, in the above embodiment, the edge server 128 and the vehicle (mainly the vehicle interior / exterior cooperation unit 164) cooperate with each other to determine the sensor data set to be transmitted from the vehicle to the edge server 128. However, this disclosure is not limited to such an embodiment, and the vehicle interior / exterior cooperation unit 164 may independently determine the sensor data set. For example, when the sensor is a camera, if an image output by the camera becomes blurred, it is possible to stop transmitting the image from that camera to the edge server 128 and send an image from another camera that can obtain a clear image to the edge server 128. Whether an image is clear or not may be determined, for example, by performing a Fourier analysis on the image and based on whether or not there are many high frequency components.

[0171] In the above embodiment, in order for the in-vehicle / outside linking unit 164 to accurately estimate the line speed, time synchronization using a timer as shown in Fig. 7 is essential. However, the accuracy of time synchronization depends on the protocol used for time synchronization and the hardware performance of the in-vehicle / outside linking unit 164 and an ECU having a similar configuration. Therefore, it is desirable to set a large margin in the calculation of the delay time when the synchronization error is large, and to set a small margin when the synchronization error is small.

[0172] In this embodiment, computer programs for operating the vehicle interior / exterior linking units 164 and 952 as the functions of the vehicle interior / exterior linking unit 952 and its components are written in the ROM 344 ​​when the vehicle interior / exterior linking units 164 and 952 are shipped. However, if necessary, the computer programs written in the ROM 344 ​​can be updated via a network. Furthermore, by connecting a removable storage medium reading device to the in-vehicle device 932, these computer programs can also be updated from a medium such as a CD-ROM (Compact Disk-Read Only Memory) or a DVD-ROM (Digital Versatile Disk-Read Only Memory). In that case, it is conceivable that CD-ROMs or DVD-ROMs, etc., on which the computer programs are recorded, are distributed on the market.

[0173] The embodiments disclosed herein are merely examples, and the disclosure is not limited to the above-described embodiments. The scope of the disclosure is indicated by the claims in the scope of the claims after taking into consideration the detailed description of the disclosure, and includes all modifications within the meaning and scope equivalent to the wording described therein. [Explanation of symbols]

[0174] 50 Real Space 52, 150, 225 Traffic situation map 70 Communication Systems 82, 84, 520, 522, 524, 914, 916 vehicles 86 Pedestrians 88 Infrastructure Camera 90 Traffic Signal Controller 92, 94, 932 On-vehicle equipment 96 Mobile Phones 98, 100 Wireless communication equipment 110, 112, 114 base station 120, 122 Metro NW 124, 130 Distributed DC 126, 128 Edge Server 140 Core Network 142 Core DC 144 Core Server 152 Communication Status Map 160, 950 NSW 162, 954 External communication device 164, 952 Vehicle Interior and Exterior Coordination Department 170 Sensors 172 Autonomous Driving ECU 174, 960 Arrow 180 Infrastructure Sensor Equipment 190 Infrastructure Sensors 192 Communication Equipment 210 Receiving processing unit 212 Motion Tracking Unit 213 Motion Tracking Results 214 Motion tracking result memory unit 216 Attribute detection unit 217 attributes 218 Attribute storage section 220 Vehicle Tracking Department 221 Vehicle Information 222 Vehicle information storage unit 224 Integrated Processing Unit 226 Traffic situation overview map memory unit 228 Information Transmission Department 230 Transmission Processing Unit 240 High Priority Area Extraction Unit 242 Sensor Priority Policy Memory 244 Policy Decision Department 246 Candidate Vehicle Selection Department 248 Sensor Data Set Table Storage 250 Dataset Query Part 252 Vehicle Selection Department 254 Inquiry Transmission and Reception Unit 256 Line speed information storage section 258 Communication Status Map Management Department 260 Communication status map storage unit 262 High-resolution map storage unit 280, 282, 284, 286 Sensor Unit 288 In-Vehicle Network 290 TCU 292, 294, 296, 298 Network switches 300 The first multi-gigabit switch 302 Second Multi-Giga Switch 320 Microprocessor 340 CPU 342 Bus 344 ROM 346 RAM 348 DMAC 350 Timer 352 Input / Output Interface 354 Interrupt Controller 400 Sensor information 402 Sensor Data Set Table 404 Sensor Priority Policy Table 406 High-resolution map data 408 Sensor data storage area 410 Packet Priority Policy Table 440, 442, 444, 446 Camera 500 Target Area 510 High-speed communication area 512 Intersection 514 Building 550, 552, 556, 570, 572, 574, 576, 580, 582, 590, 592, 594, 596, 640, 642, 644, 648, 650, 660, 664, 670, 674, 680, 682, 683, 684, 686, 688, 700, 704, 706, 708, 730, 732, 734, 736, 800, 802, 804, 806, 808, 810, 812, 814, 816, 818, 820, 850, 854, 870, 872, 874, 878, 972, 980, 982 Steps 554, 578, 702, 852 Processing 900 Traffic Support System 910 Remote Control Server 912 Internet 930 Remote Control ECU

Claims

1. A vehicle interior / exterior linking device used in a vehicle equipped with a wireless communication device that wirelessly communicates data with the outside of the vehicle and a plurality of sensors, a data receiving unit that receives sensor data from the plurality of sensors; a delay time estimation unit that estimates a transmission delay time of the sensor data from each of the plurality of sensors to a predetermined device that can communicate with the wireless communication device; a selection unit that selects a portion of the sensor data based on the transmission delay time estimated by the delay time estimation unit, and transmits the selected sensor data to the predetermined device via the wireless communication device; a value determination unit that determines a value of sensor data output from the plurality of sensors based on a state of the vehicle and states of the plurality of sensors; the selection unit includes a data selection unit that selects a portion of the sensor data based on the transmission delay time estimated by the delay time estimation unit and the value determined by the value determination unit, and transmits the selected sensor data to the predetermined device via the wireless communication device; The value determination unit is an infrastructure detection range estimation unit that estimates a detection range of an infrastructure sensor present in the vicinity of the vehicle based on a position of the vehicle; a value setting unit that estimates the detection ranges of the multiple sensors based on the position and attitude of the vehicle, and sets a value of the sensor data from each sensor so that the value of the sensor data from the sensor is greater for a sensor with a smaller area overlapping with the detection range of the infrastructure sensor estimated by the infrastructure detection range estimation unit.

2. The vehicle interior / exterior communication device according to claim 1 , wherein the delay time estimation unit includes an end-to-end delay time estimation unit that estimates a transmission delay time of the sensor data between ends from each of the plurality of sensors to a predetermined device that can communicate with the wireless communication device.

3. The end-to-end delay time estimation unit a transmission time estimation unit that estimates a delay time for each of the sensors until the sensor data from the plurality of sensors arrives at the data receiving unit; a processing time estimation unit that estimates a processing time required for processing the sensor data that has arrived at the data receiving unit until the sensor data is input to the wireless communication device; a wireless communication delay time estimation unit that estimates a wireless communication delay time required for the sensor data to be transmitted from the wireless communication device to the predetermined device; The vehicle interior / exterior linking device according to claim 2 , further comprising: a delay time calculation unit that calculates the transmission delay time based on an amount of data from each of the plurality of sensors, the delay time, the processing time, and the wireless communication delay time.

4. the infrastructure detection range estimation unit includes an infrastructure detection range update unit that repeatedly updates an estimated detection range of an infrastructure sensor present in the vicinity of the vehicle at a predetermined time interval based on a position of the vehicle; 2. The vehicle interior / exterior integration device according to claim 1, wherein the value setting unit includes a value updating unit that repeatedly updates the value of the sensor data from each sensor at a predetermined time interval based on the position and attitude of the vehicle, so that the value of a sensor having a smaller area overlapping with the detection range of the infrastructure sensor updated by the infrastructure detection range updating unit is higher.

5. Each of the plurality of sensors is classified into one of a plurality of sensor types; the infrastructure detection range estimation unit includes a sensor type detection range estimation unit that estimates the detection range of the infrastructure sensor present in the vicinity of the vehicle for each sensor type of the infrastructure sensor based on a position of the vehicle; 2. The vehicle interior / exterior collaboration device according to claim 1, wherein the value setting unit includes a sensor type sensor data value setting unit that estimates, for each of the plurality of sensor types, a detection range of each of the sensors belonging to that sensor type among the plurality of sensors based on a position and attitude of the vehicle, and sets a value of sensor data from a sensor belonging to that sensor type among the plurality of sensors such that a sensor having a smaller area overlapping with the detection range of the infrastructure sensor estimated for that sensor type by the sensor type detection range estimation unit has a higher value.

6. The vehicle further comprises an in-vehicle network for data transfer; The vehicle interior / exterior linking device according to claim 5 , wherein the data receiving unit includes a network receiving unit that receives the sensor data from the plurality of sensors via the vehicle interior network.

7. The sensor data set storage unit stores information specifying a plurality of sensor data sets together with an identifier of each of the sensor data sets, the plurality of sensor data sets include information for specifying, for each of the plurality of sensor types, the number of sensors to be selected from the plurality of sensors and a data format related to the amount of sensor data from the corresponding sensor; The vehicle interior / exterior linking device further includes: a sensor data set reading unit configured to read, in response to receiving any one of the identifiers from the predetermined device, information on the sensor data set corresponding to the identifier from the sensor data set storage unit; a sensor data selection unit that selects, for each of the plurality of sensor types, sensor data from a number of sensors designated by the information of the sensor data set read by the sensor data set reading unit, in order of decreasing value, based on information of the sensor data set read by the sensor data set reading unit and the value set by the sensor type sensor data value setting unit for the sensor data from each sensor; The vehicle interior / exterior linking device according to claim 6 , further comprising: a network setting unit that sets the vehicle interior network so that the sensor data selected by the sensor data selection unit is transferred with priority over other sensor data.

8. The vehicle further comprises an in-vehicle network for data transfer; The vehicle interior / exterior linking device according to claim 1 , wherein the data receiving unit includes a network receiving unit that receives the sensor data from the plurality of sensors via the in-vehicle network.

9. The vehicle further includes a vehicle control device that controls the vehicle using the sensor data from the plurality of sensors. The vehicle interior / exterior linking device according to claim 1 , wherein the vehicle control device includes an autonomous driving ECU that performs autonomous driving of the vehicle using the sensor data received from the plurality of sensors.

10. The vehicle further comprises: a remote control ECU that controls the vehicle in accordance with a predetermined remote control command; The vehicle interior / exterior linking device includes: Transmitting the sensor data received from the plurality of sensors to an external remote control device via the wireless communication device; The vehicle interior / exterior linking device according to claim 1 , further comprising: a remote control command received from the remote control device via the wireless communication device, the remote control ECU being configured to input the remote control command to the remote control ECU.

11. A vehicle having a wireless communication device that wirelessly communicates data with the outside of the vehicle and a plurality of sensors, the vehicle having the wireless communication device and a method for linking the inside and outside of the vehicle by transmitting sensor data from the plurality of sensors to an external predetermined device, A computer receives sensor data from a plurality of sensors mounted on a vehicle; A step of estimating a transmission delay time of the sensor data from the sensor to the predetermined device that can communicate with the wireless communication device by a computer; a step of a computer selecting a portion of the sensor data based on the transmission delay time estimated in the step of estimating the transmission delay time, and transmitting the selected sensor data to the predetermined device via the wireless communication device; and a step of determining a value of sensor data output by the plurality of sensors based on a state of the vehicle and states of the plurality of sensors by the computer; the transmitting step includes a data selection step in which a computer selects a portion of the sensor data based on the transmission delay time estimated in the estimating step and the value determined in the determining step, and transmits the selected sensor data to the predetermined device via the wireless communication device; The step of determining the value comprises: A step of estimating a detection range of an infrastructure sensor present in the vicinity of the vehicle based on a position of the vehicle by a computer; a step of a computer estimating the detection ranges of the multiple sensors based on the position and attitude of the vehicle, and setting a value of the sensor data from each sensor so that the value of the sensor data from each sensor is greater for a sensor with a smaller area overlapping with the detection range of the infrastructure sensor estimated by the infrastructure detection range estimation unit.

Images