Driver assistance system, driver assistance method, and in-vehicle device

Abstract

This system provides a driver assistance system that can determine the location of moving objects in the vehicle's blind spots. [Solution] The driving support system 150 includes: a mobile body position acquisition unit 102 that acquires first position information of a mobile body from a communication terminal 130 that measures the position of a mobile body; a matching information acquisition unit 103 that acquires matching information which associates the first position information with second position information of the mobile body measured by an external sensor mounted on the vehicle or another vehicle; a correction parameter acquisition unit 106 that acquires correction parameters generated based on the matching information for correcting the first position information; a mobile body position prediction unit 107 that predicts the position of the mobile body based on the first position information and the correction parameters; and a driving support unit 116 that provides driving support for the vehicle based on the predicted position of the mobile body.

Description

Technical Field

[0001] The present invention relates to a driving support system, a driving support method, and an in-vehicle device.

Background Art

[0002] In recent years, the spread of driving support systems that assist in vehicle driving has been progressing worldwide. However, accidents still occur frequently on local roads and narrow streets, and in particular, it is important to prevent accidents involving vulnerable road users such as pedestrians and wheelchair users who are in the blind spots of vehicles. However, in-vehicle external sensors have sometimes had difficulty detecting not only vulnerable road users in blind spots but also two-wheeled vehicles and vehicles (hereinafter collectively referred to as "moving objects"). In addition, it is unrealistic to install infrastructure sensors such as surveillance cameras on all roads because of the high cost.

[0003] In response to such a situation, Patent Document 1 discloses a map display system having a plurality of terminals that are used in a state of being mounted on a vehicle or carried by a user, and a server that generates map image data for displaying a map image representing the current location of the terminal and the state of use of the terminal.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Incidentally, in the technology described in Patent Document 1, the accuracy of the terminal's position depends on the accuracy of the GPS, so the terminal's position has an error of about 10 to 20 meters. For this reason, even if one tries to use the technology described in Patent Document 1 for driver assistance or autonomous driving, it is unclear whether a moving object is actually present in the blind spot. In this case, if there is no moving object at the location estimated by the server, there is a risk that the vehicle will come into contact with the moving object.

[0006] This invention has been made in view of the above circumstances, and aims to enable an in-vehicle device mounted on a vehicle to determine the position of a moving object located in the vehicle's blind spot. [Means for solving the problem]

[0007] The driver assistance system according to the present invention comprises an in-vehicle device mounted in a vehicle and a server that communicates with the in-vehicle device via a network, and assists in the operation of the vehicle. This driver assistance system includes: a mobile body position acquisition unit that acquires first position information of a mobile body from a communication terminal that measures the position of a mobile body; a matching information acquisition unit that acquires matching information by associating the first position information with second position information of the mobile body measured by an external sensor mounted on the vehicle or another vehicle; a correction parameter acquisition unit that acquires correction parameters generated based on the matching information for correcting the first position information; a mobile body position prediction unit that predicts the position of the mobile body based on the first position information and the correction parameters; and a driver assistance unit that provides driving assistance for the vehicle based on the predicted position of the mobile body. [Effects of the Invention]

[0008] According to the present invention, an in-vehicle device mounted on a vehicle can determine the position of a moving object located in the vehicle's blind spot. Other issues, configurations, and effects not mentioned above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0009] [Figure 1]This is a block diagram showing an example configuration of a driver assistance system according to the first embodiment of the present invention. [Figure 2] This figure shows an example of the hardware configuration of an in-vehicle device processing unit, which is an executable entity of an in-vehicle device according to the first embodiment of the present invention. [Figure 3] This figure shows an example of the hardware configuration of a server processing unit, which is an executable entity of a server according to the first embodiment of the present invention. [Figure 4] This figure shows an example of the hardware configuration of a communication terminal processing device, which is an executable entity of a communication terminal according to the first embodiment of the present invention. [Figure 5] This is a flowchart illustrating an example of operation of an in-vehicle device according to the first embodiment of the present invention. [Figure 6] This is a flowchart illustrating an example of the operation of a server according to the first embodiment of the present invention. [Figure 7] This figure shows an example of a section according to the first embodiment of the present invention. [Figure 8] This figure shows an example of a polynomial correction coefficient calculation and a correction table configuration according to the first embodiment of the present invention. [Figure 9] This is a block diagram showing an example configuration of a driver assistance system according to a second embodiment of the present invention. [Modes for carrying out the invention]

[0010] Hereinafter, embodiments for carrying out the present invention will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same function or configuration are denoted by the same reference numerals, and redundant descriptions are omitted. The present invention is applicable, for example, to a vehicle control computing device that can communicate with an in-vehicle ECU (Electronic Control Unit) for an Advanced Driver Assistance System (ADAS) or Autonomous Driving (AD).

[0011] [First Embodiment] Hereinafter, with reference to Figures 1 to 8, an example of the configuration and operation of the driver assistance system according to the first embodiment of the present invention will be described. Figure 1 is a block diagram showing an example configuration of the driver assistance system 150 according to the first embodiment.

[0012] The driver assistance system (driver assistance system 150) comprises an in-vehicle unit (in-vehicle unit 110) mounted in the vehicle, a server (server 100) that communicates with the in-vehicle unit (in-vehicle unit 110) via a network (wireless network 10), and a communication terminal (communication terminal 130).

[0013] <Example Server Configuration> Server 100 comprises a communication unit 101, a mobile object position acquisition unit 102, a matching information acquisition unit 103, a section determination unit 104, a correction parameter calculation unit 105, a correction parameter acquisition unit 106, and a mobile object position prediction unit 107. In the following description, the holder of the communication terminal 130 is also referred to as the "mobile object." When the holder of the communication terminal 130 is riding a motorcycle or vehicle, the position of the motorcycle or vehicle is the position of the mobile object. In addition, other vehicles equipped with an in-vehicle device capable of communicating with Server 100 may also be included as mobile objects.

[0014] The communication unit 101 communicates various information with the in-vehicle unit 110 and the communication terminal 130 via the wireless network 10, etc. For example, the communication unit 101 receives the first position information (mobile object position information) of the mobile object measured by the communication terminal 130, which measures the position of the mobile object. The communication unit 101 also transmits the mobile object position information predicted by the mobile object position prediction unit 107 to the in-vehicle unit 110. For the sake of drawing convenience, Figure 1 shows one in-vehicle unit 110 and one communication terminal 130, but multiple in-vehicle units 110 and communication terminals 130 are targets for communication by the communication unit 101.

[0015] The mobile body position acquisition unit (mobile body position acquisition unit 102) acquires the first position information (mobile body position information) of the mobile body from a communication terminal (communication terminal 130) that measures the position of the mobile body via the communication unit 101. The mobile body position information includes the position of the pedestrian holding the communication terminal 130 measured by the communication terminal 130, or the position of the holder of the communication terminal 130 who is riding on a two-wheeler or a vehicle. The mobile body position information includes information on latitude and longitude acquired by GPS (Global Positioning System). However, the mobile body position information includes a position error of about several meters to 20 meters, for example.

[0016] The collation information acquisition unit (collation information acquisition unit 103) acquires collation information in which the first position information is associated with the second position information of the mobile body measured by an external sensor (external sensor 112) mounted on the vehicle or another vehicle from the in-vehicle unit 110. For this reason, the collation information acquisition unit 103 acquires the collation information measured by the in-vehicle unit 110 via the communication unit 101.

[0017] The collation information is composed of the uncorrected position of the mobile body obtained from the first position information, the position of the mobile body obtained from the second position information measured by the external sensor 112, the type of the mobile body, and the speed of the mobile body. For this reason, the mobile body position acquisition unit 115 of the in-vehicle unit 110 directly acquires the mobile body position information, which is the first position information, from the communication terminal 130 and outputs the mobile body position information to the mobile body collation unit 114. For example, the collation information associates information such as pedestrians, two-wheelers, and vehicles observed by the in-vehicle unit 110 with the position information of the same mobile body acquired by the in-vehicle unit 110 from the communication terminal 130. For example, pedestrian A observed by the in-vehicle unit 110 is associated with the position information of pedestrian A acquired by the in-vehicle unit 110. The association of the mobile body position information will be described later in the description of the in-vehicle unit 110.

[0018] Furthermore, if a moving object is in a blind spot from the driver's perspective (for example, in a position obstructed by the vehicle's pillar), the external sensor 112 can measure the moving object. However, if a moving object is in a position outside the detection range of the external sensor 112 (for example, in a position obstructed by a wall at a corner), the external sensor 112 cannot measure the moving object. In such cases, if a moving object is in a position outside the detection range of the external sensor 112, the moving object detection unit 113 cannot detect the moving object from the detection result of the external sensor 112. In this case, the matching information will not include the position of the moving object obtained from the second position information.

[0019] The section determination unit (section determination unit 104) determines a predetermined section set on the map that includes the position of the moving object indicated by the first position information. The predetermined section is an area divided into equal intervals by latitude and longitude on the map shown in Figure 7, which will be described later. Note that the predetermined section may be an area divided into arbitrary lengths, such as every 10m or every 100m, rather than by latitude and longitude.

[0020] The learning unit (correction parameter calculation unit 105) updates the correction parameters for each predetermined section set on the map by learning based on the matching information, and generates a different trained model for each predetermined section. The correction parameters are inference parameters of the trained model that output a correction value for the first position information from the difference between the first position information and the second position information. These correction parameters are calculated to improve the accuracy of the mobile position acquired by the mobile position acquisition unit 102. For this reason, the correction parameters are weights in each layer, which are learned in a deep learning model, for example, using the matching information acquired by the matching information acquisition unit 103.

[0021] The mobile object position obtained by the mobile object position acquisition unit 102 includes errors due to multipath interference from surrounding structures. Therefore, the error characteristics change depending on the region in which the mobile object position acquisition unit 102 acquires the mobile object position. Error characteristics refer to the characteristics in which the distance and direction of the mobile object position information to be corrected change. For example, in a building cluster area, the error of the position information obtainable by GPS is approximately 20m, while in an agricultural area (paddy field), the error of the position information may decrease to approximately 10m. To address these regional and weather-dependent changes in error characteristics, deep learning models are constructed for each region, and a deep learning model suited to the region in which the vehicle travels is selected. In the case of a deep learning model, the weights in each layer become the correction parameters calculated by the correction parameter calculation unit 105. The correction parameters calculated by the correction parameter calculation unit 105 are used by the mobile object position prediction unit 107.

[0022] The correction parameter acquisition unit (correction parameter acquisition unit 106) is generated based on the matching information and acquires correction parameters for correcting the first position information. Here, the correction parameter acquisition unit 106 acquires correction parameters from the correction parameter calculation unit 105. The correction parameter acquisition unit 106 outputs the acquired correction parameters to the moving object position prediction unit 107.

[0023] The mobile object position prediction unit (mobile object position prediction unit 107) predicts the position of the mobile object (mobile object position) obtained by the mobile object position acquisition unit 102 based on the first position information of the mobile object and the correction parameters calculated by the correction parameter calculation unit (correction parameter calculation unit 105). When predicting the mobile object position, the mobile object position prediction unit 107 corrects the error in the first position information of the mobile object. In predicting the position of the mobile object, the inference parameters of the trained model, which are the correction parameters calculated by the correction parameter calculation unit 105, are used. Therefore, the mobile object position prediction unit (mobile object position prediction unit 107) can predict the position of the mobile object based on different inference parameters of the trained model for each predetermined interval shown in Figure 7, which will be described later. If the trained model generated by the correction parameter calculation unit 105 is, for example, constructed as a deep learning model, the mobile object position prediction unit 107 can predict the corrected, high-precision mobile object position by inputting the mobile object position before correction into the deep learning model. The mobile object position prediction unit 107 outputs the highly accurate mobile object position to the communication unit 101. The highly accurate mobile object position is used in the in-vehicle device 110 for driver assistance.

[0024] Thus, the server (server 100) includes at least a correction parameter acquisition unit (correction parameter acquisition unit 106) and a mobile object position prediction unit (mobile object position prediction unit 107), and transmits the position of the mobile object predicted by the mobile object position prediction unit (mobile object position prediction unit 107) to the in-vehicle unit (in-vehicle unit 110).

[0025] <Example of in-vehicle system configuration> Returning to the explanation of Figure 1. The in-vehicle unit 110 comprises a communication unit 111, an external sensor 112, a moving object detection unit 113, a moving object matching unit 114, a moving object position acquisition unit 115, a driving support unit 116, an output unit 117, and a control unit 118.

[0026] The communication unit 111 communicates various information with the server 100, the communication terminal 130, or an in-vehicle unit 110 installed in another vehicle via the wireless network 10, etc. For the sake of drawing convenience, only one in-vehicle unit 110 and one communication terminal 130 are shown in the diagram, but multiple in-vehicle units 110 and communication terminals 130 are the targets of communication.

[0027] The external sensor 112, the mobile object detection unit 113, and the mobile object matching unit 114 perform a process to send matching information to the server 100, which includes the mobile object position information of the communication terminal 130 detected by the external sensor 112 and the mobile object position information received from the communication terminal 130, in order to correct the location information of the communication terminal 130 to the server 100. Here, if the mobile object matching unit 114 determines that the mobile object position information obtained from the server 100 by the mobile object position acquisition unit 115 and the mobile object position information detected by the mobile object detection unit 113 are the same, the communication unit 111 sends matching information including the mobile object position information to the server 100.

[0028] The external sensor 112 is at least one of the following: a camera, radar, and LiDAR (Light Detection and Ranging) mounted on the vehicle. The moving object detection unit 113 acquires data from the external sensor 112 in order to detect moving objects that are traffic participants such as pedestrians, motorcycles, and vehicles. If the external sensor 112 is a camera, the external sensor 112 acquires image data. If the external sensor 112 is a radar, the external sensor 112 acquires a two-dimensional distribution image of radio waves reflected by the target object. If the external sensor 112 is a LiDAR, the external sensor 112 acquires point cloud data including position information of the target object. The external sensor 112 outputs each acquired data to the moving object detection unit 113.

[0029] The moving object detection unit (moving object detection unit 113) and the in-vehicle unit (in-vehicle unit 110) detect a second position information of a moving object based on information of the moving object measured by the external sensor (external sensor 112). For example, the moving object detection unit 113 uses data acquired by at least one of the cameras, radar, and LiDAR to detect a moving object and recognize the type of that moving object.

[0030] In the process by which the moving object detection unit 113 detects a moving object and recognizes the type of moving object, for example, if it detects an image of the moving object, it can be carried out by utilizing open-source software (OSS) for recognition, such as YOLO, which is one of the object detection models. The moving object detection unit 113 also calculates the position of the moving object using radar or LiDAR. The moving object detection unit 113 also calculates the speed of the moving object using radar or LiDAR. The moving object detection unit 113 detects the type of traffic participant, such as whether it is a pedestrian or a vehicle, and its position, and outputs this to the subsequent moving object matching unit 114.

[0031] The mobile object matching unit (mobile object matching unit 114) compares the position of the mobile object obtained from the communication terminal 130 by the mobile object position acquisition unit (mobile object position acquisition unit 115) with the position of the mobile object based on second position information, and outputs matching information. For example, the mobile object matching unit 114 performs a comparison between the mobile object information detected by the mobile object detection unit 113 and the mobile object position information obtained from the communication terminal 130 by the mobile object position acquisition unit 115. The mobile object position information obtained by the mobile object position acquisition unit 115 can be either uncorrected and equivalent to GPS accuracy, or corrected and highly accurate. The uncorrected mobile object position information equivalent to GPS accuracy is the mobile object position information obtained by the mobile object position acquisition unit 115 from the communication terminal 130. The corrected and highly accurate mobile object position information is the mobile object position information predicted by the mobile object position prediction unit 107 and obtained by the mobile object position acquisition unit 115 from the server 100. The mobile object matching unit 114 performs the matching only if the obtained mobile object position information is uncorrected and equivalent to GPS accuracy.

[0032] In the matching process performed by the mobile object matching unit 114, for example, a process is performed to determine whether the pedestrian observed by the mobile object position acquisition unit 115 and the pedestrian acquired by the mobile object detection unit 113 represent the same pedestrian. If the mobile object matching unit 114 determines that the pedestrians are the same, it links the position of the mobile object acquired by the mobile object position acquisition unit 115 with various information about the mobile object acquired by the mobile object detection unit 113, namely the type of mobile object and its speed. Details of the determination of whether the pedestrians are the same and the linking process performed by the mobile object matching unit 114 will be explained in Figure 5 below. The linked information is output to the communication unit 111 as matching information.

[0033] The mobile object position acquisition unit (mobile object position acquisition unit 115) outputs the uncorrected position of the mobile object, obtained from the first position information, to the mobile object matching unit (mobile object matching unit 114), and outputs the position of the mobile object predicted by the mobile object position prediction unit (mobile object position prediction unit 107) to the driving support unit (driving support unit 116). For this reason, the mobile object position acquisition unit 115 acquires mobile object position information from the server 100, other in-vehicle devices 110, or communication terminal 130 via the communication unit 111. As described above, the mobile object position information obtained by the mobile object position acquisition unit 115 can be either uncorrected and equivalent to GPS accuracy, or corrected and highly accurate. If it is uncorrected and equivalent to GPS accuracy, the mobile object position acquisition unit 115 outputs the mobile object position information to the mobile object matching unit 114. On the other hand, if it is corrected, the mobile object position acquisition unit 115 outputs the mobile object position information to the driving support unit 116.

[0034] The driver assistance unit (driver assistance unit 116) provides driving assistance to the vehicle based on the position of the moving object predicted by the moving object position prediction unit (moving object position prediction unit 107). For this reason, the driver assistance unit 116 uses the corrected high-precision position information obtained by the moving object position acquisition unit 115 to provide driving assistance to the vehicle on which the in-vehicle device 110 is installed. Driving assistance may include, for example, a warning to alert the driver, such as outputting to the output unit 117 (described later) about the position of a person in a blind spot hidden behind a parked vehicle to alert the driver. Alternatively, the driving assistance by the driver assistance unit 116 may display the positions of moving objects in the surrounding area on a map. Alternatively, the driving assistance by the driver assistance unit 116 may prompt the driver to pay attention with a warning sound or voice via the output unit 117. Alternatively, the driving assistance by the driver assistance unit 116 may use the control unit 118 (described later) to automatically decelerate the vehicle speed when the vehicle speed is so high that there is a possibility of collision even if the brakes are applied.

[0035] Furthermore, as will be described later, the communication terminal 130 can acquire attribute information such as the gender, age, and whether or not the holder of the communication terminal 130 uses a wheelchair. Children are more likely to take dangerous actions such as suddenly running off compared to adults. For this reason, the driving support unit (driving support unit 116) provides driving support for the vehicle based on the attribute information of the mobile body holding the communication terminal (communication terminal 130) obtained from the communication terminal (communication terminal 130). The output unit 117 and the control unit 118 can provide warnings and control that utilize attribute information, such as applying a larger deceleration to the vehicle, by providing driving support that takes into account the attribute information of the holder of the communication terminal 130. Alternatively, the driving support unit 116 may use the attribute information of the holder of the communication terminal 130 to provide driving support for speed arbitration during autonomous driving, or for route calculation and selection.

[0036] The output unit 117, upon receiving a command from the driver assistance unit 116, displays a person in the vehicle's blind spot, displays the location of moving objects around the vehicle, or emits an audio message.

[0037] The control unit 118 receives commands from the driving support unit 116 and performs control over the acceleration, deceleration, and steering of the vehicle.

[0038] <Example of a communication terminal configuration> The communication terminal 130 is, for example, a terminal device such as a smartphone, and comprises an attribute information acquisition unit 131, a mobile object position measurement unit 132, and a communication unit 133. The holder of the communication terminal 130 is, for example, a person present around (including blind spots) a vehicle on which the in-vehicle unit 110 is installed.

[0039] The attribute information acquisition unit 131 acquires attribute information of the holder of the communication terminal 130. The attribute information includes the holder's age and gender, and may also include the holder's preferences and usual behavior patterns. The attribute information is acquired from the holder of the communication terminal 130 with their consent, for example, through a smartphone application program, and the age, gender, preferences, etc., are information that the person has entered into the application program. Furthermore, the data on the user's usual behavior is obtained with their consent, and by accumulating and statistically processing GPS location information over time, statistical information on how the user usually behaves is represented. The attribute information obtained by the attribute information acquisition unit 131 is output to the communication unit 133 via the mobile body position measurement unit 132.

[0040] The mobile object position measurement unit 132 measures the location information of the holder of the communication terminal 130. The mobile object position information measured by the mobile object position measurement unit 132 is, for example, latitude and longitude information. The latitude and longitude information measured by the mobile object position measurement unit 132 is equivalent to GPS accuracy and includes, for example, an error of a few meters to 20 meters. The location information measured by the mobile object position measurement unit 132 is output to the communication unit 133.

[0041] The communication unit 133 transmits the attribute information of the holder of the communication terminal 130, acquired by the attribute information acquisition unit 131, and the location information of the communication terminal holder, acquired by the mobile object position measurement unit 132, via the communication network.

[0042] Furthermore, the communication terminal 130 may be a device configured to transmit only its own location information, such as one that a parent might give to their child. Such a communication terminal 130 does not have a display unit. Even with this communication terminal 130, it is possible to transmit attribute information indicating that the holder of the communication terminal 130 is a child, along with the location information.

[0043] Figure 2 shows an example of the hardware configuration of the in-vehicle processing unit 200, which is the actual implementation of the in-vehicle unit 110. The in-vehicle processing unit 200 comprises a CPU 201, ROM 202, RAM 203, external sensor 210, internal sensor 220, GPS receiver 231, communication device 232, output device 233, and vehicle control device 240. Each is connected by signal lines and exchanges various types of data.

[0044] The CPU 201 operates as the execution unit of the in-vehicle processing unit 200 by reading and executing various programs and parameters from the ROM 202. RAM203 is a read / write memory area and operates as the main memory of the in-vehicle processing unit 200.

[0045] ROM202 is a read-only memory area where the program described later is stored. This program is loaded in RAM203 and executed by CPU201. The CPU201 reads and executes the various programs loaded in RAM203 to realize the functions of the mobile object detection program 251, mobile object matching program 252, driving support program 254, and mobile object position acquisition program 253.

[0046] The moving object detection program 251 describes the actions to be performed by the moving object detection unit 113 in the in-vehicle unit 110 shown in Figure 1. The moving object matching program 252 describes the actions to be performed by the moving object matching unit 114 in the in-vehicle unit 110 shown in Figure 1. The moving object position acquisition program 253 describes the actions to be performed by the moving object position acquisition unit 115 in the in-vehicle unit 110 shown in Figure 1. The driving support program 254 describes the actions to be performed by the driving support unit 116 in the in-vehicle unit 110 shown in Figure 1.

[0047] The external sensor 210 includes a camera 211, a radar 212, and a LiDAR 213. For the sake of the diagram, only one of each is shown (camera 211, radar 212, and LiDAR 213), but multiple units may be provided.

[0048] Camera 211 is mounted around the vehicle and takes pictures of the area around the vehicle. The positional relationship between camera 211 and the vehicle is not shown in the diagram, but it is stored in ROM 202 as sensor parameters.

[0049] Radar 212 is mounted around the vehicle and observes the area around the vehicle. The positional relationship between radar 212 and the vehicle is not shown in the diagram, but it is stored in ROM 202 as sensor parameters.

[0050] The LiDAR213 is mounted on the vehicle and observes the area around the vehicle. The positional relationship between the LiDAR213 and the vehicle is not shown in the diagram, but it is stored in the ROM202 as sensor parameters.

[0051] The internal sensor 220 includes a vehicle speed sensor 221 and a steering angle sensor 222. Although not shown in the diagram, it may also include an encoder, gyro sensor, acceleration sensor, etc.

[0052] The vehicle speed sensor 221 and the steering angle sensor 222 measure the vehicle speed and steering angle of the vehicle on which the onboard processing unit 200 is installed, respectively, and output them to the CPU 201. The onboard processing unit 200 uses the outputs of the vehicle speed sensor 221 and the steering angle sensor 222 to calculate the amount and direction of movement of the vehicle on which the onboard processing unit 200 is installed, using known techniques such as dead reckoning.

[0053] The GPS receiver 231 receives signals from multiple satellites that make up the satellite navigation system, and calculates its position, i.e., latitude and longitude, based on the received signals. The accuracy of the latitude and longitude calculated by the GPS receiver 231 does not need to be high; for example, it may contain an error of a few meters to 20 meters. The GPS receiver 231 outputs the calculated latitude and longitude to the CPU 201.

[0054] The communication device 232 is used to exchange information between external equipment of the vehicle and the in-vehicle processing unit 200 via the wireless network 10, and is the actual entity of the communication unit 111 in the in-vehicle unit 110 shown in Figure 1.

[0055] The output device 233 is, for example, a liquid crystal display device such as a navigation monitor, or a head-up display device that projects images onto the vehicle's windshield. Various information output from the in-vehicle processing unit 200 is displayed on the output device 233. Alternatively, the output device 233 may be, for example, an in-vehicle speaker. Warning sounds, warning messages, etc., are emitted from the output device 233.

[0056] The vehicle control device 240 comprises a steering device 241, a drive device 242, and a braking device 243. The vehicle control device 240 controls the steering device 241, the drive device 242, and the braking device 243 based on information output from the on-board device processing device 200.

[0057] The steering device 241 controls the vehicle's steering. The drive unit 242 provides driving force to the vehicle. The drive unit 242 increases the driving force of the vehicle, for example, by increasing the target rotational speed of the engine equipped in the vehicle. The braking device 243 applies braking force to the vehicle.

[0058] Figure 3 shows an example of the hardware configuration of the server processing unit 300, which is the actual execution entity of server 100. The server processing unit 300 is comprised of a CPU 301, ROM 302, RAM 303, communication device 304, and storage unit 305. Each is connected by signal lines to exchange various types of data.

[0059] The CPU 301 operates as the execution unit of the server processing unit 300 by reading various programs and parameters from the ROM 302 and executing them. RAM303 is a read / write memory area and operates as the main memory of the server processing unit 300.

[0060] ROM302 is a read-only memory area where the program described later is stored. This program is loaded in RAM303 and executed by CPU301. By reading and executing the various programs loaded in RAM303, CPU301 operates as the mobile object position acquisition program 311, the matching information acquisition program 312, the interval determination program 313, the correction parameter calculation program 314, the correction parameter acquisition program 315, and the mobile object position prediction program 316.

[0061] The mobile object position acquisition program 311 describes the actions to be performed by the mobile object position acquisition unit 102 in the server 100 shown in Figure 1. The matching information acquisition program 312 describes the actions to be performed by the matching information acquisition unit 103 in the server 100 shown in Figure 1. The interval determination program 313 describes the actions to be performed by the interval determination unit 104 in the server 100 shown in Figure 1.

[0062] The correction parameter calculation program 314 describes the actions to be performed by the correction parameter calculation unit 105 in the server 100 shown in Figure 1. The correction parameter acquisition program 315 describes the actions to be performed by the correction parameter acquisition unit 106 in the server 100 shown in Figure 1. The mobile object position prediction program 316 describes the actions to be performed by the mobile object position prediction unit 107 in the server 100 shown in Figure 1.

[0063] The communication device 304 is used to exchange information between the server and external equipment via the wireless network 10, and is the actual entity of the communication unit 101 in the server 100 shown in Figure 1. The memory unit 305 is a non-volatile memory device and operates as an auxiliary memory device for the server processing unit 300. Correction parameters are stored in the memory unit 305.

[0064] Figure 4 shows an example of the hardware configuration of the communication terminal processing unit 400, which is the actual implementation of the communication terminal 130. The communication terminal processing unit 400 is comprised of a CPU 401, ROM 402, RAM 403, GPS receiver 404, communication device 405, and internal sensor 410. Each is connected by signal lines to exchange various types of data.

[0065] The CPU 401 operates as the execution unit of the communication terminal processing unit 400 by reading and executing various programs and parameters from the ROM 402. The RAM 403 is a read / write memory area and operates as the main memory of the communication terminal processing unit 400.

[0066] ROM 402 is a read-only memory area, and the program described later is stored in ROM 402. This program is loaded in RAM 403 and executed by CPU 401. By reading and executing the various programs loaded in RAM 403, CPU 401 operates as the attribute information acquisition program 421 and the mobile object position measurement program 422.

[0067] The attribute information acquisition program 421 is a program that describes the actions to be performed by the attribute information acquisition unit 131 in the communication terminal 130 shown in Figure 1. The mobile object position measurement program 422 is a program that describes the actions to be performed by the mobile object position measurement unit 132 in the communication terminal 130 shown in Figure 1.

[0068] The GPS receiver 404 receives signals from multiple satellites that constitute the satellite navigation system and calculates its position, i.e., latitude and longitude, based on calculations using the received signals. The accuracy of the latitude and longitude calculated by the GPS receiver 404 does not need to be high; for example, it may contain an error of a few meters to 20 meters. The GPS receiver 404 outputs the calculated latitude and longitude to the communication terminal processing unit 400.

[0069] The communication device 405 is used to exchange information between the communication terminal and the communication terminal processing device 400 via the wireless network 10, and is the actual entity of the communication unit 133 in the communication terminal 130 in Figure 1.

[0070] The internal sensor 410 includes a gyro sensor 411 and an acceleration sensor 412. The gyro sensor 411 and the acceleration sensor 412 measure the change in attitude angle and acceleration of the communication terminal on which the communication terminal processing unit 400 is mounted, respectively, and output these values ​​to the communication terminal processing unit 400.

[0071] Next, examples of processes performed in each part of the driver assistance system 150 will be explained with reference to Figures 5 to 7. The series of processes described below will perform the driver assistance method according to the first embodiment.

[0072] <Processing by in-vehicle devices> Figure 5 is a flowchart illustrating an example of the operation of the in-vehicle unit 110. The in-vehicle unit 110 performs the following operations each time it receives information from the communication unit 111. The execution entity described below is the CPU 201. The flowchart shown in Figure 5 consists of: moving object detection (S1), moving object position acquisition (S2), position correction completion determination (S3), driving assistance (S4), moving object verification (S5), and verification information transmission (S6).

[0073] In step S1, the moving object detection unit 113 detects and recognizes a moving object using at least one of the camera 211, radar 212, and LiDAR 213. The moving object detection unit 113 also calculates the position of the moving object using radar 212 or LiDAR 213. The moving object detection unit 113 also calculates the speed of the moving object using radar 212 or LiDAR 213.

[0074] The detection of moving objects in step S1 is performed by the external sensor 112 in the in-vehicle unit 110 and the moving object detection program 251, which is the execution program of the moving object detection unit 113. Here, the moving object detection program 251 detects the type of traffic participant, such as whether it is a pedestrian or a vehicle, and its position, and outputs the information to the moving object matching in step S5. Even if no moving object is detected, that information is also output to the moving object matching in step S5.

[0075] In step S2, the mobile object position acquisition unit 115 acquires mobile object position information from the server 100, other in-vehicle devices 110, and communication terminals 130 via the communication unit 111. The mobile object position acquisition in step S2 is performed by the mobile object position acquisition program 253, which is the execution program of the mobile object position acquisition unit 115 in the in-vehicle device 110. The mobile object position acquisition program 253 outputs the obtained mobile object position information to step S3. Note that the mobile object detection in step S1 and the mobile object position acquisition in step S2 are both performed independently.

[0076] In step S3, the position correction determination is performed by the mobile body position acquisition unit 115, which performs the determination of whether to output mobile body position information. The position correction determination in step S3 is performed by the mobile body position acquisition program 253, which is the execution program of the mobile body position acquisition unit 115 in the in-vehicle device 110. As described above, the mobile body position information obtained in step S2 may be uncorrected and equivalent to GPS accuracy, or corrected and highly accurate. If the mobile body position information is uncorrected and equivalent to GPS accuracy (NO in S3), the mobile body position acquisition unit 115 outputs the mobile body position information to the mobile body matching process in step S5. If the mobile body position information is corrected (YES in S3), the mobile body position acquisition unit 115 outputs the mobile body position information to the driving support process in step S4.

[0077] In step S4, the driver assistance unit 116 uses the corrected high-precision position information obtained in step S2 to perform driver assistance for the vehicle. The driver assistance processing in step S4 is performed by the driver assistance program 254, which is the execution program of the driver assistance unit 116, output unit 117, and control unit 118 in the in-vehicle unit 110. Driver assistance may include, for example, warnings about the location of a person hidden in a blind spot behind a parked vehicle, or displaying the locations of surrounding moving objects on a map. Alternatively, driver assistance may include automatic deceleration of the vehicle speed, speed arbitration during autonomous driving, and route calculation and selection.

[0078] In step S5, the mobile object matching unit 114 compares the uncorrected position information obtained in step S2 with the mobile object information detected in step S1. The mobile object matching in step S5 is performed by the mobile object matching program 252, which is the execution program of the mobile object matching unit 114 in the in-vehicle unit 110. For mobile object matching, the mobile object matching unit 114 obtains latitude and longitude information indicating the position of the in-vehicle unit 110 itself from the GPS receiver 231, and uses that information to convert the mobile object information obtained in step S1 into latitude and longitude information.

[0079] The mobile object position information is relative position information with respect to the on-board unit 110, acquired by the external sensor 112. Therefore, the mobile object matching unit 114 can convert the mobile object position information into latitude and longitude information by adding it to the latitude and longitude information of the on-board unit 110, while ensuring the units are aligned. The basic idea is to associate the mobile object with the closest latitude and longitude information as the same mobile object.

[0080] However, the uncorrected mobile body information obtained in step S2, as well as the latitude and longitude information obtained from the GPS receiver 231, both contain errors of several meters to 20 meters. For this reason, the mobile body matching unit 114 does not use the latitude and longitude information obtained from the GPS receiver 231 alone, but instead uses it in conjunction with vehicle-centered relative coordinates such as dead reckoning using the output of the internal sensor 220, SfM (Structure from Motion) using the external sensor 112, or SLAM (Simultaneous Localization and Mapping) to construct a coordinate system. By constructing such a coordinate system, the mobile body matching unit 114 can accurately determine not only the relative position between the vehicle and the mobile body, but also the position of the mobile body (absolute position expressed in latitude and longitude).

[0081] For example, the mobile object matching unit 114 performs mobile object matching in step S5, converting the mobile object position information acquired by the external sensor 112 into latitude and longitude information. This improves the accuracy of the mobile object position information. Then, the mobile object matching unit 114 comprehensively determines the linking of the mobile object position based on conditions 1 to 4, for example, as follows.

[0082] Condition 1 is that, for a predetermined period of time, the relative distance to a specific moving object remains approximately constant over time. The specific moving object is, for example, a moving object detected using the external sensor 112. Condition 2 is that the speeds of the moving objects linked by the matching information are all approximately the same. Condition 3 is that the speed of the moving object is consistent with the attributes of the object. Consistency means, for example, that a car should be around 40 km / h, and a pedestrian around 4 km / h. If a pedestrian is measured to be around 40 km / h, it does not conform. Condition 4 is that a moving object exists within a certain range from the road area for a predetermined period of time and in a time-series manner. For example, the moving object may be located in a place that could interfere with the direction in which the vehicle is traveling.

[0083] The mobile object matching unit 114 outputs the uncorrected mobile object position obtained in step S2 and the mobile object information obtained in step S1 (matching information linking the mobile object's position, velocity, and attributes) as matching information transmission in step S6 only if all of the conditions defined above are met. The server 100 can improve the accuracy of predicting the mobile object position by using the matching information that satisfies the above conditions 1 to 4. However, the above conditions 1 to 4 are not limited to these, and other methods may be used as long as the linking by the mobile object matching unit 114 can be performed appropriately.

[0084] In step S6, the verification information transmission, the communication unit 111 transmits the verification information obtained in step S5 to the wireless network 10 using the communication device 232. The verification information transmission in step S6 is performed by the communication device 232, which is the execution device of the communication unit 111 in the in-vehicle unit 110.

[0085] <Server Processing> Figure 6 is a flowchart illustrating an example of server 100's operation. The server 100 performs the following operations each time it receives information from the communication unit 111 of the in-vehicle unit 110. The execution entity described below is the CPU 301. The flowchart shown in Figure 6 consists of acquiring information (S11), acquiring matching information (S12), determining the interval (S13), setting the learning model MA for interval A, setting the learning model MX for interval X, updating the correction parameters (S14), acquiring the mobile body position (S15), determining the interval (S16), acquiring the correction parameters (S17), predicting the mobile body position (S18), and transmitting the mobile body position (S19).

[0086] In step S11, the information acquisition determination is performed by the communication unit 101 of the server 100, which changes the output destination based on the information received from the communication unit 111 of the in-vehicle device 110. The information received by the communication unit 111 may be verification information transmitted from the in-vehicle device 110 or the location of the moving object transmitted from the communication terminal 130. If the communication unit 111 receives verification information, the communication unit 101 transmits the verification information to the verification information acquisition in step S12. If the communication unit 111 receives location information, it transmits the location of the moving object to the location acquisition in step S15.

[0087] In step S12, the matching information acquisition is performed by the matching information output from the acquired information determination in step S11. The matching information acquisition in step S12 is performed by the matching information acquisition program 312, which is the execution program of the matching information acquisition unit 103 on the server 100. Here, the matching information acquisition unit (matching information acquisition unit 103) acquires matching information from the mobile object matching unit (mobile object matching unit 114). The matching information acquisition unit 103 then outputs the obtained matching information to the interval determination in step S13.

[0088] In step S13, the interval determination unit 104 determines the interval corresponding to the matching information acquired by the matching information acquisition unit 103. Based on the first location information and the matching information, the interval determination unit (interval determination unit 104) determines a first predetermined interval that includes the location of the communication terminal, and outputs the determined first predetermined interval and the matching information to the learning unit (correction parameter calculation unit 105). The learning unit (correction parameter calculation unit 105) generates a trained model based on the first predetermined interval and the matching information. Subsequently, the mobile object position prediction unit (mobile object position prediction unit 107) predicts the mobile object position using the trained model.

[0089] The error in the position of a moving object includes errors due to multipath effects from structures surrounding the object, and therefore the error characteristics vary depending on the region being acquired. To address these regional variations in error characteristics, the matching information is divided into segments, and this segment information is used to learn and update correction parameters for each segment. For example, the latitude and longitude are each divided into certain ranges, and the region as a whole is divided into a mesh, with each of these divided meshes defined as a segment.

[0090] Here, an example of an interval will be explained with reference to Figure 7. Figure 7 shows an example of a time zone. In Figure 7, latitudes La1 to La2 and longitudes Lo1 to Lo4 (hereinafter referred to as latitude and longitude) are superimposed on the map. Both the latitude and longitude are equally spaced.

[0091] Each mesh divided by latitude and longitude is shown as section A, B, and C. Assume that the vehicle is traveling on section A. For example, if the vehicle is traveling upwards in Figure 7, the other vehicles (moving objects) and pedestrians (moving objects) shown in Figure 7 are all on roads different from the vehicle's lane. Conventionally, the in-vehicle device cannot recognize these moving objects, but the in-vehicle device 110 according to this embodiment can recognize the moving objects based on the moving object position of the communication terminal 130 held in each moving object.

[0092] However, let's assume that section A includes roads running around a river, and section C includes roads running through an urban area. In this case, section C has poorer visibility and more blind spots compared to section A. Therefore, the matching information used in section A and the matching information used in section C are used to learn and update different correction parameters.

[0093] Returning to the explanation of Figure 6. In step S13, the interval determination unit 104 refers to the latitude and longitude of the mobile object's position based on the mobile object's position information included in the matching information, and determines which interval the latitude and longitude of the mobile object's position fall into. The interval determination in step S13 is performed by the interval determination program 313, which is the execution program of the interval determination unit 104 on the server 100. The interval determination unit 104 outputs the determined interval information together with the matching information to the correction parameter update in step S14.

[0094] In step S14, the correction parameter update, the correction parameter calculation unit 105 learns and updates correction parameters to improve accuracy by correcting the moving object position to a level equivalent to GPS accuracy. The correction parameter update is performed by the correction parameter calculation program 314, which is the execution program of the correction parameter calculation unit 105 on the server 100.

[0095] The correction parameters are learned, for example, by a deep learning model. The correction parameter calculation unit 105 takes as input information the mobile object position with GPS accuracy equivalent to that obtained by the in-vehicle device 110 or communication terminal 130 from the matching information acquisition unit 103 output in step S13, and as output information the mobile object position obtained by observation by the in-vehicle device 110. The correction parameter calculation unit 105 then learns the parameters of the deep learning model so that the input information and the output information match. The correction parameter calculation unit 105 may also construct a model that uses the speed, attributes, etc. of the mobile object included in the matching information.

[0096] Model training by the correction parameter calculation unit 105 is performed by training and updating the model for the relevant interval using the interval information output from the interval determination unit 104 together with the matching information. If the interval information indicates interval A, the correction parameter calculation unit 105 inputs the matching information to the training model MA for interval A and trains the training model MA. If the interval information indicates interval X, the correction parameter calculation unit 105 inputs the matching information to the training model MX for interval X and trains the training model MX. If multiple intervals are recognized as the same region, such as a wide rural area, a training model trained in one interval may be reused in other intervals.

[0097] <Example of processing using a correction table> Furthermore, the correction parameter calculation unit 105 can also calculate correction parameters using a rule-based method. This rule-based method is expected to use the correction table T1. The correction table T1 will be explained with reference to Figure 8. Figure 8 shows the calculation of the polynomial correction coefficient and the values ​​of the correction table T1 generated by the coefficient calculation.

[0098] The correction coefficients of the polynomial are calculated, for example, using the mobile object position, the mobile object position in the matching information, and the interval information. For this reason, a polynomial model relating the mobile object position in interval A, where the interval information is "1", to the mobile object position in the matching information may be defined in advance, and the coefficients of that polynomial may be used as correction parameters. In the example in Figure 8, for example, in interval A, the correction coefficients of the polynomial that best approximates the mobile object position and the mobile object position in the matching information in the predefined polynomial model are k10, k12, k13, ...

[0099] The correction coefficients of the polynomial can be derived by using an optimization method such as the Levenburg-Marquart method to minimize the difference between the mobile object's position and the mobile object's position in the matching information. Here, even if the mobile object's positions are (x1,y1), (x2,y2), (x3,y3) and the matching information's mobile object positions are (xx1,yy1), (xx2,yy2), (xx3,yy3), the correction coefficients of the polynomial will be k10, k12, k13, ... if the interval information is the same (for example, interval A). If the interval information is different, the correction coefficients will change to k20, k22, k23, ...

[0100] The correction table T1 has a structure that includes interval information and correction coefficient items. The correspondence between such interval information and correction coefficients can be changed for each region. Examples of regions include clusters of buildings and agricultural areas (paddy fields). Clusters of buildings are areas where many high-rise buildings are clustered, and the effect of multipath is significant. Agricultural areas (paddy fields) are areas without high-rise buildings, and the effect of multipath can be ignored.

[0101] If a correction table T1 is available, the server 100 may be configured to include the correction table T1, excluding the correction parameter calculation unit 105 from the configuration in Figure 1. The correction parameter acquisition unit 106 can acquire correction parameters from the correction table T1. When the mobile object position acquisition unit 102 of the server 100 acquires mobile object position information acquired by the onboard device 110 of a vehicle traveling in a certain area, the mobile object position prediction unit 107 can correct the mobile object position information using the correction parameters acquired by the correction parameter acquisition unit 106 from the correction table T1.

[0102] Therefore, the matching information acquisition unit (matching information acquisition unit 103) acquires matching information from the mobile object matching unit (mobile object matching unit 114). The matching information acquisition unit 103 then outputs the acquired matching information to the interval determination in step S13. The interval determination unit (interval determination unit 104) determines a first predetermined interval that includes the location of the communication terminal based on the first location information and the matching information. The correction parameter acquisition unit (correction parameter acquisition unit 106) acquires correction coefficients as correction parameters for a polynomial model that relates the uncorrected mobile object position obtained from the first location information, the mobile object position obtained from the second location information, and the first predetermined interval, which are included in the matching information. The mobile object position prediction unit (mobile object position prediction unit 107) predicts the mobile object position by correcting the mobile object position acquired by the mobile object position acquisition unit (mobile object position acquisition unit 102) using the correction parameters.

[0103] Furthermore, in the case of a vehicle moving between regions shown in correction table T1 (for example, a vehicle moving from a building cluster area to an agricultural area (rice paddy)), a sudden change in the amount of correction for the mobile object's position information can lead to misrecognition of the mobile object's position. Therefore, the mobile object position prediction unit 107 may, for example, prorate the distance between the building cluster area and the agricultural area (rice paddy) based on the vehicle's position, and then prorate the correction parameters based on that prorated distance.

[0104] Alternatively, the correction parameter calculation unit 105 may remain on the server 100. In this case, the correction parameter calculation unit 105 stores the correction parameters calculated for each interval information (i.e., region) in the correction table T1, and the correction parameter acquisition unit 106 may acquire the correction parameters from the correction table T1 as appropriate.

[0105] Returning to the explanation of Figure 6. In step S15, the mobile object position acquisition unit 102 acquires the mobile object position output from the acquired information determination in step S11. The mobile object position acquisition in step S15 is performed by the mobile object position acquisition program 311, which is the execution program of the mobile object position acquisition unit 102 on the server 100. The mobile object position acquisition unit 102 outputs the obtained verification information to the interval determination in step S16.

[0106] In step S16, the interval determination unit 104 determines the interval corresponding to the mobile body position acquired by the mobile body position acquisition unit 102. The interval is the same as that set in step S13. In step S13, the interval determination unit 104 refers to the latitude and longitude of the mobile body position contained in the matching information and determines which mesh the latitude and longitude of the mobile body position correspond to. The interval determination in step S16 is performed by the interval determination program 313, which is the execution program of the interval determination unit 104 on the server 100. After the interval determination in step S16, the interval determination unit 104 outputs the determined interval information together with the moving object position to the correction parameter acquisition in step S17.

[0107] In step S17, the correction parameter acquisition unit 106 acquires correction parameters using the deep learning model obtained in step S14 (correction parameter update) and the interval information output from step S16 (interval determination) as the interval determination result. If the interval determination result obtained by the correction parameter acquisition unit 106 is interval A, the learning model MA for interval A is acquired.

[0108] If the interval determination result obtained in step S16 is interval X, the learning model MA for interval X is obtained. The acquisition of correction parameters in step S17 is performed by the correction parameter acquisition program 315, which is the execution program of the correction parameter acquisition unit 106 on the server 100. The correction parameter acquisition unit 106 outputs the obtained learning model and the moving object position together to the moving object position prediction in step S18. For convenience, the above explanation has been given, but in reality, the correction parameter acquisition unit 106 does not need to acquire the learning model; it may simply need to determine the reference destination of the learning model stored in a database or the like.

[0109] In step S18, the mobile object position prediction unit 107 predicts the mobile object position with improved accuracy through correction, using the learning model obtained from the correction parameter acquisition in step S17, or a learning model referenced from a database, etc., and the mobile object position information. By inputting the mobile object position information into the learning model, the mobile object position prediction unit 107 can output the predicted mobile object position as the corrected mobile object position. The mobile object position prediction in step S18 is executed by the mobile object position prediction program 316, which is the execution program of the mobile object position prediction unit 107 on the server 100. The corrected mobile object position output by the mobile object position prediction unit 107 is output in step S19, the mobile object position transmission.

[0110] In step S19, the mobile object position transmission, the communication unit 101 transmits the corrected mobile object position obtained in step S18 to the wireless network 10. The mobile object position transmission in step S19 is performed by the communication device 304, which is the execution device of the communication unit 101 in the server 100. The corrected mobile object position is received by the communication unit 111 of the in-vehicle unit 110 and used for subsequent processing.

[0111] In the driver assistance system 150 according to the first embodiment described above, even if a pedestrian or the like is in a blind spot from the perspective of the vehicle driver, the in-vehicle unit 110 can recognize the presence of the pedestrian or the like in the blind spot by using information obtained from the communication terminal 130 held by the pedestrian or the like. Therefore, the control unit 118 of the in-vehicle unit 110 can safely control the vehicle by slowing down or stopping it.

[0112] Furthermore, attribute information of the holder of the communication terminal 130 is sent to the server 100, and the server 100 predicts the position of the moving object according to the attribute information. The in-vehicle unit 110 controls the vehicle based on the position of the moving object predicted by the server 100 according to the attribute information. As a result, if the holder of the communication terminal 130 is using a wheelchair, the vehicle can be slowed down, or if the holder is a child, the vehicle can be driven with a larger margin from the position of the moving object to account for the possibility of the child suddenly running out into the road. This makes it possible to control the vehicle based on attribute information.

[0113] Furthermore, a database containing correction parameters calculated in advance for each region may be provided on the server 100. In this case, the configuration of the correction parameter calculation unit 105 from the server 100 becomes unnecessary, and the correction parameter acquisition unit 106 can acquire the correction parameters from the database.

[0114] Furthermore, in addition to the association between the in-vehicle unit 110 and the communication terminal 130 located outside the vehicle, an association between the in-vehicle unit 110 and the communication terminal 130 located inside the vehicle may also be performed. The communication terminal 130 located inside the vehicle is, for example, held by the driver of the vehicle on which the in-vehicle unit 110 is installed. In this case, the mobile object position measured by the communication terminal 130 located inside the vehicle is transmitted to the server 100, and the server 100 predicts the mobile object position of the communication terminal 130 located inside the vehicle.

[0115] Server 100 can calculate correction parameters using the moving position of the communication terminal 130 located in the vehicle. Since the in-vehicle unit 110 and the communication terminal 130 located in the vehicle are in the same position, the position of the vehicle on which the in-vehicle unit 110 is mounted can be accurately determined based on the predicted position of the communication terminal 130. For this reason, the in-vehicle unit 110 can also provide driving assistance based on the predicted position of the communication terminal 130 located in the vehicle.

[0116] Furthermore, in the above-described embodiment, the moving object detection unit 113 detects the moving object based on the second position information (moving object position information) of the moving object measured by the external sensor 112 mounted on the vehicle, and the moving object matching unit 114 matches the moving object. However, the external sensor 112 mounted on the vehicle alone may not be able to measure moving objects that are in blind spots. Therefore, the communication unit 111 of the in-vehicle unit 110 may receive second location information (moving object location information) of a moving object measured by an external sensor 112 mounted on another vehicle traveling near its own vehicle. Then, the moving object detection unit 113 of the own vehicle may detect the moving object based on the moving object location information measured by the other vehicle, and the moving object matching unit 114 may match the moving object.

[0117] [Second Embodiment] Next, an example of the configuration of a driver assistance system according to a second embodiment of the present invention will be described. In the driver assistance system according to the second embodiment, the position of the moving object is predicted by an in-vehicle device. Figure 9 is a block diagram showing an example configuration of the driver assistance system 150A according to the second embodiment. The driver assistance system 150A includes a server 100A, an in-vehicle unit 110A, and a communication terminal 130. The configuration of the server 100A and the in-vehicle unit 110A in the driver assistance system 150A differs from the server 100 and in-vehicle unit 110 in the driver assistance system 150 shown in Figure 1.

[0118] In the driver assistance system 150A, a mobile object position prediction unit 120 is provided, which transfers the function of the mobile object position prediction unit 107 shown in Figure 1 to the in-vehicle unit 110A. The configuration has been changed with the aim of performing mobile object position prediction in the in-vehicle unit 110A using the mobile object position prediction unit 120. The configuration of the communication terminal 130 is exactly the same as in Figure 1, so its explanation is omitted.

[0119] Server 100A consists of a communication unit 101, a matching information acquisition unit 103, a section determination unit 104, a correction parameter calculation unit 105, and a correction parameter acquisition unit 106. The functions of the communication unit 101, the matching information acquisition unit 103, the section determination unit 104, the correction parameter calculation unit 105, and the correction parameter acquisition unit 106 are the same as those described in Figure 1, so their explanation is omitted.

[0120] The correction parameters calculated by the correction parameter calculation unit 105 are transmitted to the wireless network 10 via the communication unit 101. Alternatively, as described above, instead of the correction parameter calculation unit 105, a correction table T1 may be provided, and the correction parameter acquisition unit 106 may acquire the correction parameters from the correction table T1.

[0121] The in-vehicle unit 110A comprises a communication unit 111, an external sensor 112, a moving object detection unit 113, a moving object matching unit 114, a moving object position acquisition unit (moving object position acquisition unit 115), a driving support unit (driving support unit 116), an output unit 117, a control unit 118, a correction parameter acquisition unit (correction parameter acquisition unit 119), and a moving object position prediction unit (moving object position prediction unit 120). Compared to the configuration of the in-vehicle unit 110A shown in Figure 1, the correction parameter acquisition unit 119 and the moving object position prediction unit 120 are added.

[0122] The communication unit 111, the external sensor 112, the moving object detection unit 113, the moving object matching unit 114, the output unit 117, the control unit 118, and the moving object position prediction unit 120 are the same as those described in Figure 1, so a detailed explanation is omitted.

[0123] The mobile object position acquisition unit (mobile object position acquisition unit 115) acquires the mobile object position from the communication terminal (communication terminal 130).

[0124] The correction parameter acquisition unit (correction parameter acquisition unit 119) acquires correction parameters generated by the server (server 100) based on the second position information of the moving object measured by the external sensor (external sensor 112). The correction parameter acquisition unit 119 acquires the correction parameters output via the communication unit 101 of the server 100 using the communication unit 111 of the in-vehicle unit 110A. The correction parameters are, for example, parameters of a trained model, such as the trained model MA for interval A and the trained model MX for interval X.

[0125] The mobile object position prediction unit (mobile object position prediction unit 120) predicts the corrected mobile object position using correction parameters in a trained model based on the mobile object position acquired by the mobile object position acquisition unit 115. The driver assistance unit (driver assistance unit 116) provides driving assistance for the vehicle based on the predicted position of the moving object.

[0126] In the driver assistance system 150A according to the second embodiment described above, correction parameters are calculated by the server 100A, but the prediction of the moving object's position is performed by the in-vehicle unit 110A. In the driver assistance system 150 according to the first embodiment, the server 100A predicted the moving object's position for a huge number of vehicles, so the load on the server 100A increased depending on the time and region. Therefore, by transferring the moving object position prediction processing to the in-vehicle unit 110A, the processing load on the server 100A can be significantly reduced.

[0127] When multiple vehicles are traveling on the same route, the correction parameters transmitted by server 100A to the on-board unit 110A of one vehicle are also available to the other vehicles. Server 100A can then distribute the same correction parameters to other vehicles traveling on the same route. In this case, server 100A does not need to receive verification information from the other vehicles. Therefore, the number of times server 100A communicates with the on-board units 110A of other vehicles is reduced, thereby reducing the amount of data transmitted between server 100A and multiple on-board units 110A.

[0128] The correction parameter acquisition unit 119 may also refer to the correction coefficient calculation of the polynomial shown in Figure 8, and the values ​​of the correction table T1 which is generated rule-based by the coefficient calculation and stored in the in-vehicle device 110. Functionally, this is the same as the correction parameter acquisition in step S17 of Figure 6. The correction parameter acquisition unit 119 outputs the acquired correction parameters to the mobile body position prediction unit 120. The mobile body position prediction unit 120 corrects the mobile body position acquired by the mobile body position acquisition unit 115 using the correction parameters and predicts the corrected mobile body position.

[0129] [Differentiation] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are included. For example, the embodiments described above are described in detail to make the present invention easier to understand, and are not necessarily limited to those having all the configurations described. Other embodiments that can be conceivable within the scope of the technical idea of ​​the present invention are also included within the scope of the present invention. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is possible to add, delete, or replace parts of the configuration of each embodiment with other configurations.

[0130] Furthermore, the driver assistance system 150 may be equipped with an input / output interface (not shown), and a program may be read from another device via a medium available to the input / output interface and the driver assistance system 150 when necessary. Here, the medium refers to, for example, a storage medium or communication medium that can be attached to the input / output interface, i.e., a wired, wireless, optical network, or a carrier wave or digital signal propagating through such a network. In addition, some or all of the functions realized by the program may be realized by hardware circuits or FPGAs. Furthermore, each of the above configurations and functions may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as programs, tables, and files that realize each function can be stored in memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0131] The location of the holder of the communication terminal 130 can be acquired with high precision, and by referencing and utilizing this location for driving assistance, it can contribute to preventing accidents involving vulnerable road users in blind spots on residential streets and narrow alleys. Furthermore, it enables assistance based on the attribute information of the communication terminal holder. By utilizing this for speed arbitration and route setting for autonomous driving, it can contribute to the realization of safer autonomous driving systems.

[0132] When a vehicle is driving through a residential area, the server 100 acquires the mobile location of a communication terminal 130 used by a person inside a house. In this case, the amount of communication between the communication terminal 130 and the servers 100 and 100A, and between the communication terminal 130 and the in-vehicle unit 110A, increases. For this reason, the communication terminal 130 may be pre-loaded with map information including roads, and if the location measured by the communication terminal 130 is indoors, the communication terminal 130 may not transmit its mobile location. This allows the communication terminal 130 to transmit its mobile location only when it is near a road, thus suppressing the increase in communication volume. However, even if the location measured by the communication terminal 130 is indoors, the mobile location measured by GPS may contain errors and may still be transmitted to the servers 100 and 100A. In this case, if the mobile object position prediction units 107 and 120 of servers 100 and 100A determine that the predicted mobile object position is indoors, servers 100 and 100A send information to the in-vehicle unit 110A indicating that the communication terminal 130 is indoors. Upon receiving this information, the in-vehicle unit 110A may cancel the driving assistance based on the mobile object position of the communication terminal 130, which is indoors.

[0133] Furthermore, if the surveillance camera is already installed on a road or house, its location is already known. Therefore, the server 100 may use images acquired from the existing surveillance camera to detect moving objects in blind spots and correct the position of the moving objects. For this reason, the server can also predict the position of moving objects based on the camera's shooting direction and field of view information and transmit the predicted position of the moving objects to the in-vehicle unit 110. From the perspective of protecting personal information, the surveillance camera itself may perform detection and identification processing of pedestrians, etc., and only the processing result may be transmitted to the server 100 before further processing takes place.

[0134] It should be noted that the present invention is not limited to the embodiments described above, and various other applications and modifications can be taken as long as they do not deviate from the gist of the present invention as described in the claims. For example, the embodiments described above are detailed and specific explanations of the configuration of the apparatus and system in order to clearly illustrate the present invention, and are not necessarily limited to having all the configurations described. Furthermore, it is possible to replace some of the configurations of the embodiments described here with the configurations of other embodiments, and it is also possible to add the configurations of other embodiments to the configuration of one embodiment. In addition, it is possible to add, delete, or replace some of the configurations of each embodiment with other configurations. Furthermore, the control lines and information lines shown are those deemed necessary for explanatory purposes, and not all control lines and information lines are necessarily shown in the actual product. In reality, it is safe to assume that almost all components are interconnected. [Explanation of symbols]

[0135] 100...Server, 101...Communication Unit, 102...Moving Object Position Acquisition Unit, 103...Verification Information Acquisition Unit, 104...Section Determination Unit, 105...Correction Parameter Calculation Unit, 106...Correction Parameter Acquisition Unit, 107...Moving Object Position Prediction Unit, 110...In-vehicle Unit, 111...Communication Unit, 112...External Sensor, 113...Moving Object Detection Unit, 114...Moving Object Verification Unit, 115...Moving Object Position Acquisition Unit, 116...Driving Support Unit, 117...Output Unit, 118...Control Unit, 130...Communication Terminal, 131...Attribute Information Acquisition Unit, 132...Moving Object Position Measurement Unit, 133...Communication Unit, 150...Driving Support System

Claims

1. A driver assistance system comprising an in-vehicle device mounted in a vehicle and a server that communicates with the in-vehicle device via a network, which assists in the operation of the vehicle, A mobile object position acquisition unit that acquires first position information of the mobile object from a communication terminal that measures the position of the mobile object, A matching information acquisition unit acquires matching information by associating the first position information with the second position information of the moving body measured by an external sensor mounted on the vehicle or another vehicle. A correction parameter acquisition unit that acquires correction parameters for correcting the first position information, which are generated based on the aforementioned matching information, A mobile body position prediction unit predicts the position of the mobile body based on the first position information and the correction parameters, The system includes a driver assistance unit that provides driving assistance for the vehicle based on the predicted position of the moving object. Driver assistance system.

2. The server comprises the correction parameter acquisition unit and the moving body position prediction unit, and transmits the position of the moving body predicted by the moving body position prediction unit to the in-vehicle unit. The in-vehicle unit includes the driver assistance unit and provides driver assistance for the vehicle. The driver assistance system according to claim 1.

3. The aforementioned server, The section determination unit includes the position of the moving object indicated by the first position information and determines a predetermined section set on the map, The learning unit updates the correction parameters for each predetermined section set on the map based on the matching information, and generates a different trained model for each predetermined section. The correction parameter is an inference parameter of the trained model that outputs a correction value for the first position information from the difference between the first position information and the second position information. The moving object position prediction unit predicts the position of the moving object based on the inference parameters of the trained model, which differ for each predetermined interval. The driver assistance system according to claim 2.

4. The aforementioned matching information consists of the uncorrected position of the moving body obtained from the first position information, the position of the moving body obtained from the second position information measured by the external sensor, the type of the moving body, and the speed of the moving body. The driver assistance system according to claim 3.

5. The aforementioned driving support unit provides driving support for the vehicle based on the attribute information of the mobile body holding the communication terminal, which is obtained from the communication terminal. The driver assistance system according to claim 4.

6. The in-vehicle device includes a moving object detection unit that detects the second position information of the moving object based on the information of the moving object measured by the external sensor, A mobile object matching unit compares the position of the mobile object obtained from the communication terminal by the mobile object position acquisition unit with the position of the mobile object based on the second position information and outputs the matching information. The mobile body position acquisition unit outputs the uncorrected position of the mobile body obtained from the first position information to the mobile body matching unit, and outputs the position of the mobile body predicted by the mobile body position prediction unit to the driving support unit, The driving support unit comprises The driver assistance system according to claim 5.

7. The matching information acquisition unit acquires the matching information from the mobile body matching unit, The section determination unit determines a first predetermined section that includes the location of the communication terminal based on the first location information and the matching information, and outputs the determined first predetermined section and the matching information to the learning unit. The learning unit generates the trained model based on the first predetermined interval and the matching information. The driver assistance system according to claim 6.

8. The matching information acquisition unit acquires the matching information from the mobile body matching unit, The section determination unit determines a first predetermined section that includes the location of the communication terminal, based on the first location information and the matching information. The correction parameter acquisition unit acquires, as the correction parameter, a correction coefficient of a polynomial model that relates the uncorrected position of the moving body obtained from the first position information, the position of the moving body obtained from the second position information, and the first predetermined interval, which are included in the matching information. The moving object position prediction unit corrects the moving object position acquired by the moving object position acquisition unit using the correction parameter to predict the moving object position. The driver assistance system according to claim 6.

9. The in-vehicle unit comprises the moving object position acquisition unit, the correction parameter acquisition unit, the moving object position prediction unit, and the driving support unit. The mobile object position acquisition unit acquires the mobile object position from the communication terminal, The correction parameter acquisition unit acquires the correction parameter generated by the server based on the second position information of the moving body measured by the external sensor. The moving body position prediction unit predicts the moving body position using the correction parameter with respect to the moving body position acquired by the moving body position acquisition unit. The aforementioned driving support unit provides driving support for the vehicle based on the predicted position of the moving object. The driver assistance system according to claim 1.

10. A driver assistance method performed in a driver assistance system that includes an in-vehicle device mounted in a vehicle and a server that communicates with the in-vehicle device via a network, the driver assistance method being performed in a driver assistance system that assists in driving the vehicle, A step of obtaining first position information of the moving object from a communication terminal that measures the position of the moving object, A step of acquiring matching information by associating the first location information with the second location information of the moving body measured by an external sensor mounted on the vehicle or another vehicle, A step of obtaining correction parameters for correcting the first position information, which are generated based on the aforementioned matching information, A step of predicting the position of the moving body based on the first position information and the correction parameter, The step of providing driving assistance for the vehicle based on the predicted position of the moving object includes: Driving assistance methods.

11. An in-vehicle device that is installed in a vehicle and communicates with a server via a network to assist in the driving of the vehicle, A mobile object position acquisition unit that acquires first position information of the mobile object from a communication terminal that measures the position of the mobile object, A matching information acquisition unit acquires matching information by associating the first position information with the second position information of the moving body measured by an external sensor mounted on the vehicle. A correction parameter acquisition unit that acquires correction parameters for correcting the first position information, which are generated based on the aforementioned matching information, A mobile body position prediction unit predicts the position of the mobile body based on the first position information and the correction parameters, The system includes a driver assistance unit that provides driving assistance for the vehicle based on the predicted position of the moving object. Onboard machine.

Images