Sensor alignment device, driving control system, and correction amount inference method

Abstract

The present invention corrects the axial offset of a sensor. The present invention is a sensor alignment device comprising: a target position relationship processing unit that outputs positional relationship information of a first target and a second target; a sensor observation information processing unit that transforms the observation results of the first target and the second target into a predetermined unified coordinate system based on coordinate transformation parameters, synchronizes the time to a predetermined timing, and extracts first target information indicating the position of the first target and second target information indicating the position of the second target; a position estimation unit that uses the first target information, the second target information, and the positional relationship information to estimate the position of the second target; and a sensor correction amount estimation unit that uses the second target information and the estimated position of the second target to calculate the offset of the second sensor and estimate the correction amount, wherein the sensor alignment device changes the coordinate transformation parameters based on the correction amount.

Description

Technical Field

[0001] The present invention relates to a vehicle-mounted control device, and in particular to a sensor alignment device for correcting sensor data. Background Art

[0002] Driving assistance systems and automated driving systems have been developed to achieve various goals, including reducing traffic accidents, alleviating driver burdens, improving fuel efficiency to reduce the impact on the global environment, and providing mobility equipment to people with disabilities to achieve a sustainable society. These systems are equipped with multiple vehicle perimeter monitoring sensors to monitor the vehicle's surroundings on behalf of the driver. Furthermore, to ensure the safety of these systems, they require the ability to correct any deviations in the mounting angles of these vehicle perimeter monitoring sensors.

[0003] As background technology in this technical field, the following prior art exists. Patent Document 1 (Japanese Patent Laid-Open No. 2016-065759) describes an axial offset estimation device, which is a collision mitigation device that determines whether a first object and a second object are the same object. The first object is detected by an electromagnetic wave sensor that detects the position of an object by transmitting and receiving electromagnetic waves, and the second object is detected by an image sensor that detects the position of an object by performing image processing on a camera image. If the first object and the second object are determined to be the same object, the angle formed by a first line segment connecting the first object's position and the first object and a second line segment connecting the second object's position is calculated as the axial offset (see abstract).

[0004] Furthermore, Patent Document 2 (Japanese Patent Application Laid-Open No. 2004-317507) describes an axis adjustment method that uses an adjustment target, whose light and dark pattern and shape are designed to resemble the detection surface, to adjust the radar's axis offset, including in the roll direction, using an inverted W-shaped waveform of the received intensity relative to the scanning direction position. This waveform allows quantitative calculation of the axis offset for each direction. Furthermore, using a fusion method, an image of the same adjustment target is captured after the radar's axis adjustment, and the camera's axis relative to the radar is adjusted based on the coordinates of multiple feature points in the image on the image plane (see abstract).

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Laid-Open No. 2016-065759

[0008] Patent Document 2: Japanese Patent Application Laid-Open No. 2004-317507 Summary of the Invention

[0009] Problems to be solved by the invention

[0010] However, while Patent Document 1 determines whether the object detected by the radio wave sensor and the object detected by the image sensor are the same object and calculates the axis offset if they are determined to be the same object, it does not perform the axis offset calculation if the detected objects are not the same object. Furthermore, there is the problem of deviations in detection points or differences in detection points between multiple sensors, regardless of the lateral width of the detected object. Furthermore, Patent Document 2 performs axis offset adjustment using target detection information using a single radar, and then uses a camera to capture the target for axis offset adjustment. However, this ultimately requires manual correction or physical correction of the sensor's axis offset using an axis adjustment device.

[0011] Technical means to solve the problem

[0012] A representative example of the invention disclosed in this application is shown below. Specifically, a sensor alignment device receives inputs of observation results of a first target by a first sensor and observation results of a second target by a second sensor, and is characterized by comprising: a target position relationship processing unit that outputs positional relationship information of the first target and the second target; a sensor observation information processing unit that transforms the observation results of the first target and the second target into a predetermined unified coordinate system based on coordinate transformation parameters, synchronizes the time with a predetermined timing, and extracts first target information indicating the position of the first target and second target information indicating the position of the second target; a position estimation unit that estimates the position of the second target using the first target information, the second target information, and the positional relationship information; and a sensor correction amount estimation unit that calculates an offset of the second sensor using the second target information and the estimated position of the second target and estimates a correction amount, wherein the sensor alignment device changes the coordinate transformation parameters based on the correction amount.

[0013] Effects of the Invention

[0014] According to the present invention, it is possible to correct the axial offset of the sensor. Other problems, configurations, and effects than those described above will become clearer through the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a functional block diagram of a sensor fusion device with a sensor alignment function according to Example 1.

[0016] Figure 2 This is a conceptual diagram showing a processing method of the sensor fusion device with a sensor alignment function according to the first embodiment.

[0017] Figure 3 This is a conceptual diagram showing a processing method of the sensor fusion device with a sensor alignment function according to the first embodiment.

[0018] Figure 4 This is a conceptual diagram showing a processing method of the sensor fusion device with a sensor alignment function according to the first embodiment.

[0019] Figure 5 This is a diagram showing an example of sensor alignment processing using a road structure in the first embodiment.

[0020] Figure 6 This is a diagram showing an example of sensor alignment processing using a road structure in the first embodiment.

[0021] Figure 7 This is a diagram showing an example of sensor alignment processing using a road structure in the first embodiment.

[0022] Figure 8 This is a diagram showing an example of sensor alignment processing using a road structure in the first embodiment.

[0023] Figure 9 This is a diagram showing an example of sensor alignment processing using a road structure in the first embodiment.

[0024] Figure 10 This is a functional block diagram of a sensor fusion device with a sensor alignment function according to Example 2.

[0025] Figure 11 This is a functional block diagram of a sensor fusion device with a sensor alignment function according to Example 3.

[0026] Figure 12 This is a functional block diagram of a sensor fusion device with a sensor alignment function according to a fourth embodiment. DETAILED DESCRIPTION

[0027] Hereinafter, the specific embodiments of the present invention will be described in detail with reference to the accompanying drawings. In all the drawings used to describe the specific embodiments, the same reference numerals are given to blocks or elements having the same functions, and their repeated descriptions are omitted.

[0028] <Example 1>

[0029] Figure 1 FIG. 1 is a functional block diagram showing an embodiment of a sensor fusion device 1 with a sensor alignment function.

[0030] like Figure 1As shown, the sensor fusion device 1 of this embodiment includes a sensor observation information processing unit 100, a target position relationship processing unit 110, a target position relationship information storage unit 120, a position estimation unit 130, and a data integration unit 200. The sensor alignment function is implemented by the components of the sensor fusion device 1 other than the data integration unit 200, and the sensor alignment device is implemented by the components other than the data integration unit 200. Furthermore, the output signals of the first vehicle periphery monitoring sensor 10a, the second vehicle periphery monitoring sensor 10b, and the own vehicle behavior detection sensor 20 are input to the sensor fusion device 1.

[0031] The first and second vehicle periphery monitoring sensors 10a, 10b are sensors that detect road structures (landmarks or road surface indicators) around the vehicle 800. For example, the first and second vehicle periphery monitoring sensors 10a, 10b include millimeter-wave radars, cameras (visible light, near-infrared, mid-infrared, or far-infrared cameras), LiDAR (Light Detection and Ranging), sonar, Time of Flight (TOF) sensors, or a combination thereof.

[0032] The own vehicle behavior detection sensor 20 is a sensor group that detects the speed, yaw rate, and steering angle of the own vehicle 800. As an example, it includes a wheel speed sensor, a steering angle sensor, and the like.

[0033] The sensor fusion device 1 (electronic control unit) and various sensors (first vehicle periphery monitoring sensor 10a, second vehicle periphery monitoring sensor 10b, etc.) of this embodiment include a computer (microcomputer) including a computing device, a memory, and an input / output device.

[0034] The computing device includes a processor that executes the program stored in the memory. Part of the processing performed by the computing device when executing the program may also be performed by other computing devices (for example, hardware such as FPGA (Field Programmable Gate Array) and ASIC (Application Specific Integrated Circuit)).

[0035] Memory includes ROM and RAM, which are non-volatile storage elements. ROM stores fixed programs (such as BIOS). RAM, which is a high-speed, volatile storage element such as DRAM (Dynamic Random Access Memory) and a non-volatile storage element such as SRAM (Static Random Access Memory), stores programs executed by the computing device and the data used during program execution.

[0036] The input / output device is an interface for transmitting the processing contents of the electronic control device and the sensor to the outside and receiving data from the outside in accordance with a predetermined protocol.

[0037] The program for the computing device to execute is stored in the non-volatile memory, which is a non-transitory storage medium of the electronic control device and the sensor.

[0038] The target position relationship information storage unit 120 records the position information of stationary objects that serve as targets in the processing described below (e.g., guardrail posts and beams, road signs, road markings such as white lines, soundproof walls, and raised road markers (such as flutter bars, cat's eye markings, and Botts' dots)). For example, the position information of stationary objects is stored as map information. In addition to the position information of the stationary objects that serve as targets, the target category is also recorded in association with the location information. Here, the target category refers to the classification of the target that indicates the nature of the target and the arrangement of the detection points (e.g., whether the detection points are arranged in a single pattern or in a specific pattern).

[0039] Figures 2 to 4 In the example, the own vehicle 800 is traveling at a speed of 0 or higher along the direction of the own vehicle's travel path (i.e., the own vehicle 800 may be traveling or stationary). The own vehicle 800 is equipped with a first vehicle periphery monitoring sensor 10a and a second vehicle periphery monitoring sensor 10b. Furthermore, around the own vehicle 800, there are a first target 300 and a first non-target 390 observed by the first vehicle periphery monitoring sensor 10a, and a second target 400 and a second non-target 490 observed by the second vehicle periphery monitoring sensor 10b. Here, the first target 300 is a target that can be detected with high accuracy by the first vehicle periphery monitoring sensor 10a, and the second target 400 is a target that can be detected with high accuracy by the second vehicle periphery monitoring sensor 10b. Among the objects observed by the vehicle periphery monitoring sensors 10a and 10b, the targets 300 and 400 are stationary objects whose positions are recorded in the target position relationship information storage unit 120, and the non-targets 390 and 490 are stationary objects or moving bodies whose positions are not recorded in the target position relationship information storage unit 120.

[0040] Next, use Figures 1 to 4, the processing of the sensor fusion device 1 with a sensor alignment function according to an embodiment of the present invention is described.

[0041] Figure 2 (A) to Figure 2 (C) shows the concept of the processing method under the following state: the second vehicle periphery monitoring sensor 10b is installed on the own vehicle 800 with an axis offset of an angle θ1 in the horizontal direction, the first vehicle periphery monitoring sensor 10a detects a detection point, namely the first target 300 and the first non-target 390, and the second vehicle periphery monitoring sensor 10b detects a detection point, namely the second target 400 and the second non-target 490. Figure 2 (A) shows the sensor information before target extraction. Figure 2 (B) shows the extracted target location information. Figure 2 (C) shows the target position information after the offset is eliminated. In the figure, the second target 400 and the second non-target 490 indicated by the solid line represent the positions observed by the second vehicle periphery monitoring sensor 10b, and the second target 400 and the second non-target 490 indicated by the dotted line represent the actual positions.

[0042] like Figure 2 As shown in (A), the first vehicle periphery monitoring sensor 10a detects a first target 300 and a first non-target 390, and outputs the relative coordinates of the first target 300 and the first non-target 390 with respect to at least the own vehicle 800. The second vehicle periphery monitoring sensor 10b detects a second target 400 and a second non-target 490, and outputs the relative coordinates of the second target 400 and the second non-target 490 with respect to at least the own vehicle 800. If each target and non-target is a moving object, it is preferable that the absolute speed is also output.

[0043] The target position relationship processing unit 110 generates a relative position relationship between the first target 300 and the second target 400. Specifically, it receives the position information and target category of the first target 300 and the position information and target category of the second target 400 from the target position relationship information storage unit 120 as input and converts it into a relative position relationship between the first target 300 and the second target 400. Furthermore, the target position relationship processing unit 110 outputs the predetermined allowable axis misalignment range for each vehicle periphery monitoring sensor 10 to the sensor observation information processing unit 100. Furthermore, the target position relationship processing unit 110 outputs the relative position relationship between the first target 300 and the second target 400, the category of the first target 300, and the category of the second target 400 to the position estimation unit 130.

[0044] Here, the relative position relationship is the unified relative coordinates of the second target 400 with the first target 300 as the starting point, or the unified relative coordinates of the first target 300 with the second target 400 as the starting point, which are calculated based on the coordinates of the targets 300 and 400 whose positions are recorded in the target position relationship information storage unit 120. In addition, the unified relative coordinates are a coordinate system that summarizes the coordinates based on the data output by the plurality of vehicle periphery monitoring sensors 10a and 10b, for example, Figure 2 As shown in (D), the x-axis is defined as the front direction of the own vehicle 800 and the y-axis is defined as the left direction of the own vehicle 800, with the front center of the own vehicle 800 as the starting point.

[0045] The sensor observation information processing unit 100 mainly performs coordinate conversion of input sensor data, time synchronization of detection results between sensors, and target determination.

[0046] The coordinate transformation unit 100a of the sensor data uses coordinate transformation parameters to transform the relative coordinates of the first target 300 and the own vehicle 800, the relative coordinates of the first non-target 390 and the own vehicle 800, the relative coordinates of the second target 400 and the own vehicle 800, and the relative coordinates of the second non-target 490 and the own vehicle 800 into unified relative coordinates with respect to the own vehicle 800.

[0047] The detection results of the vehicle's speed, yaw rate, and steering angle from the vehicle's behavior detection sensor 20 are input to a time synchronization unit 100b, which synchronizes the detection times between the sensors. Using the input detection results of the vehicle's speed, yaw rate, and steering angle, the time synchronization unit 100b corrects the unified relative coordinates of the first target 300, the first non-target 390, the second target 400, and the second non-target 490 to unified relative coordinates at a predetermined timing, thereby synchronizing the times of the detection results from each sensor. The synchronized unified relative coordinates of the first target 300, the first non-target 390, the second target 400, and the second non-target 490 are then output to the data integration unit 200 and the target determination unit 100c. If the target and non-target are moving objects, the absolute speeds are preferably also output.

[0048] The target determination unit 100c determines the target. Specifically, the target determination unit 100c receives the unified relative coordinates of the first target 300 after time synchronization, the unified relative coordinates of the first non-target 390 after time synchronization, the unified relative coordinates of the second target 400 after time synchronization, the unified relative coordinates of the second non-target 490 after time synchronization, and the axis offset allowable range from the time synchronization unit 100b as input. The target determination unit 100c extracts the first target 300 and the second target 400 based on the axis offset allowable range of the vehicle periphery monitoring sensor 10a and the vehicle periphery monitoring sensor 10b, and outputs the unified relative coordinates of the first target 300 and the unified relative coordinates of the second target 400 to the position estimation unit 130 (reference Figure 2 (B)).

[0049] Furthermore, the sensor observation information processing unit 100 outputs the unified relative coordinates and absolute velocity of the time-synchronized first target 300, the unified relative coordinates and absolute velocity of the time-synchronized first non-target 390, the unified relative coordinates and absolute velocity of the time-synchronized second target 400, and the unified relative coordinates and absolute velocity of the time-synchronized second non-target 490 to the data integration unit 200. Furthermore, the sensor observation information processing unit 100 outputs the unified relative coordinates of the second target 400 to the correction value estimation unit 140.

[0050] The data integration unit 200 integrates all input information and outputs the integration result to the driving control device 2. The driving control device 2 is an automatic driving system (AD-ECU) or a driving support system that controls the driving of the vehicle using the output from the sensor fusion device 1.

[0051] The position estimation unit 130 estimates the true unified relative coordinates of the second target 400. Specifically, the relative positional relationship between the first target 300 and the second target 400, the category of the first target 300, the category of the second target 400, the unified relative coordinates of the first target 300, and the unified relative coordinates of the second target 400 are input to the position estimation unit 130. The position estimation unit 130 uses the relative positional relationship between the first target 300 and the second target 400 acquired from the target position relationship processing unit 110 (the positions of which are recorded in the target position relationship information storage unit 120) to eliminate the offset of the x-axis (vertical) direction and the y-axis (horizontal) direction of the first target 300 and the second target 400 (reference Figure 2 (C)). The position of first target 300 after the offset is eliminated becomes the true position of second target 400. The unified relative coordinates of first target 300 after the offset is eliminated, the type of first target 300, the unified relative coordinates of second target 400, and the type of second target 400 are output to correction amount estimation unit 140.

[0052] The correction amount estimation unit 140 estimates the offset and calculates the correction amount. Specifically, the offset estimation unit 140a receives the unified relative coordinates of the first target 300 after offset cancellation, the type of the first target 300, the unified relative coordinates of the second target 400, and the type of the second target 400 as input. Using the unified relative coordinates of the installation position of the first vehicle periphery monitoring sensor 10a and the unified relative coordinates of the installation position of the second vehicle periphery monitoring sensor 10b, and using a calculation method corresponding to the type of the first target 300 and the type of the second target 400, the correction amount calculation unit 140b calculates a sensor coordinate transformation correction value based on the axial offset of the second vehicle periphery monitoring sensor 10b calculated by the offset estimation unit 140a and outputs it to the sensor observation information processing unit 100.

[0053] The sensor observation information processing unit 100 (the sensor data coordinate conversion unit 100a) changes the coordinate conversion parameters based on the sensor coordinate conversion correction value received from the correction value estimation unit 140. Alternatively, the correction value estimation unit 140 may calculate the coordinate conversion parameters and output the changed coordinate conversion parameters to the sensor observation information processing unit 100.

[0054] Figure 3 (A) to Figure 3 (C) shows the concept of the processing method under the following state: the second vehicle periphery monitoring sensor 10b is installed on the own vehicle 800 with an axis offset of an angle θ1 in the horizontal direction, the first vehicle periphery monitoring sensor 10a detects detection points with a certain arrangement pattern, namely the first target 300, 310, 320 and the first non-target 390, and the second vehicle periphery monitoring sensor 10b detects one detection point, namely the second target 400 and the second non-target 490. Figure 3 (A) shows the sensor information before target extraction. Figure 3 (B) shows the location information of the extracted target. Figure 3 (C) shows the target position information after the offset is eliminated. In the figure, the second target 400 and the second non-target 490 indicated by the solid line represent the positions observed by the second vehicle periphery monitoring sensor 10b, and the second target 400 and the second non-target 490 indicated by the dotted line represent the actual positions.

[0055] Below, Figure 2 The main differences in the processing flow are Figure 3 The processing flow under the status of is described below.

[0056] like Figure 3As shown in (A), the first vehicle periphery monitoring sensor 10a detects first targets 300-320 and first non-target 390, and outputs the relative coordinates of at least the first targets 300-320 and first non-target 390 relative to the vehicle 800. The second vehicle periphery monitoring sensor 10b detects a second target 400 and a second non-target 490, and outputs the relative coordinates of at least the second target 400 and second non-target 490 relative to the vehicle 800. The first targets 300-320 are components of a road structure that is periodically arranged at intervals of less than a few meters. Furthermore, if each target and non-target is a moving object, it is preferable to also output its absolute velocity.

[0057] The object position relationship processing unit 110 generates a relative position relationship between the first objects 300-320 and the second object. Specifically, it receives the position information and object types of the first objects 300-320 and the position information and object type of the second object 400 from the object position relationship information storage unit 120 as input and converts it into a relative position relationship between the first objects 300-320 and the second object 400. The permissible axis misalignment range is output to the sensor observation information processing unit 100. Furthermore, the relative position relationship between the first road structure 500 having a specific arrangement pattern of the first objects 300-320 and the second object 400, the types of the first objects 300-320, and the type of the second object 400 are output to the position estimation unit 130.

[0058] The sensor observation information processing unit 100 mainly performs coordinate conversion of input sensor data, time synchronization of detection results between sensors, and target determination.

[0059] The coordinate transformation unit 100a of the sensor data uses coordinate transformation parameters to transform the relative coordinates of the first targets 300~320 and the own vehicle 800, the relative coordinates of the first non-target 390 and the own vehicle 800, the relative coordinates of the second target 400 and the own vehicle 800, and the relative coordinates of the second non-target 490 and the own vehicle 800 into unified relative coordinates with respect to the own vehicle 800.

[0060] The detection results of the vehicle's speed, yaw rate, and steering angle from the vehicle's behavior detection sensor 20 are input to a time synchronization unit 100b, which synchronizes the detection times between the sensors. Using the input detection results of the vehicle's speed, yaw rate, and steering angle, the time synchronization unit 100b corrects the unified relative coordinates of the first targets 300-320, the first non-target 390, the second target 400, and the second non-target 490 to unified relative coordinates at a predetermined timing, thereby synchronizing the times of the detection results from each sensor. The synchronized unified relative coordinates of the first targets 300-320, the first non-target 390, the second target 400, and the second non-target 490 are then output to the data integration unit 200 and the target determination unit 100c. If the targets and non-targets are moving objects, the absolute speeds are preferably also output.

[0061] The target determination unit 100c determines the target. Specifically, the target determination unit 100c receives the unified relative coordinates of the first target 300-320 after time synchronization, the unified relative coordinates of the first non-target 390 after time synchronization, the unified relative coordinates of the second target 400 after time synchronization, the unified relative coordinates of the second non-target 490 after time synchronization, and the axis offset allowable range from the time synchronization unit 100b as input. The target determination unit 100c extracts the first target 300-320 and the second target 400 based on the axis offset allowable range of the vehicle periphery monitoring sensor 10a and the vehicle periphery monitoring sensor 10b, and outputs the unified relative coordinates of the first target 300-320 and the unified relative coordinates of the second target 400 to the position inference unit 130 (reference Figure 3 (B)).

[0062] Furthermore, the sensor observation information processing unit 100 outputs the unified relative coordinates and absolute speed of the time-synchronized first targets 300 to 320, the time-synchronized unified relative coordinates and absolute speed of the time-synchronized first non-target 390, the time-synchronized unified relative coordinates and absolute speed of the time-synchronized second target 400, and the time-synchronized unified relative coordinates and absolute speed of the time-synchronized second non-target 490 to the data integration unit 200. Furthermore, the sensor observation information processing unit 100 outputs the unified relative coordinates of the second target 400 to the correction value estimation unit 140.

[0063] The position estimation unit 130 estimates the true lateral position of the second object 400. Specifically, the relative positional relationship between the first objects 300-320 and the second object 400, the types of the first objects 300-320, the type of the second object 400, the unified relative coordinates of the first objects 300-320, and the unified relative coordinates of the second object 400 are input to the position estimation unit 130. Based on the unified relative coordinates of the first objects 300-320 and the object types, the position estimation unit 130 calculates the lateral position of the first road structure 500 having the specific arrangement pattern of the first objects 300-320 in the unified coordinate system as viewed from the vehicle 800. Next, the relative positional relationship between the second target 400 and the first road structure 500 (the position of which is recorded in the target position relationship information storage unit 120) obtained from the target position relationship processing unit 110 is used to eliminate the offset between the first target 300 and the first road structure 500 in the x-axis (longitudinal) direction and the y-axis (lateral) direction (reference Figure 3 (C)). The lateral position of the first road structure 500 after the offset is eliminated becomes the true lateral position of the second target 400. The lateral position of the first road structure 500 in the unified relative coordinates after the offset is eliminated, the types of the first targets 300 to 320, the unified relative coordinates of the second target 400, and the type of the second target 400 are output to the correction amount estimation unit 140.

[0064] The correction amount estimation unit 140 estimates the offset and calculates the correction amount. Specifically, the offset estimation unit 140a receives the unified relative coordinates of the first target 300 after offset cancellation, the types of the first targets 300-320, the unified relative coordinates of the first road structure 500, and the type of the second target 400 as input. Using the unified relative coordinates of the installation position of the first vehicle periphery monitoring sensor 10a and the unified relative coordinates of the installation position of the second vehicle periphery monitoring sensor 10b, and using a calculation method appropriate to the types of the first targets 300-320 and the second target 400, the correction amount calculation unit 140b calculates a sensor coordinate transformation correction value based on the axial offset of the second vehicle periphery monitoring sensor 10b calculated by the offset estimation unit 140a and outputs it to the sensor observation information processing unit 100.

[0065] Figure 4 (A) to Figure 4(C) shows the concept of the processing method under the following state: the second vehicle periphery monitoring sensor 10b is installed on the own vehicle 800 with an axis offset of an angle θ1 in the horizontal direction, the first vehicle periphery monitoring sensor 10a detects detection points having a certain arrangement pattern, namely the first targets 300, 310, 320, and the second vehicle periphery monitoring sensor 10b detects detection points having a certain arrangement pattern, namely the second targets 400, 410, 420.

[0066] Figure 4 (A) shows the sensor information before target extraction. Figure 4 (B) shows the location information of the extracted target. Figure 4 (C) shows the target position information after the offset is eliminated. In the figure, the second target 400 and the second non-target 490 indicated by the solid line represent the positions observed by the second vehicle periphery monitoring sensor 10b, and the second target 400 and the second non-target 490 indicated by the dotted line represent the actual positions.

[0067] Below, Figure 3 The main differences in the processing flow are Figure 4 The processing flow under the status of is described below.

[0068] like Figure 4 As shown in (A), the first vehicle periphery monitoring sensor 10a detects first targets 300-320 and a first non-target 390, and outputs the relative coordinates of at least the first targets 300-320 and the first non-target 390 relative to the vehicle 800. The second vehicle periphery monitoring sensor 10b detects second targets 400-420 and a second non-target 490, and outputs the relative coordinates of at least the second targets 400-420 and the second non-target 490 relative to the vehicle 800. The first targets 300-320 are components of a road structure having a periodic arrangement pattern at intervals of less than several meters. The second targets 400-420 are components of a road structure having a periodic arrangement pattern at intervals of less than several meters. Furthermore, when each target and non-target is a moving object, it is preferable to also output the absolute velocity.

[0069] The object position relationship processing unit 110 generates a relative position relationship between a first road structure 500 having a specific arrangement pattern consisting of first objects 300-320 and a second road structure 600 having a specific arrangement pattern consisting of second objects 400-420. Specifically, the object position relationship processing unit 110 receives the position information and object types of the first objects 300-320 and the position information and object types of the second objects 400-420 from the object position relationship information storage unit 120 as input and converts it into a relative position relationship between the first road structure 500 and the second road structure 600. The permissible axis misalignment range is output to the sensor observation information processing unit 100. Furthermore, the object position relationship processing unit 110 outputs the relative position relationship between the first road structure 500 and the second road structure 600, the types of the first objects 300-320, and the types of the second objects 400-420 to the position estimation unit 130.

[0070] The sensor observation information processing unit 100 mainly performs coordinate conversion of input sensor data, time synchronization of detection results between sensors, and target determination.

[0071] The coordinate conversion unit 100a converts the coordinates of the sensor data. Specifically, the coordinate conversion unit 100a uses coordinate conversion parameters to convert the relative coordinates of the first targets 300-320 and the vehicle 800, the relative coordinates of the first non-target 390 and the vehicle 800, the relative coordinates of the second targets 400-420 and the vehicle 800, and the relative coordinates of the second non-target 490 and the vehicle 800 into unified relative coordinates relative to the vehicle 800.

[0072] The time synchronization unit 100b synchronizes the time between sensor data detected at different times. Specifically, the detection results of the speed, yaw rate, and steering angle of the vehicle 800 from the vehicle behavior detection sensor 20 are input to the time synchronization unit 100b. Using the input detection results of the speed, yaw rate, and steering angle of the vehicle 800, the time synchronization unit 100b corrects the unified relative coordinates of the first targets 300-320, the first non-target 390, the second targets 400-420, and the second non-target 490 to the unified relative coordinates at a predetermined time, thereby synchronizing the time of the detection results of each sensor. The unified relative coordinates of the first targets 300-320 after time synchronization, the unified relative coordinates of the first non-target 390 after time synchronization, the unified relative coordinates of the second targets 400-420 after time synchronization, and the unified relative coordinates of the second non-target 490 after time synchronization are output to the data integration unit 200 and the target determination unit 100c. If the targets and non-targets are moving objects, the absolute speed is preferably also output.

[0073] The target determination unit 100c determines the target. Specifically, the target determination unit 100c receives the unified relative coordinates of the first target 300-320 after time synchronization, the unified relative coordinates of the first non-target 390 after time synchronization, the unified relative coordinates of the second target 400-420 after time synchronization, the unified relative coordinates of the second non-target 490 after time synchronization, and the axis offset allowable range from the time synchronization unit 100b as input. The target determination unit 100c extracts the first target 300-320 and the second target 400-420 based on the axis offset allowable range of the vehicle periphery monitoring sensor 10a and the vehicle periphery monitoring sensor 10b, and outputs the unified relative coordinates of the first target 300-320 and the unified relative coordinates of the second target 400-420 to the position inference unit 130 (reference Figure 4 (B)).

[0074] Furthermore, the sensor observation information processing unit 100 outputs the unified relative coordinates and absolute speed of the time-synchronized first targets 300-320, the time-synchronized unified relative coordinates and absolute speed of the time-synchronized first non-target 390, the time-synchronized unified relative coordinates and absolute speed of the time-synchronized second targets 400-420, and the time-synchronized unified relative coordinates and absolute speed of the time-synchronized second non-target 490 to the data integration unit 200. Furthermore, the sensor observation information processing unit 100 outputs the unified relative coordinates of the second targets 400-420 to the correction value estimation unit 140.

[0075] The position estimation unit 130 estimates an approximate function in the real unified relative coordinates of the second road structure 600, which is composed of the second objects 400-420. Specifically, the relative positional relationship between the first objects 300-320 and the second objects 400-420, the types of the first objects 300-320, the types of the second objects 400-420, the unified relative coordinates of the first objects 300-320, and the unified relative coordinates of the second objects 400-420 are input to the position estimation unit 130. Based on the unified relative coordinates and the object types of the first objects 300-320, the position estimation unit 130 calculates an approximate function in the unified relative coordinates of the first road structure 500, which has a specific arrangement pattern composed of the first objects 300-320, as viewed from the vehicle 800. Furthermore, the position estimation unit 130 calculates an approximate function of the second road structure 600 having a specific arrangement pattern of the second objects 400 to 420 in the unified relative coordinates as viewed from the vehicle 800, based on the unified relative coordinates of the second objects 400 to 420 and the object type. Next, the relative positional relationship between the first road structure 500 and the second road structure 600 (whose positions are recorded in the object position relationship information storage unit 120) obtained from the object position relationship processing unit 110 is used to eliminate the offset between the first road structure 500 and the second road structure 600 in the x-axis (longitudinal) direction and the y-axis (lateral) direction (reference). Figure 4 (C)). The approximate function of the first road structure 500 in the unified relative coordinates after the offset is eliminated becomes the true approximate function of the second road structure 600. The approximate function of the first road structure 500 in the unified relative coordinates after the offset is eliminated, the categories of the first targets 300 to 320, the approximate function of the second road structure 600 in the unified relative coordinates, and the categories of the second targets 400 to 420 are output to the correction amount estimation unit 140.

[0076] The correction amount estimation unit 140 estimates the offset and calculates the correction amount. Specifically, the offset estimation unit 140a receives as input the offset-eliminated approximate function of the first road structure 500, the types of the first objects 300-320, the unified relative coordinates of the second road structure 600, and the types of the second objects 400-420. Using the unified relative coordinates of the installation position of the first vehicle periphery monitoring sensor 10a and the unified relative coordinates of the installation position of the second vehicle periphery monitoring sensor 10b, and using a calculation method appropriate to the types of the first objects 300-320 and the second objects 400-420, the correction amount calculation unit 140b calculates a sensor coordinate transformation correction value based on the axial offset of the second vehicle periphery monitoring sensor 10b calculated by the offset estimation unit 140a and outputs it to the sensor observation information processing unit 100.

[0077] In the sensor fusion device 1 of this embodiment, the relative position relationship between the targets can be used to calculate the correction amount caused by the horizontal axis offset (angle θ1) of the second vehicle periphery monitoring sensor 10b, thereby integrating the positions of the target objects obtained by multiple vehicle periphery monitoring sensors 10 into a unified relative coordinate.

[0078] Next, use Figures 5 to 9 An example of sensor alignment using an actually installed road structure will be described. Consider a case where the first vehicle periphery monitoring sensor 10a is a camera and the second vehicle periphery monitoring sensor 10b is a radar.

[0079] Figure 5 3 is a diagram showing observation results of road signs by the vehicle periphery monitoring sensor 10 .

[0080] Regarding the road sign as a road structure, the first vehicle periphery monitoring sensor 10a (camera) observes a signboard 1000 of a certain size as a first target 300 . Meanwhile, the second vehicle periphery monitoring sensor 10b (radar) detects a pillar 1010 as a second target 400 .

[0081] In roadway signage, the size of signboard 1000 and the height below the signboard (the length of support 1010) are fixed. Therefore, the position estimation unit 130 uses the relative positional relationship between the first target 300 (signboard 1000) and the second target 400 (support 1010) to calculate the offset of the first target 300. The offset of the first target 300 is then eliminated using the observed position and relative positional relationship of the first target 300. The correction amount estimation unit 140 uses the offset-eliminated position of the first target 300 to calculate the axial offset of the second vehicle perimeter monitoring sensor 10b and the corresponding correction amount.

[0082] Figure 6 3 is a diagram showing the observation result of the guardrail by the vehicle periphery monitoring sensor 10 .

[0083] Regarding the guardrail, a road structure, the spacing between pillars 1110 is determined based on the length of the crossbeam 1100. Therefore, the first vehicle perimeter monitoring sensor 10a (camera) observes the crossbeam 1100, a structure of a certain size, as the second targets 300, 310, and 320. Meanwhile, the second vehicle perimeter monitoring sensor 10b (radar) observes the pillars 1110 arranged in a certain pattern as the second targets 400, 410, and 420.

[0084] In a guardrail, a road structure, the relative positional relationship between the beam and the support is fixed. Therefore, the position estimation unit 130 uses the relative positional relationship between the first targets 300, 310, 320 (beam 1100) and the second targets 400, 410, 420 (support 1110) to calculate the offset of the first targets 300, 310, 320. The offset is then eliminated using the observed positions and relative positional relationship of the first targets 300, 310, 320. The correction amount estimation unit 140 uses the offset-eliminated positions of the first targets 300, 310, 320 to calculate the axial offset of the second vehicle perimeter monitoring sensor 10b and the corresponding correction amount.

[0085] Figure 7 3 is a diagram showing observation results of white lines and road markings by the vehicle periphery monitoring sensor 10 .

[0086] The first vehicle periphery monitoring sensor 10a (camera) observes white lines (lane outer lines, lane boundary lines, etc.) 1200 painted on the road surface as first objects 300, 310, and 320. Meanwhile, the second vehicle periphery monitoring sensor 10b (radar) detects a road sign 1210 as a second object 400.

[0087] The positions of white lines and road signs, which are road structures, are measured in advance and recorded in the object position relationship information storage unit 120. The position estimation unit 130 uses the relative positional relationship between the first objects 300, 310, and 320 (white lines 1210) and the second object 400 (road sign 1210) to calculate the offset of the first objects 300, 310, and 320. The position estimation unit 130 then uses the observed positions and relative positional relationship of the first objects 300, 310, and 320 to cancel the offset of the first objects 300, 310, and 320. The correction amount estimation unit 140 uses the offset-canceled positions of the first objects 300, 310, and 320 to calculate the axial offset of the second vehicle periphery monitoring sensor 10b and the corresponding correction amount.

[0088] Figure 8 3 is a diagram showing observation results of the white line and the soundproof wall by the vehicle periphery monitoring sensor 10 .

[0089] The first vehicle perimeter monitoring sensor 10a (camera) observes white lines (lane outer lines, lane boundary lines, etc.) 1300 painted on the road surface as first targets 300, 310, and 320. Meanwhile, the second vehicle perimeter monitoring sensor 10b (radar) detects a soundproof wall 1310 as second targets 400, 410, and 420.

[0090] The positions of white line 1300 (a road structure) and soundproof wall 1310 are pre-measured and recorded in target position relationship information storage unit 120. Position estimation unit 130 calculates offsets for first targets 300, 310, 320 (white line 1300) and second targets 400, 410, 420 (soundproof wall 1310). It then uses the observed positions and relative positional relationships of first targets 300, 310, 320 to cancel the offsets. Correction amount estimation unit 140 uses the offset-canceled positions of first targets 300, 310, 320 to calculate the axis offset of second vehicle perimeter monitoring sensor 10b and the corresponding correction amount.

[0091] Figure 9 1 and 2 are diagrams showing observation results of white lines and raised road markings (such as fender bars, cat's eye markings, and Botts' Dots) by the vehicle periphery monitoring sensor 10 .

[0092] The first vehicle perimeter monitoring sensor 10a (camera) observes white lines (lane outer lines, lane boundary lines, etc.) 1400 painted on the road surface as first targets 300, 310, and 320. Meanwhile, the second vehicle perimeter monitoring sensor 10b (radar) detects raised road signs 1410 as second targets 400, 410, and 420.

[0093] The positions of white lines and raised road signs, which serve as road structures, are pre-measured and stored in the object position relationship information storage unit 120. The position estimation unit 130 calculates the offset of the first objects 300, 310, and 320 using the relative positional relationship between the first objects 300, 310, and 320 (white lines 1400) and the second objects 400, 410, and 420 (raised road signs 1410). The position estimation unit 130 then uses the observed positions and relative positional relationship of the first objects 300, 310, and 320 to cancel the offset. The correction amount estimation unit 140 uses the offset-canceled positions of the first objects 300, 310, and 320 to calculate the axis offset of the second vehicle periphery monitoring sensor 10b and the corresponding correction amount.

[0094] <Example 2>

[0095] Figure 10 This is a functional block diagram showing another embodiment of the sensor fusion device 1 having a sensor alignment function. In the second embodiment, the differences from the first embodiment are mainly described, and the same components are denoted by the same reference numerals and their description is omitted.

[0096] like Figure 10As shown, the sensor fusion device 1 of this embodiment includes a factory shipping environment information storage unit 120a in place of the target position relationship information storage unit 120 in the configuration of the first embodiment. The factory shipping environment information storage unit 120a receives factory shipping environment information from the target relative position setting unit 30a as input and outputs it to the target position relationship processing unit 110 as needed. The factory shipping environment information storage unit 120a contains the position information of the first target 300 observed by the first vehicle periphery monitoring sensor 10a and the position information of the second target 400 observed by the second vehicle periphery monitoring sensor 10b, which are predefined for alignment within the factory or repair shop, as well as the category of the first target 300 and the category of the second target 400. To uniquely define the relative positional relationship between the vehicle 800 and the first target 300 and / or the second target 400, the information may also include the unified relative coordinates of the first target 300 and the unified relative coordinates of the second target 400. Furthermore, the position information may be absolute coordinates on a map.

[0097] In this embodiment, by having the factory shipping environment information storage unit 120a and setting the factory shipping mode to perform sensor alignment, the alignment implementer can use the predetermined first target 300 and second target 400 to construct the surrounding environment for sensor alignment and perform sensor alignment.

[0098] <Example 3>

[0099] Figure 11 This is a functional block diagram showing another embodiment of the sensor fusion device 1 having a sensor alignment function. In the third embodiment, the differences from the first embodiment are mainly described, and the same components are denoted by the same reference numerals and their description is omitted.

[0100] like Figure 11As shown, the sensor fusion device 1 of this embodiment includes a normal driving environment information storage unit 120b in place of the object position relationship information storage unit 120 in the configuration of the first embodiment. The normal driving environment information storage unit 120b receives normal driving environment information from the sensing information transmission unit 30b as input and outputs the normal driving environment information to the object position relationship processing unit 110 as needed. The normal driving environment information storage unit 120b contains the driving position of the vehicle 800 on the map stored in the sensing information transmission unit 30b, the types of road structures located around the vehicle, and the relative positions (e.g., arrangement patterns) of the road structures as specified by traffic regulations. In other words, the normal driving environment information storage unit 120b contains the absolute coordinates of the first object 300 observed by the first vehicle periphery monitoring sensor 10a, the category of the first object 300, the absolute coordinates of the second object 400, and the category of the second object 400. The normal driving environment information storage unit 120b may also contain the relative positional relationship between the first object 300 and the second object 400.

[0101] In this embodiment, by having a normal driving environment information storage unit 120b and setting it to the normal driving mode to perform sensor alignment, the sensor fusion device 1 can use the observation information (first target 300, second target 400) of the first vehicle surrounding monitoring sensor 10a and the second vehicle surrounding monitoring sensor 10b during normal driving to implement sensor alignment.

[0102] <Example 4>

[0103] Figure 12 This is a functional block diagram showing another embodiment of the sensor fusion device 1 having a sensor alignment function. In the fourth embodiment, the differences from the first embodiment are mainly described, and the same components are denoted by the same reference numerals and their description is omitted.

[0104] like Figure 12 As shown, the sensor fusion device 1 of this embodiment further includes a time series estimation unit 150 in addition to the configuration of the first embodiment. The time series estimation unit 150 receives the unified relative coordinates and absolute velocity of the first target 300 after offset elimination from the position estimation unit 130 as input, and uses the unified relative coordinates and absolute velocity of the first target 300 to track and accumulate the position of the first target 300 that changes with time, thereby determining that the accumulated data has a certain arrangement pattern, and using Figure 3 and Figure 4 By using the time series position data of the target, it is possible to align the sensors even if several frames are required for the recognition of road structures.

[0105] According to the fourth embodiment, alignment is performed using a plurality of observation points, so that alignment errors can be reduced.

[0106] In the embodiments described above, the first vehicle periphery monitoring sensor 10a and the second vehicle periphery monitoring sensor 10b may be of the same or different types. Furthermore, the vehicle periphery monitoring sensors 10a and 10b may be any sensor, such as millimeter-wave radar, cameras (visible light, near-infrared, mid-infrared, or far-infrared cameras), LiDAR (Light Detection and Ranging), sonar, Time of Flight (TOF) sensors, or combinations thereof.

[0107] Furthermore, the configurations of the functional blocks, processing flows, and actions described in the various embodiments may be arbitrarily combined.

[0108] Furthermore, in the above description, the sensor coordinate conversion correction value is calculated by the vehicle-mounted device (ECU), but the sensor coordinate conversion correction value may be calculated by a computer connected to the vehicle in a communicable manner.

[0109] As described above, the sensor fusion device 1 (sensor alignment device) of this embodiment comprises: a target position relationship processing unit 110, which outputs positional relationship information of the first target 300 and the second target 400; a sensor observation information processing unit 100, which transforms the observation results of the first target 300 and the observation results of the second target 400 into a prescribed unified coordinate system according to the coordinate transformation parameters, and synchronizes the time to a prescribed timing, extracting the first target information indicating the position of the first target 300 and the second target information indicating the position of the second target 400; a position inference unit 130, which uses the first target information , the second target information and the position relationship information are used to infer the position of the second target 400; and a correction amount inference unit 140, which uses the second target information and the inferred position of the second target to calculate the offset of the second sensor and infer the correction amount. The sensor fusion device 1 (sensor alignment device) changes the coordinate transformation parameters according to the correction amount, so the relative position relationship between the targets can be used to calculate the correction amount caused by the horizontal axis offset (angle θ1) of the second vehicle periphery monitoring sensor 10b, so that the positions of the target objects obtained by multiple vehicle periphery monitoring sensors 10 can be integrated into a unified relative coordinate.

[0110] Furthermore, the target position relationship processing unit 110 outputs predefined position relationship information in factory shipping mode. This allows accurate correction of sensor axial offset using a predefined target, enabling alignment regardless of the environment. Furthermore, simple targets, even those used in repair shops, can be used to correct sensor axial offset. Furthermore, targets with high recognition accuracy can be used to accurately correct sensor axial offset.

[0111] Furthermore, the position estimation unit 130 extracts the first target information and the second target information using the observation results of the first target 300 and the second target 400 during driving in the normal driving mode, so that the axis offset of the sensor can be corrected during normal driving without relying on maintenance.

[0112] In addition, there is a time series inference unit 150, which uses the position and absolute speed of the first target 300 in the unified coordinate system to accumulate the position of the first target 300 that changes with time, determines the accumulated position as a certain arrangement pattern, and uses the determined arrangement pattern to identify the first target, so that multiple observation points can be used to accurately correct the axis offset of the sensor.

[0113] Furthermore, the present invention encompasses various variations and equivalent configurations within the spirit of the appended claims and is not limited to the aforementioned embodiments. For example, the aforementioned embodiments are provided for the purpose of providing a detailed description of the present invention in an easily understandable manner, and the present invention is not necessarily limited to all of the configurations described. Furthermore, portions of the configurations of one embodiment may be replaced with configurations of another embodiment. Furthermore, configurations of another embodiment may be added to the configurations of one embodiment.

[0114] Furthermore, part of the configuration of each embodiment may be added, deleted, or replaced with another configuration.

[0115] In addition, the various structures, functions, processing units, processing methods, etc. described above can be partially or entirely implemented in hardware, for example, by designing using integrated circuits, or can be implemented in software by having a processor interpret and execute programs that implement various functions.

[0116] Information such as programs, tables, and files that implement various functions can be stored in storage devices such as memories, hard disks, and SSDs (Solid State Drives), or recording media such as IC cards, SD cards, DVDs, and BDs.

[0117] Furthermore, the control lines and information lines shown are those considered necessary for explanation, and not all control lines and information lines required for implementation are necessarily shown. In reality, it can be considered that almost all components are connected to each other.

[0118] Explanation of symbols

[0119] 1...Sensor fusion device, 2...Driving control device, 10a, 10b...Vehicle periphery monitoring sensors, 20...Own vehicle behavior detection sensor, 30a...Target relative position setting unit, 30b...Sensed information transmission unit, 100...Sensor observation information processing unit, 100a...Coordinate transformation unit, 100b...Time synchronization unit, 100c...Target determination unit, 110...Target position relationship processing unit, 120...Target position relationship information storage unit, 120a...Factory shipping environment information storage unit, 120b...Normal driving environment information storage unit, 130...Position estimation unit, 140...Correction amount estimation unit, 140a...Offset amount estimation unit, 140b...Correction amount calculation unit, 150...Time series estimation unit, 200...Data integration unit

Claims

1. A sensor alignment device, which receives as input an observation result of a first sensor on a first target and an observation result of a second sensor on a second target, characterized in that: have: an object position relationship processing unit that outputs positional relationship information between the first object and the second object; a sensor observation information processing unit that transforms the observation results of the first target and the observation results of the second target into a predetermined unified coordinate system based on coordinate transformation parameters, synchronizes the time with a predetermined timing, and extracts first target information indicating the position of the first target and second target information indicating the position of the second target; a position estimating unit that uses the first object information, the second object information, and the positional relationship information to eliminate an offset of the first object and estimates the position of the first object after the offset is eliminated as the position of the second object; as well as a correction amount estimation unit that uses the second target information and the estimated position of the second target, the unified relative coordinates of the installation position of the first sensor, and the unified relative coordinates of the installation position of the second sensor, and uses a calculation method corresponding to the type of the first target and the type of the second target to calculate the offset of the second sensor and estimate the correction amount. The sensor alignment device changes the coordinate transformation parameter according to the correction amount.

2. The sensor alignment device according to claim 1, wherein: The first object and the second object are components of one road structure.

3. The sensor alignment device according to claim 1, wherein: The first target and the second target are different objects.

4. The sensor alignment device according to claim 1, wherein: The target position relationship processing unit outputs predetermined position relationship information in a factory shipment mode.

5. The sensor alignment device according to claim 1, wherein: The position estimation unit extracts the first object information and the second object information using the observation result of the first object and the observation result of the second object during driving in the normal driving mode.

6. The sensor alignment device according to claim 1, wherein: A time series inference unit is provided, which uses the position and absolute speed of the first target in a unified coordinate system to accumulate the position of the first target that changes with time, determines the accumulated position as a certain arrangement pattern, and uses the determined arrangement pattern to identify the first target.

7. A driving control system for controlling the driving of a vehicle, characterized in that: have: A sensor fusion device that integrates and outputs observation results from two or more object sensors; and a driving control device that uses the output from the sensor fusion device to control the driving of the vehicle; The sensor fusion device receives as input an observation result of a first sensor on a first target and an observation result of a second sensor on a second target, and has: an object position relationship processing unit that outputs positional relationship information between the first object and the second object; a sensor observation information processing unit that transforms the observation results of the first target and the observation results of the second target into a predetermined unified coordinate system based on coordinate transformation parameters, synchronizes the time with a predetermined timing, and extracts first target information indicating the position of the first target and second target information indicating the position of the second target; a position estimating unit that uses the first object information, the second object information, and the positional relationship information to eliminate an offset of the first object and estimates the position of the first object after the offset is eliminated as the position of the second object; as well as a correction amount estimation unit that uses the second target information and the estimated position of the second target, the unified relative coordinates of the installation position of the first sensor, and the unified relative coordinates of the installation position of the second sensor, and uses a calculation method corresponding to the type of the first target and the type of the second target to calculate the offset of the second sensor and estimate the correction amount. The sensor fusion device changes the coordinate transformation parameter according to the correction amount.

8. A method for estimating a correction amount of sensor data performed by a sensor alignment device that uses observation results of two or more sensors to calculate correction parameters of the observation results of the sensors, characterized in that: The inference method has the following features: An input step of inputting an observation result of the first sensor on the first target and an observation result of the second sensor on the second target; an object position relationship processing step of outputting position relationship information of the first object and the second object; a sensor observation information processing step of transforming the observation results of the first target and the observation results of the second target into a predetermined unified coordinate system according to coordinate transformation parameters, synchronizing the time to a predetermined timing, and extracting first target information indicating the position of the first target and second target information indicating the position of the second target; a position estimation step of eliminating an offset of the first target using the first target information, the second target information, and the position relationship information, and estimating the position of the first target after the offset is eliminated as the position of the second target; as well as a correction amount estimating step of calculating an offset of the second sensor and estimating a correction amount using the second target information and the estimated position of the second target, the unified relative coordinates of the installation position of the first sensor, and the unified relative coordinates of the installation position of the second sensor, and using a calculation method corresponding to the category of the first target and the category of the second target; The method further includes the step of changing the coordinate transformation parameters according to the correction amount.

Images