Processing system, processing device, processing method, and processing program

Abstract

To provide a processing system for securing the accuracy of calibration between a plurality of optical sensors.SOLUTION: A processor of a processing device is configured so as to execute: acquiring point group data (Dp) that represents scanning point groups for optical sensors; extracting road surface point groups (Ps) produced by scanning a traveling road surface in the point group data (Dp) for each of the optical sensors; monitoring an inclination angle (θi) by which an approximating plane that approximates the road surface point group for each of the optical sensors is inclined with respect to a horizontal reference plane defined so as to align with a left-right direction and a front-rear direction of a host vehicle; searching for a reference sensor for which a variation index (Iθ) that increases in accordance with angular variation (δθ) occurring in the inclination angle (θi) is smallest; and outputting calibration data (Dc) for calibrating scanning positions of scanning point groups in the point group data (Dp) for the optical sensors other than the reference sensor, by using as reference the scanning position of the scanning point group in the point group data (Dp) for the reference sensor.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to processing techniques for multiple optical sensors mounted on a moving object that scan the outside world. [Background technology]

[0002] Patent Document 1 discloses a technology that uses LiDAR, an optical sensor, as a plurality of sensors mounted on a vehicle as a moving body. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] US Patent No. 10099630 Summary of the Invention [Problem to be solved by the invention]

[0004] However, deviations occur in the scanning positions of each LiDAR that scans the external environment from the vehicle due to, for example, vehicle vibrations, aging, etc. Therefore, the accuracy of calibrating the scanning positions between each LiDAR is important.

[0005] An object of the present disclosure is to provide a processing system that ensures the accuracy of calibration between multiple optical sensors. Another object of the present disclosure is to provide a processing device that ensures the accuracy of calibration between multiple optical sensors. Yet another object of the present disclosure is to provide a processing method that ensures the accuracy of calibration between multiple optical sensors. Yet another object of the present disclosure is to provide a processing program that ensures the accuracy of calibration between multiple optical sensors. [Means for solving the problem]

[0006] The technical means of the present disclosure for solving the problems will be described below. Note that the claims and the reference characters in parentheses in this section indicate the correspondence with the specific means described in the embodiments described later in detail, and do not limit the technical scope of the present disclosure.

[0007] A first aspect of the present disclosure is 1. A processing system for calibrating scan positions between a plurality of optical sensors (40) mounted on a host vehicle (2) that scan an external world, the processing system having a processor (12), comprising: The processor Acquiring point cloud data (Dp) representing the scanned point clouds by each optical sensor; Extracting a scanning point cloud obtained by scanning a road surface (3) on which the host mobile body is traveling as a road surface point cloud (Ps) from the point cloud data of each optical sensor; Monitoring the inclination angles (θi, θip, θir) of an approximate plane (30) approximating the road surface point cloud for each optical sensor with respect to a reference horizontal plane (20) defined along the left-right direction and the front-rear direction of the host moving body; Among the optical sensors, an optical sensor that minimizes a fluctuation index (Iθ) that increases in accordance with a fluctuation angle (δθ) generated in the tilt angle is searched for as a reference sensor (40a); The optical sensor is configured to output calibration data (Dc) that calibrates the scanning positions of the scanning point clouds in the point cloud data of the optical sensors other than the reference sensor based on the scanning positions of the scanning point clouds in the point cloud data of the reference sensor.

[0008] A second aspect of the present disclosure is A processing device having a processor (12), configured to be mountable on a host mobile body (2), for calibrating scanning positions between a plurality of optical sensors (40) that scan an external environment of the host mobile body, the processing device comprising: The processor Acquiring point cloud data (Dp) representing the scanned point clouds by each optical sensor; Extracting a scanning point cloud obtained by scanning a road surface (3) on which the host mobile body is traveling as a road surface point cloud (Ps) from the point cloud data of each optical sensor; Monitoring the inclination angles (θi, θip, θir) of an approximate plane (30) approximating the road surface point cloud for each optical sensor with respect to a reference horizontal plane (20) defined along the left-right direction and the front-rear direction of the host moving body; Among the optical sensors, an optical sensor that minimizes a fluctuation index (Iθ) that increases in accordance with a fluctuation angle (δθ) generated in the tilt angle is searched for as a reference sensor (40a); The optical sensor is configured to output calibration data (Dc) that calibrates the scanning positions of the scanning point clouds in the point cloud data of the optical sensors other than the reference sensor based on the scanning positions of the scanning point clouds in the point cloud data of the reference sensor.

[0009] A third aspect of the present disclosure is A processing method executed by a processor (12) for calibrating scanning positions between a plurality of optical sensors (40) mounted on a host vehicle (2) and scanning an external environment, comprising: Acquiring point cloud data (Dp) representing the scanned point clouds by each optical sensor; Extracting a scanning point cloud obtained by scanning a road surface (3) on which the host mobile body is traveling as a road surface point cloud (Ps) from the point cloud data of each optical sensor; Monitoring the inclination angles (θi, θip, θir) of an approximate plane (30) approximating the road surface point cloud for each optical sensor with respect to a reference horizontal plane (20) defined along the left-right direction and the front-rear direction of the host moving body; Among the optical sensors, an optical sensor that minimizes a fluctuation index (Iθ) that increases in accordance with a fluctuation angle (δθ) generated in the tilt angle is searched for as a reference sensor (40a); This includes outputting calibration data (Dc) that calibrates the scanning positions of the scanning point clouds in the point cloud data of other optical sensors other than the reference sensor among the optical sensors, based on the scanning positions of the scanning point clouds in the point cloud data of the reference sensor.

[0010] A fourth aspect of the present disclosure is A processing program stored in a storage medium (10) for calibrating scanning positions among a plurality of optical sensors (40) mounted on a host moving body (2) and scanning an external environment, the processing program including instructions for causing a processor (12) to execute the calibration, the processing program comprising: Acquiring point cloud data (Dp) representing the scanned point clouds by each optical sensor; Extracting a scanning point cloud obtained by scanning a road surface (3) on which the host mobile body is traveling as a road surface point cloud (Ps) from the point cloud data of each optical sensor; Monitoring the inclination angles (θi, θip, θir) of an approximate plane (30) approximating the road surface point cloud for each optical sensor with respect to a reference horizontal plane (20) defined along the left-right direction and the front-rear direction of the host moving body; Among the optical sensors, an optical sensor that minimizes a fluctuation index (Iθ) that increases in accordance with a fluctuation angle (δθ) generated in the tilt angle is searched for as a reference sensor (40a); The program includes an instruction to execute the following: outputting calibration data (Dc) that calibrates the scanning positions of the scanning point clouds in the point cloud data of the optical sensors other than the reference sensor among the optical sensors, based on the scanning positions of the scanning point clouds in the point cloud data of the reference sensor.

[0011] As described above, in the first to fourth aspects, in the point cloud data acquired by each optical sensor, a scanning point cloud obtained by scanning the road surface on which the host mobile body is traveling is extracted as a road surface point cloud. Therefore, according to the first to fourth aspects, the inclination angle of the approximation plane approximating the road surface point cloud for each optical sensor is monitored with respect to a reference horizontal plane defined along the left-right and front-rear directions of the host mobile body. As a result, the fluctuation index that increases in accordance with the fluctuation angle, which occurs when the approximation plane of the road surface is at an inclination angle that should ideally maintain a constant angle with respect to the reference horizontal plane, becomes smaller for optical sensors with smaller deviations in the scanning positions of the scanning point cloud in the point cloud data. Therefore, by calibrating the scanning positions of the scanning point clouds of other optical sensors based on the scanning position with the smallest deviation of the scanning point cloud by the reference sensor found to have the smallest fluctuation index that increases in accordance with the fluctuation angle of the inclination angle, it is possible to output calibration data with ensured accuracy. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing the overall configuration of a first embodiment. [Figure 2] FIG. 1 is a plan view showing a traveling environment of a host vehicle to which a first embodiment is applied. [Figure 3] 1 is a block diagram showing a functional configuration of a processing system according to a first embodiment. [Figure 4] 3 is a flowchart showing a processing flow according to the first embodiment. [Figure 5] FIG. 2 is a schematic diagram for explaining a processing flow according to the first embodiment. [Figure 6] FIG. 2 is a schematic diagram for explaining a processing flow according to the first embodiment. [Figure 7] FIG. 2 is a schematic diagram for explaining a processing flow according to the first embodiment. [Figure 8] FIG. 2 is a schematic diagram for explaining a processing flow according to the first embodiment. [Figure 9] FIG. 2 is a schematic diagram for explaining a processing flow according to the first embodiment. [Figure 10]FIG. 2 is a schematic diagram for explaining a processing flow according to the first embodiment. [Figure 11] FIG. 2 is a schematic diagram for explaining a processing flow according to the first embodiment. [Figure 12] 10 is a flowchart showing a processing flow according to a second embodiment. [Figure 13] 10 is a flowchart showing a processing flow according to a third embodiment. [Figure 14] FIG. 10 is a schematic diagram for explaining a processing flow according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, multiple embodiments of the present disclosure will be described with reference to the drawings. Note that corresponding components in each embodiment are designated by the same reference numerals, and redundant description may be omitted. Furthermore, when only a portion of the configuration is described in each embodiment, the configuration of another previously described embodiment may be applied to the remaining portions of the configuration. Furthermore, in addition to the combinations of configurations explicitly stated in the description of each embodiment, configurations of multiple embodiments may be partially combined together even if not explicitly stated, provided that there is no particular problem with the combination.

[0014] The processing system 1 of the first embodiment shown in FIG. 1 generates calibration data Dc to calibrate the scanning positions among a plurality of optical sensors 40 mounted on a host vehicle 2 (see FIG. 2) as a host moving body and scanning the external world. Here, the host vehicle 2 can be said to be an ego-vehicle from a viewpoint centered on the host vehicle 2. The host vehicle 2 is a moving body, such as an automobile, capable of traveling on a traveling road surface 3 with an occupant on board. Therefore, directions in the following description are defined based on the host vehicle 2 on an ideal traveling road surface 3 along a horizontal plane.

[0015] The host vehicle 2 is provided with an autonomous driving mode that is divided into levels according to the degree of manual intervention by the occupant in the dynamic driving task. The autonomous driving mode may be realized by autonomous driving control, such as conditional driving automation, high driving automation, or full driving automation, in which the system performs all dynamic driving tasks when activated. The autonomous driving mode may also be realized by advanced driving assistance control, such as driving assistance or partial driving automation, in which the occupant performs some or all of the dynamic driving tasks. The autonomous driving mode may be realized by either autonomous driving control or advanced driving assistance control, or by a combination of these, or by switching between them.

[0016] A pair of optical sensors 40 shown in FIG. 2 of the first embodiment are provided on the host vehicle 2 so as to scan the area ahead of the host vehicle 2 in the external environment. Each optical sensor 40 is a so-called LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) that acquires optical information that can be used in the autonomous driving mode of the host vehicle 2. The sensing areas Rs of the optical sensors 40 partially overlap each other to form an overlap area Rsc (the dot-hatched area in FIG. 2). As shown in FIGS. 1 and 3, each optical sensor 40 has a light-emitting unit 400, a scanning unit 410, and a light-receiving unit 420.

[0017] The light-emitting unit 400 is mainly composed of a light-emitting element, such as a laser diode, that emits directional laser light in the infrared range. The light-emitting unit 400 projects light toward the outside world of the host vehicle 2 in the form of an intermittent pulse beam. The scanning unit 410 is mainly composed of a scanning mirror. The scanning unit 410 reflects the light emitted from the light-emitting unit 400 according to the rotation angle of the scanning mirror, thereby scanning the outside world of the host vehicle 2 with the light. The light-receiving unit 420 is composed of a light-receiving element, such as a SPAD (Single Photon Avalanche Diode), that is highly sensitive to the light emitted, combined with an integrated circuit. The light-receiving unit 420 receives return light, which is a reflection of the light emitted from an object point in the outside world of the host vehicle 2, after it has been reflected by the scanning unit 410. The light receiving unit 420 outputs point cloud data Dp representing a group of scanning points, i.e., a scanning point cloud, each given information on the scanning positions, based on the light receiving signals generated for each light receiving pixel of the light receiving element in accordance with the scanning positions of the object points that reflect the irradiated light in the external world of the host vehicle 2.

[0018] 1, the processing system 1 is configured to include at least one dedicated computer. The processing system 1 is connected to a sensor system 4 via at least one of, for example, a LAN (Local Area Network) line, a wire harness, an internal bus, or a wireless communication line. When the processing system 1 is configured with multiple dedicated computers, the connections between these dedicated computers are similar.

[0019] The dedicated computer constituting the processing system 1 may be a driving control ECU (Electronic Control Unit) that controls the driving of the host vehicle 2. The dedicated computer constituting the processing system 1 may be a navigation ECU that navigates the driving route of the host vehicle 2. The dedicated computer constituting the processing system 1 may be a locator ECU that estimates the self-state quantity of the host vehicle 2. The dedicated computer constituting the processing system 1 may be an actuator ECU that controls the driving actuator of the host vehicle 2. The dedicated computer constituting the processing system 1 may be an HCU (Human Machine Interface (HMI) Control Unit) that controls the presentation of information in the host vehicle 2. The dedicated computer constituting the processing system 1 may be a computer other than the host vehicle 2 that constitutes an external center or mobile terminal that can communicate via the communication system of the host vehicle 2.

[0020] The dedicated computer constituting the processing system 1 has at least one memory 10 and one processor 12. The memory 10 is at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores computer-readable programs, data, and the like. Here, "storage" may refer to accumulation in which data is retained even when the host vehicle 2 is powered off, or may refer to temporary storage in which data is erased when the host vehicle 2 is powered off. The processor 12 includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), a reduced instruction set computer (RISC)-CPU, a data flow processor (DFP), or a graph streaming processor (GSP).

[0021] In the processing system 1, the processor 12 executes a plurality of instructions included in a processing program stored in the memory 10 to generate calibration data Dc for calibrating the scanning position between each optical sensor 40 that scans the external environment of the host vehicle 2. As a result, the processing system 1 constructs a plurality of function blocks for generating the calibration data Dc for calibrating the scanning position between each optical sensor 40. The plurality of function blocks constructed in the processing system 1 include an acquisition block 100, an extraction block 110, a monitoring block 120, a search block 130, and an output block 140, as shown in FIG.

[0022] The processing method for generating calibration data Dc, which is used by the processing system 1 to calibrate the scanning positions between the optical sensors 40, is executed in accordance with the processing flow shown in Fig. 4 by cooperation of these blocks 100, 110, 120, 130, and 140. This processing flow is repeatedly executed during the autonomous driving mode by the driving control ECU of the host vehicle 2. Note that each "S" in this processing flow represents a step executed by multiple commands included in the processing program.

[0023] In S100, the acquisition block 100 acquires point cloud data Dp representing a cloud of scanning points scanned by each optical sensor 40, as shown in Fig. 5. At this time, it is preferable that the acquisition of the point cloud data Dp between each optical sensor 40 is substantially synchronized.

[0024] In S110 following S100 in the processing flow shown in FIG. 4, the extraction block 110 determines the amount of deviation ΔP between the scanning positions of the scanning point clouds in the point cloud data Dp of each optical sensor 40. Specifically, first, the scanning point clouds within the overlap region Rsc in the point cloud data Dp of each optical sensor 40 are compared with each other. As a result, pairs of scanning points in the point cloud data Dp of each optical sensor 40 with the smallest mutual distance are matched with each other, for example, by nearest neighbor search processing or approximate neighbor search processing, and the sum of the mutual distances representing the distribution feature is calculated as the amount of deviation ΔP. At this time, the point cloud data Dp of each optical sensor 40 is calibrated with respect to each other using the most recent data Dc from the calibration data Dc output when S170 (described later) was executed in a previous processing flow, and then the amount of deviation ΔP is calculated.

[0025] Therefore, in S110, it is determined whether the calculated deviation amount ΔP has increased beyond the allowable range. At this time, the deviation amount ΔP is determined based on the allowable range up to the allowable upper limit of deviation, which is the range within which calibration is not required between the scanning positions of the scanning point clouds in the point cloud data Dp of each optical sensor 40. In other words, increasing beyond the allowable range means that the deviation amount ΔP has increased to a size that requires calibration between the scanning positions of the scanning point clouds in the point cloud data Dp of each optical sensor 40.

[0026] If a negative determination is made in S110, the current execution of the processing flow ends. On the other hand, if a positive determination is made in S110, the processing flow proceeds to S120. In S120, the extraction block 110 generates a control command to the driving control ECU that controls the autonomous driving mode of the host vehicle 2, thereby guiding the host vehicle 2 toward the driving road surface 3 where the calibration condition Cc is satisfied. This guidance may be automatic guidance in the autonomous driving mode, or manual driving operation guidance, for example, by display and / or audio, to an occupant who has canceled the autonomous driving mode and has been transferred driving authority.

[0027] The calibration condition Cc being met in S120 means that at least one of the flatness condition Ccp and the horizontality condition Cch, which are predetermined, is met. Specifically, the flatness condition Ccp is met when the flatness of the traveling road surface 3 falls within an allowable flatness range up to an allowable upper flatness limit. Here, flatness refers to the maximum difference in unevenness height within a set area of ​​the traveling road surface 3. Therefore, the allowable flatness range is set to a range that can ensure a required level of approximation accuracy between the approximated plane 30 (described below) and the traveling road surface 3. Furthermore, in the first embodiment, flatness may be recognized based on at least one of, for example, sensing data acquired by a sensor of the host vehicle 2 and communication data acquired via a communication unit of the host vehicle 2.

[0028] In contrast, the horizontality condition Cch is met when the horizontality of the travel road surface 3 is within an allowable horizontal range up to an allowable horizontal upper limit value. Here, horizontality refers to the inclination height of the travel road surface 3 per set distance, i.e., the gradient. Therefore, the horizontally allowable range of horizontality is set to a range that can ensure a required level of monitoring accuracy of the inclination angle θi between the approximate plane 30 and the reference horizontal plane 20, which will be described later. Furthermore, in the first embodiment, the horizontality may be recognized based on at least one of, for example, sensing data acquired by a sensor of the host vehicle 2 and communication data acquired via a communication unit of the host vehicle 2.

[0029] In S130 following S120 in the processing flow, the extraction block 110 determines whether the calibration condition Cc is met. As long as a negative determination is made in S130, the processing flow returns to S100, and S100, S110, S120, and S130 are repeated. On the other hand, if a positive determination is made in S130, the processing flow proceeds to S140.

[0030] In S140, the extraction block 110 extracts, from the point cloud data Dp for each optical sensor 40, a scanning point cloud obtained by scanning the traveling road surface 3 on which the host vehicle 2 is traveling, as a road surface point cloud Ps as shown in Fig. 6. Specifically, by performing a filtering process such as cross simulation on the point cloud data Dp for each optical sensor 40, the road surface point cloud Ps from the traveling road surface 3 that satisfies the flatness and horizontality of S120 is extracted.

[0031] In S150 following S140 in the processing flow shown in FIG. 4, the monitoring block 120 monitors the tilt angle θi of the approximation plane 30 (FIG. 8) that approximates the road surface point cloud Ps for each optical sensor 40, relative to the reference horizontal plane 20 (FIG. 7) of the host vehicle 2 on the traveling road surface 3. At this time, the reference horizontal plane 20 is defined along the pitch axis Ap in the left-right direction and the roll axis Ar in the front-to-rear direction of the host vehicle 2, as shown in FIG. 7, and is therefore assumed to extend left-right, front-to-rear, and rearward of the host vehicle 2. Therefore, as the tilt angle θi, a two-dimensional tilt angle θip about the left-to-right axis Op along the pitch axis Ap in the left-to-right direction, as shown in FIG. 9, and another two-dimensional tilt angle θir about the front-to-rear axis Or along the roll axis Ar in the front-to-rear direction, as shown in FIG. 10, are monitored individually.

[0032] The approximate plane 30 on which the tilt angle θi is monitored is derived as shown in FIG. 8 by fitting processing such as the least squares method or principal component analysis, using as input the Cartesian coordinates x, y, and z in the following equation (1), which represent the scanning position of each scanning point constituting the road surface point cloud Ps. In this case, the approximate plane 30 becomes a virtual plane that approximately imagines the traveling road surface 3 by following equation (1) using coefficients a, b, c, and d. Therefore, for the reference horizontal plane 20, an Cartesian coordinate system 200 defined by X-, Y-, and Z-axes along the left-right, front-rear, and height directions of the host vehicle 2, respectively, is assumed, as shown in FIGS. 8 to 10. The X- and Y-axes of such Cartesian coordinate system 200 substantially coincide with the left-right pitch axis Ap and the front-rear roll axis Ar, respectively.

number

[0033] 9 in the Cartesian coordinate system 200 is monitored as the angle formed around the left-right axis Op by the normal vector N expressed with respect to the reference horizontal plane 20 by the following equation 2 and the normal vector n expressed with respect to the approximate plane 30 by the following equation 3. At the same time, the tilt angle θir in FIG. 10 in the Cartesian coordinate system 200 is monitored as the angle formed around the front-rear axis Or by the normal vector N expressed with respect to the reference horizontal plane 20 by the following equation 2 and the normal vector n expressed with respect to the approximate plane 30 by the following equation 3. As described above, in S150 after the affirmative determination in S130 due to the establishment of the calibration condition Cc, monitoring of the tilt of the approximate plane 30 with respect to the reference horizontal plane 20 is performed.

number

number

[0034] In S160 following S150 in the processing flow shown in FIG. 4, the search block 130 searches for the optical sensor 40 that has the smallest fluctuation index Iθ correlated with the fluctuation angle δθ of the inclination angle θi for each optical sensor 40, as the reference sensor 40a. At this time, the fluctuation angle δθ for each optical sensor 40 is defined as the deviation between the inclination angle θi extracted by the immediately preceding monitoring at S150 and a reference angle that serves as the reference for the inclination angle θi. Here, the reference angle for the inclination angle θi may be an initial value measured when each optical sensor 40 is installed on the host vehicle 2. The reference angle for the inclination angle θi may also be the previous inclination angle θi extracted by the monitoring at S150 in the previous processing flow.

[0035] In the first embodiment in particular, the fluctuation angle δθ is defined as the deviation of the tilt angles θip and θir from the corresponding reference angles, as two-dimensional fluctuation angles δθp and δθr that constitute the following equation (4). The fluctuation index Iθ is calculated for each optical sensor 40 according to the following equation (4) so ​​as to correlate with the sum of the squares of the fluctuation angle δθp of the tilt angle θip and the fluctuation angle δθr of the tilt angle θir and gradually increase in response to an increase in at least one of these fluctuation angles δθp and δθr. As a result, one optical sensor 40 whose calculated fluctuation index Iθ is the minimum, as shown in FIG. 11, is selected as the reference sensor 40a. At this time, an optical sensor 40 other than the reference sensor 40a, i.e., the other optical sensor 40 whose fluctuation index Iθ is not the minimum, is selected as the target sensor 40b to be calibrated, as described below.

number

[0036] 4, in S170 following S160, the output block 140 calibrates the scanning positions of the scanning point cloud in the point cloud data Dp of the target sensor 40b based on the scanning positions of the scanning point cloud in the point cloud data Dp of the reference sensor 40a. At this time, the pair matched in the immediately preceding S110 in the currently executed processing flow is applied to the pair of scanning positions to be matched one-to-one between the reference side and the calibration side. Therefore, in S170, an optimization calculation is performed to minimize the deviation amount ΔP calculated in accordance with S110 by shifting the scanning positions of each scanning point in the point cloud data Dp of the target sensor 40b toward the scanning position of the matching scanning point in the point cloud data Dp of the reference sensor 40a.

[0037] In this optimization calculation, coordinate transformation parameters are calculated to transform coordinates so as to shift the scanning position of each scanning point in the point cloud data Dp of the target sensor 40b toward the scanning position of the matching scanning point in the point cloud data Dp of the reference sensor 40a. Therefore, in S170, the calculated transformation parameters are output as calibration data Dc. At this time, in S170, which occurs after a positive determination in S110 due to the deviation amount ΔP increasing beyond the allowable range, the output of the calibration data Dc is realized by storing the data in at least memory 10.

[0038] The calibration data Dc output in this manner at S170 is used to generate point cloud data Dp in which the scanning position of the calibration target sensor 40b is calibrated based on the scanning position of the reference sensor 40a. Therefore, the output of the calibration data Dc at S170 may be at least one of the following, in addition to the above-mentioned data storage: providing data to a driving control ECU in the host vehicle 2; and transmitting data to an external center via a communication unit of the host vehicle 2. As described above, the completion of the output at S170 ends the current execution of the processing flow.

[0039] (Action and effect) The effects of the first embodiment described above will be explained below.

[0040] In the first embodiment, in the point cloud data Dp acquired separately for each optical sensor 40, a scanning point cloud obtained by scanning the travel road surface 3 on which the host vehicle 2 travels is extracted as the road surface point cloud Ps. Therefore, according to the first embodiment, the inclination angle θi of the approximate plane 30 that approximates the road surface point cloud Ps for each optical sensor 40 is monitored with respect to the reference horizontal plane 20 defined along the left-right direction and the front-rear direction of the host vehicle 2. As a result, the fluctuation index Iθ, which increases following the fluctuation angle δθ that occurs at the inclination angle θi of the approximate plane 30 of the travel road surface 3 with respect to the reference horizontal plane 20, which should ideally maintain a constant angle, becomes smaller for an optical sensor 40 with less deviation in the scanning position of the scanning point cloud in the point cloud data Dp. Therefore, by calibrating the scanning positions of the scanning point clouds by other optical sensors 40 (i.e., target sensors 40b) based on the scanning position of the reference sensor 40a that has been searched for to have the smallest variation index Iθ, which increases in accordance with the variation angle δθ of the tilt angle θi, it becomes possible to output calibration data Dc with ensured accuracy.

[0041] According to the first embodiment, when the flatness of the traveling road surface 3 falls within the plane tolerance range up to the allowable upper plane limit value, the inclination angle θi of the approximation plane 30, which can improve the approximation accuracy with the traveling road surface 3, relative to the reference horizontal plane 20 is monitored. Therefore, it is possible to accurately search for the reference sensor 40a with the minimum fluctuation index Iθ, which increases in accordance with the fluctuation angle δθ of the inclination angle θi, and ensure the accuracy of the calibration using the calibration data Dc.

[0042] According to the first embodiment, when the horizontality of the traveling road surface 3 is within an allowable horizontal range up to an allowable upper horizontal limit value, the tilt angle θi of the approximate plane 30 that approximates the road surface point group Ps of the traveling road surface 3 is monitored with respect to the reference horizontal plane 20 of the host vehicle 2, whose posture on the traveling road surface 3 can be stable. Therefore, it is possible to accurately search for the reference sensor 40a with the smallest fluctuation index Iθ, which increases in accordance with the fluctuation angle δθ of the tilt angle θi, and ensure the accuracy of the calibration using the calibration data Dc.

[0043] According to the first embodiment, when the deviation amount ΔP between the scanning positions of the scanning point clouds in the point cloud data Dp of each optical sensor 40 increases beyond the allowable deviation range, the driving positions at which calibration is performed between the reference sensor 40a and another optical sensor 40 (i.e., the target sensor 40b) can be accurately matched. This makes it possible to ensure the accuracy of calibration using the calibration data Dc. Particularly in the first embodiment, when the deviation amount ΔP between the scanning positions of the scanning point clouds in the point cloud data Dp of each optical sensor 40 acquired during the autonomous driving mode of the host vehicle 2 increases beyond the allowable deviation range, accurate matching of the scanning positions can be achieved. This makes it possible to ensure the calibration accuracy required for the autonomous driving mode.

[0044] According to the first embodiment, the inclination angle θi of the approximate plane 30 of the road surface 3 relative to the reference horizontal plane 20 along the left-right and front-rear directions of the host vehicle 2 is determined by focusing on the inclination angle θip about the left-right axis Op in the left-right direction and the inclination angle θir about the front-rear axis Or in the front-rear direction. Therefore, the fluctuation index Iθ, which correlates with the sum of the squares of the fluctuation angles δθ occurring in these two-dimensional inclination angles θip and θir, makes it possible to accurately search for the reference sensor 40a with the smallest scanning position deviation of the scanning point cloud. This makes it possible to ensure the accuracy of calibration using the calibration data Dc.

[0045] Second Embodiment The second embodiment is a modification of the first embodiment. In the processing flow of the second embodiment shown in Fig. 12, S120 is omitted, and accordingly, S130 is executed after S140 is executed.

[0046] However, if the calibration condition Cc in S130 of the second embodiment is the flatness condition Ccp, then the satisfaction of the condition Ccp is determined by recognizing the flatness of the traveled road surface 3 based on the distance between the scanning position and the approximate plane 30 in the road surface point cloud Ps extracted immediately before in S130 as the scanning point cloud by at least one optical sensor 40. Similarly, if the calibration condition Cc in S130 of the second embodiment is the horizontality condition Cch, then the satisfaction of the condition Cch is determined by recognizing the horizontality of the approximate plane 30 that approximates the scanning position in the road surface point cloud Ps extracted immediately before in S130 as the scanning point cloud by at least one optical sensor 40. Note that if the calibration condition Cc is both the flatness condition Ccp and the horizontality condition Cch, then either one of the conditions Ccp or Cch may be recognized in accordance with the first embodiment.

[0047] As long as a negative determination is made in S130, the process returns to S100, and S100, S110, S140, and S130 are repeated. On the other hand, if a positive determination is made in S130, the process proceeds to S150.

[0048] In S150 of the second embodiment, when the flatness based on the distance between the scanning position of the road surface point cloud Ps by at least one optical sensor 40 and the approximate plane 30 falls within the plane tolerance range, the inclination angle θi of the approximate plane 30 with respect to the reference horizontal plane 20 is monitored. This allows the road surface point cloud Ps required for monitoring the inclination angle θi to also be effectively used to determine whether or not such monitoring is necessary based on the flatness, thereby reducing the processing load for ensuring calibration accuracy.

[0049] Furthermore, in S150 of the second embodiment, when the horizontality of the approximate plane 30, which approximates the scanning position of the road surface point cloud Ps by at least one optical sensor 40, is within the horizontal tolerance range, the inclination angle θi of the approximate plane 30 with respect to the reference horizontal plane 20 is monitored. This allows the road surface point cloud Ps required for monitoring the inclination angle θi to be effectively used for determining whether or not such monitoring is necessary based on the horizontality, thereby reducing the processing load for ensuring calibration accuracy.

[0050] (Third embodiment) The third embodiment is a modification of the first embodiment. As shown in Fig. 13, in the processing flow of the third embodiment, steps S3150 and S3160 are executed instead of steps S150 and S160, respectively.

[0051] Specifically, the monitoring block 120 in S3150 monitors the three-dimensional tilt angle θi, which is a combination of the tilt about the left-right axis Op in the left-right direction and the tilt about the front-rear axis Or in the front-rear direction, with respect to the assumed reference horizontal plane 20 of the Cartesian coordinate system 200, as shown in Fig. 14. At this time, the monitoring block 120 monitors the tilt angle θi that satisfies the following equation (5), which is the angle between the normal vectors N and n, which are expressed for each surface 20 and 30 by the same equations (2) and (3) as in the first embodiment, on a virtual plane containing these vectors N and n. Therefore, the search block 130 in S3160 searches for the reference sensor 40a by comparing, for each optical sensor 40, the fluctuation index Iθ that matches the fluctuation angle δθ generated in the three-dimensional tilt angle θi, according to the following equation (6), i.e., the fluctuation index Iθ whose correlation coefficient, which increases in accordance with the fluctuation angle δθ, is substantially 1.

number

number

[0052] According to the third embodiment, the angle of inclination of the approximation plane 30 of the road surface 3 relative to the reference horizontal plane 20 of the host vehicle 2 is focused on as the three-dimensional inclination angle θi, which is a combination of the inclination about the left-right axis Op along the left-right direction and the inclination about the front-rear axis Or along the front-rear direction. Therefore, using the fluctuation index Iθ that coincides with the fluctuation angle δθ generated in the three-dimensional inclination angle θi about these axes Op and Or, it is possible to accurately search for the reference sensor 40a with the minimum scanning position deviation of the scanning point cloud. Therefore, it is possible to ensure the accuracy of calibration using the calibration data Dc.

[0053] (Other embodiments) Although multiple embodiments have been described above, the present disclosure should not be construed as being limited to those embodiments, and can be applied to various embodiments and combinations within the scope that does not deviate from the gist of the present disclosure.

[0054] In a modified example, the dedicated computer constituting the processing system 1 may have at least one of a digital circuit and an analog circuit as a processor. Here, the digital circuit is at least one of the following: an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a system on a chip (SOC), a programmable gate array (PGA), and a complex programmable logic device (CPLD). Such a digital circuit may also have a memory that stores a program.

[0055] In the processing flow of the modified example, S3150 and S3160 of the third embodiment may be executed instead of S150 and S160 of the second embodiment, respectively. In the processing flow of the modified example, S110 of the first to third embodiments may be omitted, and matching for calculating the deviation amount ΔP may be performed as in S110 in the subsequent S170 of the first to third embodiments. In the processing flow of the modified example, S120 of the first and third embodiments may be omitted. In the processing flow of the modified example, S130 of the second embodiment may be omitted.

[0056] In a modified example, three or more optical sensors 40 may be mounted on the host vehicle 2 in a state in which all of them form a common overlap region Rsc. In the processing flow in this case, in S160 and S3160 of the first to third embodiments, one reference sensor 40a may be selected, and the remaining optical sensors 40 may be selected as target sensors 40b.

[0057] In a modified example, three or more optical sensors 40 may be mounted on the host vehicle 2 in a state in which the overlapping area Rsc is formed so that at least two of the optical sensors 40 share the overlapping area Rsc. In the processing flow in this case, in S160 and 3160 of the first to third embodiments, one reference sensor 40a may be selected from the optical sensors 40 that share the overlapping area Rsc, and the remaining optical sensors 40 may be selected as target sensors 40b.

[0058] In a modified example, the host mobile body to which the processing system 1 is applied may be, for example, an autonomous robot capable of autonomously or remotely traveling to transport luggage or collect information, etc. In addition to the forms described so far, the above-described embodiments and modified examples may be implemented in the form of a processing circuit (e.g., a processing ECU, etc.) or a semiconductor device (e.g., a semiconductor chip, etc.) as a processing device that is configured to be mountable on a host mobile body and has at least one processor 12 and one memory 10.

[0059] (Additional remarks) This specification discloses the following technical ideas and their combinations. Note that the reference symbols in parentheses in the appended remarks indicate the correspondence with the specific means described in the above detailed embodiments, and do not limit the technical scope of the present disclosure.

[0060] (Technical thought 1) 1. A processing system for calibrating scan positions between a plurality of optical sensors (40) mounted on a host vehicle (2) that scan an external world, the processing system having a processor (12), comprising: The processor: Acquiring point cloud data (Dp) representing a scanning point cloud by each of the optical sensors; Extracting the scanning point cloud obtained by scanning the road surface (3) on which the host moving body is traveling as a road surface point cloud (Ps) from the point cloud data for each of the optical sensors; monitoring the inclination angles (θi, θip, θir) of an approximate plane (30) approximating the road surface point cloud for each of the optical sensors with respect to a reference horizontal plane (20) defined along the left-right direction and the front-rear direction of the host moving body; searching for, among the optical sensors, the optical sensor that has the smallest fluctuation index (Iθ) that increases in accordance with the fluctuation angle (δθ) generated in the tilt angle as a reference sensor (40a); and outputting calibration data (Dc) that calibrates the scanning positions of the scanning point clouds in the point cloud data of the other optical sensors other than the reference sensor among the optical sensors, based on the scanning positions of the scanning point clouds in the point cloud data of the reference sensor.

[0061] (Technical thought 2) Monitoring the tilt angle includes: The processing system according to Technical Idea 1 includes monitoring the inclination angle when the flatness of the road surface falls within an allowable flatness range up to an allowable upper flatness limit value.

[0062] (Technical Thought 3) Monitoring the tilt angle includes: A processing system described in technical idea 2, which includes monitoring the inclination angle when the flatness based on the distance between the scanning position of the road surface point cloud by at least one of the optical sensors and the approximate plane falls within the plane tolerance range.

[0063] (Technical Thought 4) Monitoring the tilt angle includes: The processing system according to any one of Technical Ideas 1 to 3 includes monitoring the inclination angle when the horizontality of the road surface falls within a horizontal tolerance range up to an allowable horizontal upper limit value.

[0064] (Technical Thought 5) Monitoring the tilt angle includes: A processing system described in technical idea 4, which includes monitoring the inclination angle when the horizontality of the approximate plane that approximates the scanning position of the road surface point cloud by at least one of the optical sensors is within the horizontal tolerance range.

[0065] (Technical Thought 6) outputting the calibration data A processing system described in any one of technical ideas 1 to 5, which includes performing calibration between matching scanning positions when the amount of deviation (ΔP) between the scanning positions of the scanning point cloud in the point cloud data of each of the optical sensors increases beyond the allowable deviation range.

[0066] (Technical Thought 7) outputting the calibration data The processing system described in Technical Idea 6 includes performing calibration between the matched scanning positions when the amount of deviation between the scanning positions of the scanning point cloud in the point cloud data of each of the optical sensors acquired during the autonomous driving mode of the host mobile body increases beyond the deviation tolerance range.

[0067] (Technical Thought 8) Probing the reference sensor comprises: A processing system described in any one of technical ideas 1 to 7, which includes searching for the reference sensor having the smallest fluctuation index correlated to the sum of the squares of the fluctuation angles occurring in the tilt angle (θip) around the left-right axis (Op) in the left-right direction and the tilt angle (θir) around the front-rear axis (Or) in the front-rear direction.

[0068] (Technical Thought 9) Probing the reference sensor comprises: A processing system described in any one of technical ideas 1 to 7, which includes searching for the reference sensor with the smallest fluctuation index that matches the fluctuation angle of the tilt angle due to the combination of the tilt around the left-right axis (Op) in the left-right direction and the tilt around the front-to-back axis (Or) in the front-to-back direction relative to the reference horizontal plane.

[0069] The above-mentioned technical concepts 1 to 9 may be understood as the respective technical concepts of a method and a program. [Explanation of symbols]

[0070] 1: Processing system, 2: Host vehicle, 3: Traveling road surface, 10: Memory, 12: Processor, 20: Reference horizontal plane, 30: Approximate plane, 40: Optical sensor, 40a: Reference sensor, Dc: Calibration data, Dp: Point cloud data, Iθ: Fluctuation index, Op: Left-right axis, Or: Front-rear axis, Ps: Road surface point cloud, ΔP: Deviation amount, δθ: Fluctuation angle, θi, θip, θir: Inclination angle

Claims

1. 1. A processing system for calibrating scan positions between a plurality of optical sensors (40) mounted on a host vehicle (2) that scan an external world, the processing system having a processor (12), comprising: The processor: Acquiring point cloud data (Dp) representing a point cloud scanned by each of the optical sensors; Extracting the scanning point cloud obtained by scanning the road surface (3) on which the host moving body is traveling as a road surface point cloud (Ps) from the point cloud data for each of the optical sensors; monitoring the inclination angles (θi, θip, θir) of an approximate plane (30) approximating the road surface point cloud for each of the optical sensors with respect to a reference horizontal plane (20) defined along the left-right direction and the front-rear direction of the host moving body; Searching for the optical sensor among the optical sensors that has the smallest fluctuation index (Iθ) that increases in accordance with the fluctuation angle (δθ) generated in the tilt angle as a reference sensor (40a); and outputting calibration data (Dc) that calibrates the scanning positions of the scanning point clouds in the point cloud data of the other optical sensors other than the reference sensor among the optical sensors, based on the scanning positions of the scanning point clouds in the point cloud data of the reference sensor.

2. Monitoring the tilt angle The processing system according to claim 1 , further comprising monitoring the tilt angle when the flatness of the traveled road surface is within a flatness tolerance range up to an allowable flatness upper limit value.

3. Monitoring the tilt angle The processing system of claim 2, further comprising monitoring the inclination angle when the flatness based on the distance between the scanning position of the road surface point cloud by at least one of the optical sensors and the approximate plane falls within the plane tolerance range.

4. Monitoring the tilt angle The processing system according to claim 1 , further comprising monitoring the tilt angle when the horizontality of the road surface falls within a horizontal tolerance range up to an allowable horizontal upper limit value.

5. Monitoring the tilt angle The processing system of claim 4, further comprising monitoring the inclination angle when the horizontality of the approximate plane that approximates the scanning position of the road surface point cloud by at least one of the optical sensors falls within the horizontal tolerance range.

6. outputting the calibration data The processing system of claim 1, further comprising: performing calibration between the matched scanning positions when the amount of deviation (ΔP) between the scanning positions of the scanning point cloud in the point cloud data of each of the optical sensors increases beyond an allowable deviation range.

7. outputting the calibration data The processing system of claim 6, further comprising: performing calibration between matched scanning positions when the amount of deviation between the scanning positions of the scanning point cloud in the point cloud data of each of the optical sensors acquired during an autonomous driving mode of the host mobile body increases to outside the deviation tolerance range.

8. Probing the reference sensor comprises: The processing system according to any one of claims 1 to 7, further comprising searching for the reference sensor having the smallest fluctuation index correlated to the sum of the squares of the fluctuation angles occurring in the tilt angle (θip) around the left-right axis (Op) in the left-right direction and the tilt angle (θir) around the front-rear axis (Or) in the front-rear direction.

9. Probing the reference sensor comprises: The processing system according to any one of claims 1 to 7, further comprising searching for the reference sensor having the smallest fluctuation index that matches the fluctuation angle of the tilt angle resulting from the combination of the tilt around the left-right axis (Op) in the left-right direction and the tilt around the front-to-rear axis (Or) in the front-to-rear direction relative to the reference horizontal plane.

10. A processing device for calibrating scanning positions between a plurality of optical sensors (40) that scan an external environment of the host mobile (2), the processing device having a processor (12) and configured to be mountable on the host mobile (2), the processing device comprising: The processor: Acquiring point cloud data (Dp) representing a point cloud scanned by each of the optical sensors; Extracting the scanning point cloud obtained by scanning the road surface (3) on which the host moving body is traveling as a road surface point cloud (Ps) from the point cloud data for each of the optical sensors; monitoring the inclination angles (θi, θip, θir) of an approximate plane (30) approximating the road surface point cloud for each of the optical sensors with respect to a reference horizontal plane (20) defined along the left-right direction and the front-rear direction of the host moving body; Searching for the optical sensor among the optical sensors that has the smallest fluctuation index (Iθ) that increases in accordance with the fluctuation angle (δθ) generated in the tilt angle as a reference sensor (40a); and outputting calibration data (Dc) that calibrates the scanning positions of the scanning point clouds in the point cloud data of the other optical sensors other than the reference sensor among the optical sensors, based on the scanning positions of the scanning point clouds in the point cloud data of the reference sensor.

11. A processing method executed by a processor (12) for calibrating scanning positions between a plurality of optical sensors (40) mounted on a host vehicle (2) and scanning an external environment, comprising: Acquiring point cloud data (Dp) representing a point cloud scanned by each of the optical sensors; Extracting the scanning point cloud obtained by scanning the road surface (3) on which the host moving body is traveling as a road surface point cloud (Ps) from the point cloud data for each of the optical sensors; monitoring the inclination angles (θi, θip, θir) of an approximate plane (30) approximating the road surface point cloud for each of the optical sensors with respect to a reference horizontal plane (20) defined along the left-right direction and the front-rear direction of the host moving body; Searching for the optical sensor among the optical sensors that has the smallest fluctuation index (Iθ) that increases in accordance with the fluctuation angle (δθ) generated in the tilt angle as a reference sensor (40a); and outputting calibration data (Dc) that calibrates the scanning positions of the scanning point clouds in the point cloud data of the other optical sensors other than the reference sensor among the optical sensors, based on the scanning positions of the scanning point clouds in the point cloud data of the reference sensor.

12. A processing program stored in a storage medium (10) for calibrating scanning positions among a plurality of optical sensors (40) mounted on a host moving body (2) and scanning an external environment, the processing program including instructions for causing a processor (12) to execute the calibration, the processing program comprising: Acquiring point cloud data (Dp) representing a point cloud scanned by each of the optical sensors; Extracting the scanning point cloud obtained by scanning the road surface (3) on which the host moving body is traveling as a road surface point cloud (Ps) from the point cloud data for each of the optical sensors; monitoring the inclination angles (θi, θip, θir) of an approximate plane (30) approximating the road surface point cloud for each of the optical sensors with respect to a reference horizontal plane (20) defined along the left-right direction and the front-rear direction of the host moving body; Searching for the optical sensor among the optical sensors that has the smallest fluctuation index (Iθ) that increases in accordance with the fluctuation angle (δθ) generated in the tilt angle as a reference sensor (40a); and outputting calibration data (Dc) that calibrates the scanning positions of the scanning point clouds in the point cloud data of the other optical sensors other than the reference sensor among the optical sensors, based on the scanning positions of the scanning point clouds in the point cloud data of the reference sensor.

Images