Processing system, processing device, processing method, processing procedure

Abstract

The processor of the processing device is configured to perform: acquiring point group data (Dp) representing the scan point groups obtained by each optical sensor; extracting the road surface point groups (Ps) that scanned the driving road surface from the point group data (Dp) corresponding to each optical sensor; monitoring the tilt angle (θi) of the approximate plane relative to a reference horizontal plane defined along the left-right and front-back directions of the main vehicle, the approximate plane being approximated by the road surface point groups corresponding to each optical sensor; identifying the reference sensor whose variation index (Iθ) is minimized as it increases with the variation angle (δθ) generated in the tilt angle (θi); and outputting calibration data (Dc) that calibrates the scan positions of the scan point groups in the point group data (Dp) of the other optical sensors (excluding the reference sensor) based on the scan positions of the scan point groups in the point group data (Dp) of the reference sensor.

Description

[0001] Cross-reference of related applications

[0002] This application is based on Japanese Patent Application No. 2024-45332 filed on March 21, 2024, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] This disclosure relates to processing techniques for multiple optical sensors mounted on a main mobile body that scan the external environment. Background Technology

[0004] Patent document 1 discloses a technique for using LiDAR, which is an optical sensor, as multiple sensors mounted on a vehicle that is a moving body.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: U.S. Patent No. 10099630. Summary of the Invention

[0008] However, the scanning positions of the various LiDARs scanning the outside world from the vehicle can shift due to factors such as vehicle vibration or age-related degradation. Therefore, the accuracy of the scanning position calibration between the various LiDARs becomes important.

[0009] The present disclosure addresses the problem of providing a processing system that ensures the accuracy of calibration among multiple optical sensors. Another problem is providing a processing apparatus that ensures the accuracy of calibration among multiple optical sensors. Yet another problem is providing a processing method that ensures the accuracy of calibration among multiple optical sensors. Still another problem is providing a processing procedure that ensures the accuracy of calibration among multiple optical sensors.

[0010] The following describes the technical means employed by this disclosure to solve the problem.

[0011] The first aspect of this disclosure is a processing system having a processor for calibrating the scanning positions among multiple optical sensors mounted on a main moving body and scanning the external environment, wherein...

[0012] The processor is configured to execute:

[0013] Acquire point group data representing the scan point groups obtained by each optical sensor;

[0014] From the point group data corresponding to each optical sensor, the scan point group obtained by scanning the road surface where the main moving body travels is extracted and used as the road surface point group.

[0015] The monitoring approximate plane is tilted relative to a reference horizontal plane, which is defined along the left-right and front-back directions of the main moving body. This approximate plane is obtained by approximating the road surface point group corresponding to each optical sensor.

[0016] The optical sensor with the smallest variation index due to the change in the tracking angle generated by the tilt angle was selected as the reference sensor; and

[0017] The system outputs calibration data, which uses the scanning position of the scanning point group in the point group data of the reference sensor as a reference to calibrate the scanning position of the scanning point group in the point group data of other optical sensors besides the reference sensor.

[0018] A second aspect of this disclosure is a processing device having a processor and configured to be mounted on a main mobile body for calibrating the scanning positions among multiple optical sensors scanning the exterior of the main mobile body, wherein...

[0019] The processor is configured to execute:

[0020] Acquire point group data representing the scan point groups obtained by each optical sensor;

[0021] From the point group data corresponding to each optical sensor, the scan point group obtained by scanning the road surface where the main moving body travels is extracted and used as the road surface point group.

[0022] The monitoring approximate plane is tilted relative to a reference horizontal plane, which is defined along the left-right and front-back directions of the main moving body. This approximate plane is obtained by approximating the road surface point group corresponding to each optical sensor.

[0023] The optical sensor with the smallest variation index due to the change in the tracking angle generated by the tilt angle was selected as the reference sensor; and

[0024] The system outputs calibration data, which uses the scanning position of the scanning point group in the point group data of the reference sensor as a reference to calibrate the scanning position of the scanning point group in the point group data of other optical sensors besides the reference sensor.

[0025] The third aspect of this disclosure is a processing method, executed by a processor, for calibrating the scanning positions among multiple optical sensors mounted on a main moving body and scanning the external environment, wherein the method includes:

[0026] Acquire point group data representing the scan point groups obtained by each optical sensor;

[0027] From the point group data corresponding to each optical sensor, the scan point group obtained by scanning the road surface where the main moving body travels is extracted and used as the road surface point group.

[0028] The monitoring approximate plane is tilted relative to a reference horizontal plane, which is defined along the left-right and front-back directions of the main moving body. This approximate plane is obtained by approximating the road surface point group corresponding to each optical sensor.

[0029] The optical sensor with the smallest variation index due to the change in the tracking angle generated by the tilt angle was selected as the reference sensor; and

[0030] The system outputs calibration data, which uses the scanning position of the scanning point group in the point group data of the reference sensor as a reference to calibrate the scanning position of the scanning point group in the point group data of other optical sensors besides the reference sensor.

[0031] The fourth aspect of this disclosure is a processing program stored in a storage medium for calibrating the scanning positions among multiple optical sensors mounted on a main mobile body and scanning the external environment, comprising commands for causing a processor to perform the calibration, wherein the commands include commands for performing the following operations:

[0032] Acquire point group data representing the scan point groups obtained by each optical sensor;

[0033] From the point group data corresponding to each optical sensor, the scan point group obtained by scanning the road surface where the main moving body travels is extracted and used as the road surface point group.

[0034] The monitoring approximate plane is tilted relative to a reference horizontal plane, which is defined along the left-right and front-back directions of the main moving body. This approximate plane is obtained by approximating the road surface point group corresponding to each optical sensor.

[0035] The optical sensor with the smallest variation index due to the change in the tracking angle generated by the tilt angle was selected as the reference sensor; and

[0036] The system outputs calibration data, which uses the scanning position of the scanning point group in the point group data of the reference sensor as a reference to calibrate the scanning position of the scanning point group in the point group data of other optical sensors besides the reference sensor.

[0037] In the first to fourth methods, the scanned point groups of the road surface on which the main vehicle travels are scanned from the point group data acquired by each optical sensor are extracted as road surface point groups. Therefore, according to the first to fourth methods, the tilt angle of the approximate plane obtained by approximating the road surface point groups corresponding to each optical sensor relative to a reference horizontal plane defined along the left-right and front-back directions of the main vehicle is monitored. Thus, the variation index, which increases due to the change in the tilt angle that ideally maintains a certain angle relative to the reference horizontal plane of the road surface, becomes smaller for optical sensors with smaller offsets in the scanned position of the scanned point groups in the point group data. Therefore, using the scanned position of the scanned point group with the smallest offset explored by the reference sensor whose variation index increases with the tilt angle is minimized, the scanned positions of the scanned point groups acquired by other optical sensors are calibrated, thereby outputting calibration data that ensures accuracy. Attached Figure Description

[0038] Figure 1 This is a block diagram showing the overall structure of the first embodiment.

[0039] Figure 2 This is a top view showing the driving environment of the main vehicle using the first embodiment.

[0040] Figure 3 This is a block diagram illustrating the functional structure of the processing system according to the first embodiment.

[0041] Figure 4 This is a flowchart illustrating the processing flow of the first embodiment.

[0042] Figure 5 This is a schematic diagram used to illustrate the processing flow of the first embodiment.

[0043] Figure 6 This is a schematic diagram used to illustrate the processing flow of the first embodiment.

[0044] Figure 7 This is a schematic diagram used to illustrate the processing flow of the first embodiment.

[0045] Figure 8 This is a schematic diagram used to illustrate the processing flow of the first embodiment.

[0046] Figure 9 This is a schematic diagram used to illustrate the processing flow of the first embodiment.

[0047] Figure 10 This is a schematic diagram used to illustrate the processing flow of the first embodiment.

[0048] Figure 11This is a schematic diagram used to illustrate the processing flow of the first embodiment.

[0049] Figure 12 This is a flowchart illustrating the processing flow of the second embodiment.

[0050] Figure 13 This is a flowchart illustrating the processing flow of the third embodiment.

[0051] Figure 14 This is a schematic diagram illustrating the processing flow of the third embodiment. Detailed Implementation

[0052] Hereinafter, several embodiments of the present disclosure will be described based on the accompanying drawings. Furthermore, in each embodiment, corresponding constituent elements are given the same reference numerals, and repeated descriptions are sometimes omitted. Also, in cases where only a portion of the structure is described in each embodiment, structures from other previously described embodiments can be applied to the other parts of the structure. Moreover, not only combinations of structures explicitly shown in the descriptions of each embodiment are possible, but structures from multiple embodiments can be partially combined with each other even without explicit description, as long as there is no obstacle to combination.

[0053] (First Implementation)

[0054] Figure 1 The processing system 1 of the first embodiment shown is for processing the main vehicle 2 (refer to) which is the main moving body. Figure 2 Furthermore, the scanning positions of multiple optical sensors 40 scanning the external environment are calibrated to generate calibration data Dc. Here, the main vehicle 2 can be considered an ego-vehicle from the perspective centered on that vehicle. The main vehicle 2 is a moving body, such as a car, capable of traveling on the road surface 3 with occupants. Therefore, the directions described below are defined based on the main vehicle 2 on an ideal road surface 3 along a horizontal plane.

[0055] In the main vehicle 2, an autonomous driving mode is provided, which is graded according to the degree of manual intervention by the occupants in dynamic driving tasks. The autonomous driving mode can be achieved through autonomous driving control, such as conditional driving automation, high driving automation, or full driving automation, where the system performs all dynamic driving tasks. The autonomous driving mode can also be achieved through high driving support control, such as driving assistance or partial driving automation, where the occupants perform some or all of the dynamic driving tasks. The autonomous driving mode can also be achieved through any one, combination, or switching between these autonomous driving controls and high driving support controls.

[0056] The first embodiment Figure 2The optical sensors 40 shown are mounted in pairs on the vehicle 2 for scanning the area in front of the main vehicle 2. Each optical sensor 40 is a so-called LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) sensor that acquires optical information that can be used in the main vehicle 2 up to the autonomous driving mode. The sensing areas Rs of each optical sensor 40 partially overlap to form an overlapping area Rsc. Figure 2 (The shaded area of ​​the dots). For example... Figure 1 , Figure 3 As shown, each optical sensor 40 includes a light-emitting unit 400, a scanning unit 410, and a light-receiving unit 420.

[0057] The light-emitting unit 400 is mainly composed of a light-emitting element, such as a laser diode, that emits directional laser light in the infrared domain. The light-emitting unit 400 projects illumination light directed towards the outside of the main vehicle 2 in a discontinuous pulse beam. The scanning unit 410 is mainly composed of a scanning mirror. The scanning unit 410 scans the outside of the main vehicle 2 by reflecting the illumination light from the light-emitting unit 400 according to the rotation angle of the scanning mirror. The light-receiving unit 420 is composed of a light-receiving element, such as a SPAD (Single Photon Avalanche Diode), combined with an integrated circuit. The light-receiving unit 420 receives reflected light from objects outside the main vehicle 2 that are in contact with the illumination light, after reflection at the scanning unit 410. Based on the scanning positions of the objects that reflected the illumination light outside the main vehicle 2, and based on the light-receiving signal generated by the light-receiving pixel of each light-receiving element, the light-receiving unit 420 outputs a group of scan points, representing the scan point group data Dp, which contains information about these scanning positions.

[0058] like Figure 1 As shown, the processing system 1 is configured to include at least one dedicated computer. The processing system 1 is connected to each optical sensor 40 via at least one of, for example, a LAN (Local Area Network) line, a wiring harness, an internal bus, and a wireless communication line. If there are multiple dedicated computers constituting the processing system 1, the connections between these dedicated computers are also the same.

[0059] The dedicated computer constituting processing system 1 can be a driving control ECU (Electronic Control Unit) that controls the driving of the main vehicle 2. The dedicated computer constituting processing system 1 can be a navigation ECU that navigates the driving path of the main vehicle 2. The dedicated computer constituting processing system 1 can be a locator ECU that estimates the self-state of the main vehicle 2. The dedicated computer constituting processing system 1 can be an actuator ECU that controls the driving actuators of the main vehicle 2. The dedicated computer constituting processing system 1 can be an HCU (Human Machine Interface) Control Unit that controls information prompts in the main vehicle 2. The dedicated computer constituting processing system 1 can be a computer outside the main vehicle 2 that can communicate via the communication system of the main vehicle 2 and establish an external center or mobile terminal.

[0060] The dedicated computer constituting the processing system 1 has at least one memory 10 and one processor 12. The memory 10 is a non-transitory physical storage medium that non-transitorily stores programs and data that can be read by the computer, such as at least one of semiconductor memory, magnetic media, and optical media. This storage can be either an accumulation of data that continues to be stored regardless of the start-up or disconnection of the main vehicle 2, or a temporary storage that deletes data upon the start-up or disconnection of the main vehicle 2. The processor 12 includes at least one of the following as its core: CPU (Central Processing Unit), GPU (Graphics Processing Unit), RISC (Reduced Instruction Set Computer) – CPU, DFP (Data Flow Processor), and GSP (Graph Streaming Processor).

[0061] In processing system 1, processor 12 executes multiple commands contained in a processing program stored in memory 10 to generate calibration data Dc for calibrating the scanning positions between the optical sensors 40 scanning the external environment of the main vehicle 2. Therefore, processing system 1 constructs multiple functional blocks to generate calibration data Dc for calibrating the scanning positions between the optical sensors 40. Among the multiple functional blocks constructed in processing system 1, such as... Figure 3 As shown, it includes an acquisition block 100, an extraction block 110, a monitoring block 120, an exploration block 130, and an output block 140.

[0062] Through the collaborative operation of these blocks 100, 110, 120, 130, and 140, the processing system 1 generates calibration data Dc for calibrating the scanning positions between each optical sensor 40. This process follows a method... Figure 4 The processing flow shown is executed. This processing flow is repeatedly executed in the automatic driving mode controlled by the driving control ECU of the main vehicle 2. Furthermore, each "S" in this processing flow refers to a step executed by multiple commands contained in the processing program.

[0063] In S100, block 100 is retrieved as follows: Figure 5 As shown, point group data Dp representing the scan point group scanned by each optical sensor 40 is acquired. Preferably, the acquisition of point group data Dp is substantially synchronized among each optical sensor 40.

[0064] exist Figure 4 In S110 following S100 of the processing flow shown, extraction block 110 determines the offset ΔP between the scan positions of the scan point groups in the point group data Dp of each optical sensor 40. Specifically, firstly, in the point group data Dp of each optical sensor 40, the scan point groups within the overlapping region Rsc are compared with each other. As a result, for each scan point in the point group data Dp of each optical sensor 40, the total value of the distances between them, representing the distribution characteristic quantity, is calculated as the offset ΔP by matching the pairs with the smallest distance between them, for example, through nearest neighbor exploration processing or approximate nearest neighbor exploration processing. At this time, the point group data Dp of each optical sensor 40 is calibrated with the most recent data Dc of the calibration data Dc output when S170 (described later) in the previous processing flow, and then used to calculate the offset ΔP.

[0065] Therefore, in S110, it is determined whether the calculated offset ΔP has risen beyond the allowable range. At this time, the offset ΔP is determined based on the allowable range up to the upper limit of the allowable offset, which is the range within which the scanning positions of the scanning point groups in the point group data Dp of each optical sensor 40 no longer require calibration. In other words, rising beyond the allowable range means that the offset ΔP between the scanning positions of the scanning point groups in the point group data Dp of each optical sensor 40 increases to a size that requires calibration.

[0066] If a negative decision is made in S110, the current execution of the processing flow ends. Conversely, if a positive decision is made in S110, the processing flow proceeds to S120. In S120, the extraction block 110 generates control commands for the driving control ECU controlling the master vehicle 2 in its autonomous driving mode, guiding the vehicle 2 towards the driving surface 3 where calibration condition Cc is met. This guidance can be automatic guidance based on the autonomous driving mode, or it can be manual driving guidance, such as through display and / or voice, for occups who have deactivated the autonomous driving mode and transferred driving authority.

[0067] The fulfillment of calibration condition Cc in S120 means that at least one of the pre-defined flatness condition Ccp and levelness condition Cch is fulfilled. Specifically, the flatness condition Ccp is fulfilled when the flatness of the driving surface 3 is within the allowable flatness range up to the upper limit of the allowable flatness value. Here, flatness refers to the maximum difference in the height of the unevenness within a set area of ​​the driving surface 3. Therefore, the allowable flatness range is set within a range that ensures the approximate accuracy of the approximate plane 30 and the driving surface 3 described later to the required level. Furthermore, in the first embodiment, flatness is preferably identified based on at least one of, for example, sensing data acquired by sensors of the main vehicle 2 and communication data acquired by the communication unit of the main vehicle 2.

[0068] In contrast, the levelness condition Cch is satisfied when the levelness of the driving surface 3 is within the permissible upper limit of the levelness. Here, levelness refers to the inclination height, i.e., the slope, of the driving surface 3 at each set distance. Therefore, the permissible levelness range is set within a range that ensures the monitoring accuracy of the inclination angle θi of the approximate plane 30 and the reference level plane 20 (described later) to the required levelness. Furthermore, in the first embodiment, the levelness is preferably identified based on at least one of, for example, sensing data acquired by sensors of the main vehicle 2 and communication data acquired by the communication unit of the main vehicle 2.

[0069] In S130, following S120 in the processing flow, extraction block 110 determines whether calibration condition Cc is met. If a negative determination is made in S130, the processing flow returns to S100, thus repeating S100, S110, S120, and S130. Conversely, if a positive determination is made in S130, the processing flow proceeds to S140.

[0070] In S140, the extraction block 110 is located in the point group data Dp corresponding to each optical sensor 40, such as... Figure 6As shown, the scanned point group of the driving road surface 3 on which the main vehicle 2 travels is scanned is extracted as the road surface point group Ps. Specifically, the point group data Dp corresponding to each optical sensor 40 is filtered by, for example, cross-simulation, to extract the road surface point group Ps from the driving road surface 3 that satisfies the flatness and levelness of S120.

[0071] exist Figure 4 In the processing flow shown, following S140 to S150, monitoring block 120 monitors... Figure 8 The approximate plane 30 is relative to the main vehicle 2 on the driving surface 3. Figure 7 The reference horizontal plane 20 is tilted at an angle θi, and the approximate plane 30 is obtained by approximating the road surface point group Ps corresponding to each optical sensor 40. At this time, as... Figure 7 As shown, the reference horizontal plane 20 is defined by the pitch axis Ap along the left-right direction and the roll axis Ar along the front-rear direction in the main vehicle 2, and is imagined to extend along the left, right, front, and rear directions of the vehicle 2. Therefore, as the tilt angle θi, it is monitored individually as follows: Figure 9 The two-dimensional tilt angle θip is shown about the left and right axes Op of the pitch axis Ap along the left and right directions, and as shown in the figure. Figure 10 The figure shows another two-dimensional tilt angle θir about the front-rear axis Or about the tilt axis Ar along the front-rear direction.

[0072] The approximate plane 30 for monitoring the tilt angle θi is fitted using, for example, the orthogonal coordinates x, y, and z in Equation 1 representing the scanning positions of each scanning point constituting the road surface point group Ps, and the fitting process, such as least squares method or principal component analysis. Figure 8 As shown, this is derived. At this point, the approximate plane 30, by following Equation 1 using coefficients a, b, c, and d, becomes a virtual plane that approximates the driving surface 3. Therefore, in the reference horizontal plane 20, as... Figures 8-10 As shown, imagine an orthogonal coordinate system 200 defined by the X, Y, and Z axes along the left-right, front-back, and height directions of the main vehicle 2, respectively. The X and Y axes of this orthogonal coordinate system 200 are substantially aligned with the pitch axis Ap in the left-right direction and the roll axis Ar in the front-back direction, respectively.

[0073] [Formula 1]

[0074]

[0075] Therefore, in the orthogonal coordinate system 200 Figure 9 The tilt angle θip is monitored as the angle formed by the normal vector N, expressed by the following equation 2 with respect to the reference horizontal plane 20, and the normal vector n, expressed by the following equation 3 with respect to the approximate plane 30, about the left and right axes Op. Meanwhile, in the orthogonal coordinate system 200... Figure 10The tilt angle θir is monitored as the angle formed by the normal vector N (expressed by Equation 2 for the reference horizontal plane 20) and the normal vector n (expressed by Equation 3 for the approximate plane 30) around the front and rear axes Or. In summary, in S150, after a positive determination is made in S130 due to the fulfillment of calibration condition Cc, tilt monitoring of the approximate plane 30 relative to the reference horizontal plane 20 will be performed.

[0076] [Formula 2]

[0077]

[0078] [Formula 3]

[0079]

[0080] exist Figure 4 In S160, following S150 of the processing flow shown, the exploration block 130 searches among the optical sensors 40 to find the optical sensor 40 with the smallest variation index Iθ related to the variation angle δθ generated in the respective tilt angle θi of each optical sensor 40, and uses it as the reference sensor 40a. At this time, the variation angle δθ of each optical sensor 40 is defined by the deviation between the tilt angle θi extracted by the monitoring in the previous S150 and the reference angle used as its reference. The reference angle for this tilt angle θi can be the initial value measured when setting each optical sensor 40 to the main vehicle 2. The reference angle for the tilt angle θi can also be the tilt angle θi of the past cycle extracted by the monitoring in S150 in the past processing flow.

[0081] This variation angle δθ, particularly in the first embodiment, is defined as the deviation of the tilt angles θip and θir from the corresponding reference angles, and is a two-dimensional variation angle δθp and δθr constituting the following equation 4. Therefore, the variation index Iθ is related to the sum of the squares of the variation angles δθp and δθr of the tilt angle θip, and is calculated for each optical sensor 40 according to the following equation 4, gradually increasing as at least one of these variation angles δθp and δθr increases. As a result, the calculated variation index Iθ is as follows: Figure 11 The optical sensor 40 that becomes the minimum value is selected as the reference sensor 40a. Meanwhile, the optical sensor 40 other than the reference sensor 40a, i.e., the optical sensor 40 whose variation index Iθ becomes the other than the minimum value, is selected as the object sensor 40b, which will be described later as the calibration target.

[0082] [Formula 4]

[0083]

[0084] exist Figure 4In S170, following S160 of the processing flow shown, output block 140 calibrates the scanning positions of the scanning point groups in the point group data Dp of the object sensor 40b, using the scanning positions of the scanning point groups in the point group data Dp of the reference sensor 40a as a reference. At this time, a 1:1 matching of scanning positions between the reference side and the calibration side is performed, applying the matching established in the previous S110 of the current processing flow. Therefore, in S170, by shifting the scanning positions of each scanning point in the point group data Dp of the object sensor 40b towards the scanning position side of the matching scanning point in the point group data Dp of the reference sensor 40a, an optimization calculation is performed to minimize the offset ΔP calculated based on S110.

[0085] In this optimization calculation, coordinate transformation parameters are calculated to shift the scanning positions of each scan point in the point group data Dp of the object sensor 40b towards the scanning positions of the scan points matched in the point group data Dp of the reference sensor 40a. Therefore, in S170, the calculated transformation parameters are output as calibration data Dc. At this time, in S170, after a positive determination is made in S110 due to the offset ΔP rising outside the allowable range, the output of calibration data Dc is achieved at least through data storage in the memory 10.

[0086] Thus, the calibration data Dc output in S170 is used to generate point group data Dp, which is obtained by calibrating the scanning position of the target sensor 40b based on the scanning position of the reference sensor 40a. Therefore, in addition to the data storage described above, the output of the calibration data Dc in S170 can also be, for example, providing data to the driving control ECU in the main vehicle 2, or sending data to an external center through the communication unit of the main vehicle 2. As described above, this execution of the processing flow ends with the output of S170.

[0087] (Effects)

[0088] The effects of the first embodiment will now be explained.

[0089] In the first embodiment, the scanned point group obtained by scanning the driving surface 3 of the main vehicle 2 from the point group data Dp acquired by each optical sensor 40 is extracted as the road surface point group Ps. Therefore, according to the first embodiment, the tilt angle θi of the approximate plane 30 relative to the reference horizontal plane 20 is monitored. The reference horizontal plane 20 is defined along the left-right and front-back directions of the main vehicle 2, and the approximate plane 30 is obtained by approximating the road surface point group Ps corresponding to each optical sensor 40. As a result, the variation index Iθ, which increases with the variation angle δθ generated by the tilt angle θi that should ideally keep the approximate plane 30 of the driving surface 3 at a certain angle relative to the reference horizontal plane 20, becomes smaller in optical sensors 40 with smaller scan position offsets of the scanned point groups in the point group data Dp. Therefore, by using the scan position of the scan point group obtained by the reference sensor 40a, which is explored with the smallest deviation of the variation index Iθ that increases with the variation angle δθ following the tilt angle θi, as a reference, the scan position of the scan point group obtained by other optical sensors 40 (i.e., object sensor 40b) is calibrated, and calibration data Dc that can ensure accuracy can be output.

[0090] According to the first embodiment, when the flatness of the driving surface 3 is within the allowable flatness range up to the upper limit of the allowable flatness, the tilt angle θi of the approximate plane 30 relative to the reference horizontal plane 20, which can improve the approximation accuracy with respect to the driving surface 3, is monitored. Therefore, the reference sensor 40a can accurately detect the minimum change index Iθ that increases with the change angle δθ following the tilt angle θi, thus ensuring the accuracy of the calibration performed by the calibration data Dc.

[0091] According to the first embodiment, when the levelness of the driving surface 3 is within the permissible level range up to the upper limit of the permissible level, the tilt angle θi of the approximate plane 30 obtained by approximating the road surface point group Ps of the driving surface 3 relative to the reference level plane 20 that allows the main vehicle 2 to maintain a stable posture on the driving surface 3 is monitored. Therefore, the reference sensor 40a can accurately detect the minimum change index Iθ that increases with the change angle δθ following the tilt angle θi, thus ensuring the accuracy of the calibration performed by the calibration data Dc.

[0092] According to the first embodiment, when the offset ΔP between the scan positions of the scan points in the point group data Dp of each optical sensor 40 rises beyond the allowable offset range, the driving positions for which calibration is performed between the reference sensor 40a and the other optical sensors 40 (i.e., the target sensor 40b) can be precisely matched. Therefore, the accuracy of the calibration performed using the calibration data Dc can be ensured. In particular, in the first embodiment, when the offset ΔP between the scan positions of the scan points in the point group data Dp of each optical sensor 40 acquired in the autonomous driving mode of the main vehicle 2 rises beyond the allowable offset range, precise matching of the scan positions can be achieved. Therefore, the calibration accuracy required for the autonomous driving mode can be ensured.

[0093] According to the first embodiment, the tilt angle θi of the approximate plane 30 of the driving surface 3 relative to the reference horizontal plane 20 along the left-right and front-back directions of the main vehicle 2 is considered, focusing on the tilt angle θip about the left-right axis Op along the left-right direction and the tilt angle θir about the front-back axis Or in the front-back direction. Therefore, based on the variation index Iθ associated with the sum of the squares of the variation angles δθ generated in these two-dimensional tilt angles θip and θir respectively, the reference sensor 40a with the smallest deviation of the scanning position of the scanning point group can be accurately explored. Therefore, the accuracy of the calibration performed by the calibration data Dc can be ensured.

[0094] (Second Implementation)

[0095] The second embodiment is a variation of the first embodiment. Figure 12 In the processing flow of the second embodiment shown, with the omission of S120, S130 is executed after the execution of S140.

[0096] However, in the second embodiment, when the calibration condition Cc is the flatness condition Ccp in S130, the flatness of the driving surface 3 is identified based on the separation distance between the scan position under the road surface point group Ps extracted in the previous S130 and the scan point group obtained by at least one optical sensor 40, thereby determining that the condition Ccp is valid. Similarly, in the second embodiment, when the calibration condition Cc is the levelness condition Cch in S130, the levelness of the approximate plane 30 obtained by approximating the scan position under the road surface point group Ps extracted in the previous S130 based on the scan point group obtained by at least one optical sensor 40 is identified, thereby determining that the condition Cch is valid. Furthermore, when the calibration condition Cc is both the flatness condition Ccp and the levelness condition Cch, either of these conditions Ccp or Cch can be identified according to the first embodiment.

[0097] During the negative decision period in S130, the processing flow returns to S100, and S100, S110, S140, and S130 are repeated. On the other hand, after the positive decision is made in S130, the processing flow moves to S150.

[0098] In S150 of this second embodiment, if the flatness of the separation distance between the scanned position of the road surface point group Ps acquired based on at least one optical sensor 40 and the approximate plane 30 is within the allowable range of flatness, the tilt angle θi of the approximate plane 30 relative to the reference horizontal plane 20 is monitored. Thus, the road surface point group Ps required for monitoring the tilt angle θi can be effectively utilized in the necessity determination of this monitoring based on flatness, thereby reducing the processing load for ensuring calibration accuracy.

[0099] Furthermore, in S150 of the second embodiment, if the levelness of the approximate plane 30 obtained by approximating the scanning position of the road surface point group Ps acquired by at least one optical sensor 40 is within the allowable level range, the tilt angle θi of the approximate plane 30 relative to the reference horizontal plane 20 is monitored. Thus, the road surface point group Ps required for monitoring the tilt angle θi can be effectively utilized in determining the necessity of this monitoring based on the levelness, thereby reducing the processing load required to ensure calibration accuracy.

[0100] (Third Implementation)

[0101] The third embodiment is a variation of the first embodiment. For example... Figure 13 As shown, in the processing flow of the third embodiment, S3150 and S3160 are executed instead of S150 and S160, respectively.

[0102] Specifically, in S3150, such as Figure 14 As shown, monitoring block 120 monitors the three-dimensional tilt angle θi generated by the combination of the tilt of the left-right axis Op in the left-right direction and the tilt of the front-back axis Or in the front-back direction relative to the reference horizontal plane 20 of the hypothetical orthogonal coordinate system 200. At this time, as the angle between the normal vectors N and n of each surface 20 and 30, which are represented by the same equations 2 and 3 as in the first embodiment, on the virtual plane containing these vectors N and n, the monitoring block 130 monitors the tilt angle θi that satisfies the following equation 5. Therefore, the exploration block 130 in S3160 explores the reference sensor 40a by comparing the variation index Iθ, which increases with the variation angle δθ generated in such a three-dimensional tilt angle θi and follows the following equation 6, that is, the variation index Iθ that increases with the variation angle δθ and has a correlation coefficient of substantially 1, between each optical sensor 40.

[0103] [Formula 5]

[0104]

[0105] [Formula 6]

[0106]

[0107] According to this third embodiment, the angle of inclination of the approximate plane 30 of the driving surface 3 relative to the reference horizontal plane 20 of the main vehicle 2 is considered to be the three-dimensional tilt angle θi generated by the combination of the tilt about the left and right axes Op in the left-right direction and the tilt about the front and rear axes Or in the front-rear direction. Therefore, based on the variation index Iθ which is consistent with the variation angle δθ generated in the three-dimensional tilt angle θi about these two axes Op and Or, the reference sensor 40a with the smallest scanning position deviation of the scanning point group can be accurately explored. Therefore, the accuracy of the calibration performed by the calibration data Dc can be ensured.

[0108] (Other implementation methods)

[0109] The above describes several implementation methods, but this disclosure is not limited to these implementation methods and can be applied to various implementation methods and combinations without departing from the spirit of this disclosure.

[0110] In a variation, the dedicated computer constituting the processing system 1 may have at least one of digital circuitry and analog circuitry as a processor. Here, the digital circuitry is at least one of, for example, an ASIC (Application Specific Integrated Circuit), a FPGA (Field Programmable Gate Array), a SOC (System on a Chip), a PGA (Programmable Gate Array), and a CPLD (Complex Programmable Logic Device). Furthermore, such digital circuitry may include a memory storing a program.

[0111] In the modified example's processing flow, S3150 and S3160 of the third embodiment can be executed instead of S150 and S160 of the second embodiment. In the modified example's processing flow, S110 of the first to third embodiments is omitted, and the matching performed to calculate the offset ΔP in the subsequent S170 of the first to third embodiments can be performed in the manner of S110. In the modified example's processing flow, S120 of the first and third embodiments can be omitted. In the modified example's processing flow, S130 of the second embodiment can be omitted.

[0112] In a modified example, three or more optical sensors 40 may be mounted on the main vehicle 2 with all of them sharing a common overlapping area Rsc. In this case, during the processing flow of the first to third embodiments, in S160 and S3160, while selecting a reference sensor 40a, the remaining optical sensors 40 may be selected as target sensors 40b.

[0113] In a modified example, three or more optical sensors 40 may be mounted on the main vehicle 2 in a state where they form an overlapping region Rsc in a manner shared by at least two of them. In the processing flow of this case, in S160, 3160 of the first to third embodiments, a reference sensor 40a may be selected from the optical sensors 40 that share the overlapping region Rsc, and the remaining optical sensors 40 may be selected as object sensors 40b.

[0114] The main mobile body used in the modified processing system 1 can be, for example, an autonomous driving robot capable of transporting luggage or collecting information through autonomous driving or remote driving. In addition to the description above, the above embodiments and modifications can also be configured as processing devices that can be mounted on the main mobile body and each has at least one processor 12 and a memory 10, and can be implemented in the form of processing circuits (e.g., processing ECUs) or semiconductor devices (e.g., semiconductor chips).

[0115] (Postscript)

[0116] This specification discloses several technical ideas and combinations thereof, as listed below. Furthermore, the symbols in parentheses in this appendix indicate a correspondence with the specific means described in the previously detailed embodiments, and should not be construed as limiting the technical scope of this disclosure.

[0117] (Technical Idea 1)

[0118] A processing system having a processor (12) for calibrating the scanning positions among a plurality of optical sensors (40) mounted on a main moving body (2) and scanning the external environment, wherein,

[0119] The processor is configured to execute:

[0120] Obtain point group data (Dp) representing the scan point group obtained by each of the optical sensors;

[0121] In the point group data corresponding to each optical sensor, the scan point group obtained by scanning the driving road surface (3) of the main moving body is extracted and used as the road surface point group (Ps).

[0122] The tilt angle (θi, θip, θir) of the monitoring approximate plane (30) relative to the reference horizontal plane (20) is defined along the left-right and front-back directions of the main moving body. The approximate plane is obtained by approximating the group of road surface points corresponding to each of the optical sensors.

[0123] The optical sensor that minimizes the variation index (Iθ) as it increases with the variation angle (δθ) generated in the tilt angle is selected as the reference sensor (40a); and

[0124] Output calibration data (Dc), which uses the scanning position of the scanning point group in the point group data of the reference sensor as a reference to calibrate the scanning position of the scanning point group in the point group data of the other optical sensors besides the reference sensor.

[0125] (Technical Idea 2)

[0126] The processing system as described in technical concept 1, wherein

[0127] Monitoring the tilt angle includes:

[0128] The tilt angle is monitored when the flatness of the road surface is within the allowable flatness limit.

[0129] (Technical Idea 3)

[0130] The processing system as described in technical concept 2, wherein,

[0131] Monitoring the tilt angle includes:

[0132] The tilt angle is monitored when the flatness of the separation distance between the scanned position of the road surface point group obtained based on at least one of the optical sensors and the approximate plane is within the allowable range of the plane.

[0133] (Technical Idea 4)

[0134] The processing system as described in any one of technical concepts 1 to 3, wherein,

[0135] Monitoring the tilt angle includes:

[0136] The tilt angle is monitored when the levelness of the road surface is within the permissible level range up to the upper limit of the permissible level.

[0137] (Technical Idea 5)

[0138] The processing system described in technical concept 4, wherein...

[0139] Monitoring the tilt angle includes:

[0140] The tilt angle is monitored when the levelness of the approximate plane obtained by approximating the scanned position of the road surface point group obtained by at least one of the optical sensors is within the level tolerance range.

[0141] (Technical Idea 6)

[0142] The processing system as described in any one of technical concepts 1 to 5, wherein,

[0143] The output calibration data includes:

[0144] If the offset (ΔP) between the scan positions of the scan point groups in the point group data of each of the optical sensors rises beyond the allowable offset range, calibration is performed between the matched scan positions.

[0145] (Technical Idea 7)

[0146] The processing system described in technical concept 6, wherein,

[0147] The output calibration data includes:

[0148] If the offset between the scan positions of the scan point groups in the point group data of each of the optical sensors acquired in the autonomous driving mode of the main mobile body rises beyond the allowable offset range, calibration is performed between the matched scan positions.

[0149] (Technical Idea 8)

[0150] The processing system as described in any one of technical concepts 1 to 7, wherein,

[0151] The exploration of the benchmark sensor includes:

[0152] The reference sensor that minimizes the sum of the squares of the variation indices associated with the tilt angles (θip) generated in the left-right axis (Op) about the left-right direction and the tilt angles (θir) generated in the front-back axis (Or) about the front-back direction.

[0153] (Technical Idea 9)

[0154] The processing system as described in any one of technical concepts 1 to 7, wherein,

[0155] The exploration of the benchmark sensor includes:

[0156] The reference sensor that minimizes the variation index of the tilt angle, which is the result of the combination of the tilt angle relative to the reference horizontal plane about the left-right axis (Op) in the left-right direction and the tilt angle about the front-back axis (Or) in the front-back direction.

[0157] Furthermore, the aforementioned technical ideas 1 to 9 can also be grasped through various technical ideas of methods and procedures.

Claims

1. A processing system having a processor (12) for calibrating the scanning positions among a plurality of optical sensors (40) mounted on a main moving body (2) and scanning the external environment, characterized in that, The processor is configured to execute: Obtain point group data (Dp) representing the scan point group obtained by each of the optical sensors; In the point group data corresponding to each optical sensor, the scan point group obtained by scanning the driving road surface (3) of the main moving body is extracted and used as the road surface point group (Ps). The tilt angle (θi, θip, θir) of the monitoring approximate plane (30) relative to the reference horizontal plane (20) is defined along the left-right and front-back directions of the main moving body. The approximate plane is obtained by approximating the group of road surface points corresponding to each of the optical sensors. The optical sensor with the smallest variation index (Iθ) that increases with the variation angle (δθ) generated in the tilt angle is selected as the reference sensor (40a). as well as Output calibration data (Dc), which uses the scanning position of the scanning point group in the point group data of the reference sensor as a reference to calibrate the scanning position of the scanning point group in the point group data of the other optical sensors besides the reference sensor.

2. The processing system as described in claim 1, characterized in that, Monitoring the tilt angle includes: The tilt angle is monitored when the flatness of the road surface is within the allowable flatness limit.

3. The processing system as described in claim 2, characterized in that, Monitoring the tilt angle includes: The tilt angle is monitored when the flatness of the separation distance between the scanned position of the road surface point group obtained based on at least one of the optical sensors and the approximate plane is within the allowable range of the plane.

4. The processing system as described in claim 1, characterized in that, Monitoring the tilt angle includes: The tilt angle is monitored when the levelness of the road surface is within the permissible level range up to the upper limit of the permissible level.

5. The processing system as described in claim 4, characterized in that, Monitoring the tilt angle includes: The tilt angle is monitored when the levelness of the approximate plane obtained by approximating the scanned position of the road surface point group obtained by at least one of the optical sensors is within the level tolerance range.

6. The processing system as described in claim 1, characterized in that, The output calibration data includes: If the offset (ΔP) between the scan positions of the scan point groups in the point group data of each of the optical sensors rises beyond the allowable offset range, calibration is performed between the matched scan positions.

7. The processing system as described in claim 6, characterized in that, The output calibration data includes: If the offset between the scan positions of the scan point groups in the point group data of each of the optical sensors acquired in the autonomous driving mode of the main mobile body rises beyond the allowable offset range, calibration is performed between the matched scan positions.

8. The processing system according to any one of claims 1 to 7, characterized in that, The exploration of the benchmark sensor includes: The reference sensor that minimizes the sum of the squares of the variation indices associated with the tilt angles (θip) generated in the left-right axis (Op) about the left-right direction and the tilt angles (θir) generated in the front-back axis (Or) about the front-back direction.

9. The processing system according to any one of claims 1 to 7, characterized in that, The exploration of the benchmark sensor includes: The reference sensor that minimizes the variation index of the tilt angle, which is the result of the combination of the tilt angle relative to the reference horizontal plane about the left-right axis (Op) in the left-right direction and the tilt angle about the front-back axis (Or) in the front-back direction.

10. A processing apparatus having a processor (12) and configured to be mounted on a main moving body (2) for calibrating the scanning positions among a plurality of optical sensors (40) scanning the outside of the main moving body, characterized in that, The processor is configured to execute: Obtain point group data (Dp) representing the scan point group obtained by each of the optical sensors; In the point group data corresponding to each optical sensor, the scan point group obtained by scanning the driving road surface (3) of the main moving body is extracted and used as the road surface point group (Ps). The tilt angle (θi, θip, θir) of the monitoring approximate plane (30) relative to the reference horizontal plane (20) is defined along the left-right and front-back directions of the main moving body. The approximate plane is obtained by approximating the group of road surface points corresponding to each of the optical sensors. The optical sensor with the smallest variation index (Iθ) that increases with the variation angle (δθ) generated in the tilt angle is selected as the reference sensor (40a). as well as Output calibration data (Dc), which uses the scanning position of the scanning point group in the point group data of the reference sensor as a reference to calibrate the scanning position of the scanning point group in the point group data of the other optical sensors besides the reference sensor.

11. A processing method, executed by a processor (12), for calibrating the scanning positions among a plurality of optical sensors (40) mounted on a main moving body (2) and scanning the external environment, characterized in that, Include: Obtain point group data (Dp) representing the scan point group obtained by each of the optical sensors; In the point group data corresponding to each optical sensor, the scan point group obtained by scanning the driving road surface (3) of the main moving body is extracted and used as the road surface point group (Ps). The tilt angle (θi, θip, θir) of the monitoring approximate plane (30) relative to the reference horizontal plane (20) is defined along the left-right and front-back directions of the main moving body. The approximate plane is obtained by approximating the group of road surface points corresponding to each of the optical sensors. The optical sensor with the smallest variation index (Iθ) that increases with the variation angle (δθ) generated in the tilt angle is selected as the reference sensor (40a). as well as Output calibration data (Dc), which uses the scanning position of the scanning point group in the point group data of the reference sensor as a reference to calibrate the scanning position of the scanning point group in the point group data of the other optical sensors besides the reference sensor.

12. A processing program stored in a storage medium (10) for calibrating the scanning positions among a plurality of optical sensors (40) mounted on a main moving body (2) and scanning the outside world, comprising commands for causing a processor (12) to perform the calibration, characterized in that, Includes the commands used to perform the following operations: Obtain point group data (Dp) representing the scan point group obtained by each of the optical sensors; In the point group data corresponding to each optical sensor, the scan point group obtained by scanning the driving road surface (3) of the main moving body is extracted and used as the road surface point group (Ps). The tilt angle (θi, θip, θir) of the monitoring approximate plane (30) relative to the reference horizontal plane (20) is defined along the left-right and front-back directions of the main moving body. The approximate plane is obtained by approximating the group of road surface points corresponding to each of the optical sensors. The optical sensor with the smallest variation index (Iθ) that increases with the variation angle (δθ) generated in the tilt angle is selected as the reference sensor (40a). as well as Output calibration data (Dc), which uses the scanning position of the scanning point group in the point group data of the reference sensor as a reference to calibrate the scanning position of the scanning point group in the point group data of the other optical sensors besides the reference sensor.

Images