Information processing device, information processing method, and information processing system

Abstract

Achieves highly accurate calibration. [Solution] The information processing device is equipped with a processor and is a device that acquires data for calibrating a target device which is a device having a LiDAR (Light Detection and Ranging) sensor mounted on a vehicle. The processor calculates calibration parameters for the target device based on images of the vehicle and the target device acquired by an imaging device which is a device having a LiDAR sensor positioned outside the vehicle to capture images from a position overlooking the vehicle and the target device.

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing apparatus, an information processing method, and an information processing system.

Background Art

[0002] LiDAR (Light Detection and Ranging) technology is widely used in fields such as detecting people, animals, obstacles, etc. and measuring the distances to them in manned or unmanned vehicles such as automobiles. In order to accurately perform detection and ranging by this LiDAR, it is necessary to perform appropriate calibration based on the positions and orientations of an irradiation device and an image sensor that receives the light emitted by the irradiation device.

[0003] For example, when mounted on an automobile, calibration is performed at the time of shipment at a factory where a LiDAR sensor is mounted on the automobile. However, in order to achieve more accurate measurement, etc., re-calibration is performed at a dealership, or calibration for confirmation is performed during maintenance.

[0004] At the time of shipment from the manufacturing factory, high-precision calibration can be performed if a calibration system is installed on the line of the factory. On the other hand, at facilities such as dealerships and repair shops during maintenance, it is difficult to achieve calibration under the same conditions as the manufacturing factory due to issues such as ensuring the space for arranging the vehicle with respect to the test board and the accuracy of the vehicle's orientation.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

[0006] Therefore, one of the non-limiting problems that the embodiment aims to solve is to achieve accurate calibration even under limited conditions. Further examples of problems that the embodiment aims to solve include those corresponding to the effects described in the following description. That is, any problem corresponding to at least one of the effects described in the description of the embodiment may be a problem that the embodiment aims to solve. [Means for solving the problem]

[0007] According to one embodiment, the information processing device is a device that includes a processor and acquires data for calibrating a target device which is a device having a LiDAR (Light Detection and Ranging) sensor mounted on a vehicle. The aforementioned processor, Based on images of the vehicle and the target device acquired by an imaging device having a LiDAR sensor positioned outside the vehicle to capture images from a position overlooking the vehicle and the target device, parameters related to calibration in the target device are calculated.

[0008] In the above configuration, the processor may extract the shape of the top surface of the target device from the image acquired from the imaging device.

[0009] In the above configuration, the processor may acquire information relating to the position and orientation of the target device based on the shape of the top surface.

[0010] In the above configuration, the processor may acquire the position of the target device relative to the vehicle, the yaw angle with respect to the reference axis, and the pitch angle with respect to the reference axis based on the shape of the upper surface of the target device.

[0011] In any of the above configurations, the processor may calculate the parameters based on information relating to the position and orientation of the target device.

[0012] In any of the above configurations, the processor may extract the shapes of the emission surface and light-receiving surface of the target device from the image acquired from the imaging device.

[0013] In the above configuration, the processor may acquire information relating to the position and orientation of the target device relative to the vehicle based on the shapes of the ejection surface and the light-receiving surface.

[0014] In the above configuration, the processor may acquire the height at which the target device is installed and the roll angle with respect to the reference axis, based on the shapes of the light-emitting surface and the light-receiving surface of the target device.

[0015] In any of the above configurations, the processor may acquire information relating to the vehicle from the image, and may acquire information relating to the position and orientation of the vehicle based on the information relating to the vehicle.

[0016] In the above configuration, the processor may acquire the position of the vehicle from a reference point and the yaw angle of the vehicle with respect to a reference axis, based on the information relating to the vehicle.

[0017] In the above configuration, the processor may correct the calibration parameters based on information regarding the vehicle's position from a reference point and the vehicle's yaw angle with respect to a reference axis.

[0018] According to one embodiment, the information processing method may comprise the following steps. - The vehicle and the target device, which is a device having a LiDAR sensor mounted on the vehicle, are captured from a position overlooking the vehicle by an imaging device, which is a device having a LiDAR sensor located outside the vehicle. - The processor acquires images of the vehicle and the target device from the image information acquired by the imaging device. - The processor calculates parameters related to calibration of the target device based on the image.

[0019] In the above method, the processor may transmit the parameters to the target device.

[0020] According to an embodiment, an information processing system is a system for calibrating a target device having a LiDAR sensor mounted on a vehicle, and may include the following configuration. - An imaging device having a LiDAR sensor that images from a position overlooking the vehicle and the target device, - A processor that calculates parameters related to calibration in the target device from the images of the vehicle and the target device output from the imaging device.

[0021] In the above system, the processor may transmit the parameters to the target device.

Brief Description of Drawings

[0022] [Figure 1] A schematic diagram showing an example of an information processing system according to an embodiment. [Figure 2] A flowchart showing an example of the processing of an information processing system according to an embodiment. [Figure 3] A diagram showing an example of a cover according to an embodiment. [Figure 4] A diagram showing an example of the extracted upper surface shape according to an embodiment. [Figure 5] A diagram showing an example of the calculated center line and center point of the upper surface according to an embodiment. [Figure 6] A top view of a vehicle or the like according to an embodiment. [Figure 7] A top view of a vehicle or the like according to an embodiment. [Figure 8]A front view of a vehicle or the like according to one embodiment. [Figure 9] A top view of a vehicle or the like according to one embodiment. [Figure 10] A top view of a vehicle or the like according to one embodiment. [Modes for carrying out the invention]

[0023] The embodiments will be described below with reference to the drawings. Please note that in the drawings, some parts are highlighted for illustrative purposes, and the proportions of the size, distance, etc. of each element are not necessarily shown accurately.

[0024] Figure 1 is a schematic diagram showing an example of an information processing system 1 according to one embodiment. The information processing system 1 is a system for calibrating a target device 22, which is a device having a LiDAR sensor mounted on a vehicle 20. More specifically, the information processing system 1 can be a system for acquiring parameters for the calibration of the target device 22. The information processing system 1 can include, as an open configuration, an information processing device 10 and an imaging device 12. For convenience, the first direction (x), second direction (y), and third direction (z) may be defined as shown in the drawing.

[0025] The information processing device 10 may be, for example, a computer. The information processing device 10 may have, for example, a processing circuit (processor) and a memory area.

[0026] The processing circuit may be a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphical Processing Unit). Alternatively, the information processing device 10 may have a configuration with specialized circuits such as a DSP (Digital Signal Processor) or an ASIC (Application Specified Integrated Circuit), or it may have a configuration with programmable circuits such as an FPGA (Field Programmable Gate Array).

[0027] If the information processing device 10 has a general-purpose processing circuit, the processing of the information processing system may be configured in which software-based information processing is specifically implemented by the processing circuit, which is a hardware resource. In this case, the information processing device 10 may store programs, executable files, etc. related to the execution of software in its storage area, and the processing circuit may refer to these programs, executable files, etc. in the storage area to execute the information processing.

[0028] The imaging device 12 is a device that takes images of the vehicle 20 equipped with the target device 22, which is the target of calibration. The imaging device 12 may be a device equipped with a LiDAR sensor as an image sensor. Alternatively, the imaging device 12 may be a device having a light source and optical system that emits light in a band within the range that can be received by the LiDAR sensor. As an example, the imaging device 12 may be the same device as the target device 22, or a device having the same resolution as the target device 22.

[0029] The imaging device 12 is positioned, for example, to have an overhead view of the vehicle 20 and the target device 22, and can acquire overhead image information of the vehicle 20 and the target device 22. The imaging device 12 can photograph the vehicle 20 and the target device 22 within the range shown by the dashed line in the figure.

[0030] The imaging device 12 is positioned so as to acquire at least the shape of the top surface of the target device 22. It is also desirable that the imaging device 12 be positioned so as to acquire the shapes of the illumination surface and light-receiving surface of the target device 22. For example, by positioning the imaging device 12 to acquire an image of the vehicle 20 and the target device 22 viewed from above from the front, the imaging device 12 can acquire images of the top surface and the front (illumination surface and light-receiving surface) of the target device 22.

[0031] Hereafter, the term "top surface" will be used, but depending on the context, this "top surface" may be understood as the top surface of the housing of the target device 22 mounted on the vehicle 20. Similarly, the term "front surface" will be used, but depending on the context, this "front surface" may be understood as the surface of the housing of the target device 22 having an illumination surface and a light-receiving surface. In the housing of the target device 22, the illumination surface and the light-receiving surface may not intersect perpendicularly with the top surface. In this case, depending on the context, the "front surface" will not be limited to the surface that intersects perpendicularly with the top surface, but will be referred to as the "front surface" even if the intersection is at an angle other than 90 degrees.

[0032] The information processing system 1 obtains calibration parameters for the target device 22 by analyzing the image of the target device 22, or the image of the vehicle 20 and the target device 22, acquired by the imaging device 12, in the information processing device 10. In other words, the information processing device 10 acquires the image captured from the imaging device 12 and obtains calibration parameters for the target device 22 based on the acquired image of the target device 22, or the image of the vehicle 20 and the target device 22.

[0033] The parameters related to calibration may, for example, be external parameters of the target device 22. Alternatively, the parameters related to calibration may be data necessary to determine the parameters for calibrating the target device 22.

[0034] For example, this parameter may include information that includes at least one of the following: the position of the target device 22 relative to the vehicle 20, or the orientation of the target device 22 with respect to its axis. The orientation information may include, for example, at least one of the following: pitch angle, roll angle, or yaw angle with respect to a reference axis.

[0035] (First Embodiment)

[0036] Figure 2 is a flowchart showing the processing flow of an information processing system 1 according to one embodiment. Note that this flowchart is shown as an example of the processing flow, and it is possible to omit some of the processing, add processing that is not shown, or change the order of some of the processing without departing from the gist of this disclosure.

[0037] First, the imaging device 12 acquires an overhead image of the vehicle 20 and the target device 22 (S100). If necessary, the imaging device 12 can take an image by irradiating the LiDAR sensor with light in a frequency band that can be received by the LiDAR sensor and receiving the reflected light. The acquired image may be output as image information obtained in that frequency band, or it may be output as a depth map. By using a LiDAR sensor, highly accurate information can be obtained.

[0038] The imaging device 12 acquires, for example, an image and / or depth map (hereinafter referred to as "image, etc.") including the shape of the top surface and the front surface of the target device 22, and outputs it to the information processing device 10. If necessary, the imaging device 12 may store the information related to this image in a storage area. The output from the imaging device 12 to the information processing device 10 can use any interface that can operate appropriately.

[0039] The information processing device 10 extracts the shape of the top surface of the target device 22 from the image output from the imaging device 12 (S102). The information processing device 10 can also extract the shape of the front surface from the image output from the imaging device 12. Depending on the color of the target device 22 and the reflectivity of the imaging band, it may be difficult to obtain the shape from the image. In this case, the cover 24 can be used.

[0040] Figure 3 shows an example of a cover 24 according to one embodiment. The cover 24 is formed, for example, in part to reflect light in the frequency band that the LiDAR sensor of the imaging device 12 can receive with high efficiency. As shown in Figure 3, the cover 24 is formed to be highly reflective at least the edges and borders (solid lines) that form the top and front surfaces, as an example without limitation. By using this cover 24, the information processing device 10 can easily extract the shape of the top and front surfaces from the image acquired by the imaging device 12.

[0041] The cover 24 is shown as an example, and other configurations are possible. For example, the reflectivity may be high even in the dotted line area of ​​Figure 3, or at least only the area around the vertices may be highly reflective. Alternatively, the entire cover 24 may be highly reflective, or, as yet another example, only the edges may have high reflectivity, and the surface formed by the edges may have low reflectivity.

[0042] The information processing device 10 extracts the shape of the top (and front) surface from the image acquired from the imaging device 12. General methods can be used for this extraction. For example, the information processing device 10 can obtain the shape of the top (and front) surface by extracting edges from the image. For example, the information processing device 10 can obtain the shape of the top surface by referring to a depth map. For example, the information processing device 10 can obtain the shape by using a trained model that outputs the shape of the top surface when an image is input.

[0043] In this way, the information processing device 10 extracts the shape of the top surface, etc., from an image, etc., using an appropriate method. Figure 4 shows some examples of the shape of the top surface extracted from the acquired image. As shown in each, if the top surface of the target device 22 is rectangular, the shape of the top surface is extracted as a rectangle, trapezoid, parallelogram, or other distorted rectangle.

[0044] Furthermore, if the image acquired by the imaging device 12 contains pincushion distortion or barrel distortion, the information processing device 10 or the imaging device 12 may correct these distortions. Generally known methods can be used to correct these distortions. Alternatively, when correcting distortions, the information processing device 10 or the imaging device 12 may prepare a correction function using the parameters of the optical system mounted on the imaging device 12. The information processing device 10 can then perform subsequent processing using the corrected image.

[0045] Returning to Figure 2, the information processing device 10 calculates the centerline for the extracted top surface and other shapes (S104). Figure 5 shows an example where the centerline has been calculated and added to the extraction results in Figure 4. Simply put, the information processing device 10 can calculate the centerline by extracting the midpoint of each side and connecting the midpoints of opposite sides. Alternatively, the information processing device 10 may calculate the center point by using the point where the centerlines intersect.

[0046] The information processing device 10 calculates the position and orientation of the target device 22 based on the calculated reference point and center point information (S106). Here, the position and orientation of the target device 22 may be relative to the vehicle 20. The information processing device 10 obtains, for example, the position from the reference point set for the vehicle 20 to the center point, and the angle between the reference line set for the vehicle 20 and the center line.

[0047] The information processing device 10 can obtain a reference axis for calculating angles by, for example, calculating reference lines extending in the forward and backward directions of the vehicle 20. Based on these reference lines indicating the direction of the reference axis, the yaw angle ψ can be calculated.

[0048] Figure 6 is a top view of a vehicle 20 and a target device 22 according to one embodiment. The information processing device 10 calculates, for example, the relative position (xo, yo) of the center point Ol with respect to a reference point Oc defined for each type of vehicle 20. Based on the difference between the coordinates of the calculated center point Ol and the coordinates of the reference point Oc, the information processing device 10 can calculate the position of the target device 22 relative to the standard position of the vehicle 20.

[0049] The position (coordinates) of the reference point may be obtained, for example, by marking the vehicle 20 and capturing the image with the imaging device 12. In another, non-limiting example, the position of the reference point can be obtained based on the image acquired by the imaging device 12, based on rules defined for each vehicle type, or using a trained model that outputs a reference point when the vehicle type and image are input.

[0050] Figure 7 is a top view of a vehicle 20 and a target device 22 according to one embodiment. The information processing device 10 can, for example, calculate the angle of the center line of the target device 22 in the x-axis direction relative to a reference line defined for each type of vehicle 20 as the yaw angle ψ of the target device 22 with respect to the reference line.

[0051] Furthermore, the information processing device 10 can calculate the pitch angle of the target device 22 from the reference axis based on the distortion of the target device 22's shape in the top view. For example, if the shape of the target device 22, which should normally be a rectangle in the top view, is the third shape from the left in Figure 4, it can be seen that it has a negative pitch angle. The information processing device 10 can, for example, calculate the pitch angle θ from the degree of this distortion.

[0052] Since the imaging device 12 is positioned to capture images from a bird's-eye view of the vehicle 20 and the target device 22, the information processing device 10 can perform calculations after converting the images acquired by the imaging device 12 into a top view using perspective transformation before calculating the above-mentioned position and angle. These perspective transformations and the calculation of the pitch angle θ can be performed using known and appropriate methods.

[0053] The information processing device 10 may, in other, but not limited to, calculate the positional deviation of the target device 22 from the standard position relative to the vehicle 20. For example, the information processing device 10 may calculate the deviation from the center point of the target device 22 to point Ol, which is defined for each type of vehicle 20.

[0054] Furthermore, the information processing device 10 can calculate the roll angle φ from the standard orientation of the target device 22 based on the distortion of the shape of the upper surface. In addition, the information processing device 10 can calculate the height of the target device 22 from the ground by referring to a depth map. For example, the information processing device 10 can calculate the height information (z component) of the center point of the target device 22 from the ground based on the depth information of the point corresponding to the coordinates of the center point and the installation position information of the imaging device 12.

[0055] The information processing device 10 may store the position (x, y, z) and orientation (φ, θ, ψ) information obtained as described above in its memory. The information processing device 10 may also output the position and orientation information to the target device 22. The target device 22 can then perform calibration using these acquired position and orientation parameters.

[0056] As described above, according to the system of this embodiment, calibration parameters can be calculated by taking images of a target device equipped with a LiDAR sensor using an imaging device equipped with a LiDAR sensor. Calibration of the target device 22 having a LiDAR sensor must be achieved with very high precision due to the nature of LiDAR. In this calibration, by taking measurements using the imaging device 12 having a LiDAR sensor, it becomes possible to acquire various parameters with higher resolution in terms of length and angle than when using a normal camera.

[0057] In other words, by acquiring the parameters according to this embodiment, it becomes possible to achieve more accurate calibration of the target device 22. The target device 22 can use the various parameters calculated by the information processing device 10 to, for example, calculate external parameters related to the LiDAR sensor, or even perform calibration using a test board or the like placed on the front of the vehicle 20.

[0058] (Second Embodiment)

[0059] In the first embodiment described above, parameters related to position and orientation were obtained from the shape of the top surface, but the embodiments in this disclosure are not limited thereto. The information processing system 1 can calculate parameters with even greater accuracy by also using the shape of the front of the target device 22 captured by the imaging device 12. The information processing device 10 can improve the estimation accuracy of the roll angle φ by, for example, extracting the shape of the front of the target device 22 from an image captured by the imaging device 12.

[0060] Figure 8 is a front view of a vehicle 20 according to one embodiment. The height h and roll angle φ of the target device 22 are shown as shown in the figure. The height h may be the distance from the ground to a predetermined point on the target device 22, for example, the center point. Alternatively, the height h may be the height from a predetermined position on the vehicle, rather than from the ground. The roll angle φ represents the inclination around a predetermined axis, for example, the x-axis.

[0061] The information processing device 10 can extract the front shape of the target device 22 from images acquired from the imaging device 12, and calculate the height h and roll angle φ from the extracted front shape. The information processing device 10 can calculate the center line and center point of the front shape of the target device 22, for example, in the same manner as the processing for the top shape in the above embodiment.

[0062] The information processing device 10 can acquire height h information based on the position of the center point. The information processing device 10 can also acquire roll angle φ information based on the inclination of the center line. These parameters, height h and roll angle φ, can be calculated using known methods based on the extracted frontal shape of the target device 22.

[0063] Similar to the embodiments described above, the information processing device 10 can also perform a perspective transformation on the image acquired from the imaging device 12 so that it becomes a view of the vehicle 20 from the front, and acquire the respective parameters. Based on these acquired parameters, the information processing device 10 or the target device 22 can perform calibration of the target device 22.

[0064] According to this embodiment, for example, by using a front view shape that more readily shows changes in roll angle, instead of a top view where changes due to roll angle are less apparent, it is possible to calculate the roll angle with greater accuracy. Similarly, the accuracy of height information can also be improved by using a front view where changes are more readily apparent.

[0065] These parameters can also be estimated based on both the top view and the front view. Similarly, the pitch angle θ can also be calculated based on both the top view and the front view.

[0066] (Third embodiment)

[0067] In the embodiments described above, the state of the vehicle 20 was not explained, but it is also possible to calculate the parameters by correcting the state of the vehicle 20 using an external indicator. That is, in the embodiments described above, the parameters were obtained when the vehicle 20 was parked in an appropriate position, but in this embodiment, parking accuracy is required. Even if the parking accuracy is not good, the information processing device 10 can obtain sufficient accuracy for position information using the method of the embodiments described above, but the accuracy for attitude information may decrease.

[0068] In this embodiment, before employing the methods of each of the embodiments described above, the position and orientation of the vehicle 20 are corrected, and based on the corrected position and orientation of the vehicle 20, parameters related to the relative position and orientation of the target device 22 with respect to the vehicle 20 can be acquired with greater accuracy. The target device 22 is used to detect and measure distances to objects in a predetermined direction of the vehicle 20, for example, in front of it. For this reason, it is desirable to be able to acquire parameters of the target device 22 relative to the front of the vehicle 20 with greater accuracy.

[0069] Figure 9 is a top view showing an example of the arrangement of a vehicle 20 and a target device 22 according to one embodiment. Marks may be placed in the space where calibration is performed to allow confirmation that the vehicle 20 is properly parked. These marks can also be used to adjust the parameters used for calibration if the vehicle 20 is not properly parked.

[0070] In vehicle 20, for example, a mark may be placed on the hood as shown in the figure.

[0071] Based on the marks placed in space and the marks mounted on the vehicle 20, the information processing device 10 can acquire information related to the deviation of the vehicle 20 from a reference position (position information) and the deviation of the vehicle 20 from a reference angle (yaw angle Ψ of the vehicle 20) (attitude information) based on the image information acquired by the imaging device 12.

[0072] The information processing device 10 can correct the calibration parameters calculated in the above embodiment based on this information.

[0073] Figure 10 shows an example of a top view of the calibration space. The target board 30 is a board on which a test pattern is displayed for use in the calibration of the target device 22. In this way, the board is placed in the calibration space, and calibration is performed by imaging this board with the LiDAR sensor of the target device 22.

[0074] In the previously described embodiment, it was possible to acquire information on the relative position and orientation of the target device 22 with respect to the vehicle 20, and for example, by placing the target board 30 in an appropriate location relative to the vehicle 20, it was possible to achieve highly accurate calibration. On the other hand, in this embodiment, the board can be fixed in place, and calibration of the target device 22 can be performed based on the position and orientation of the vehicle 20 with respect to the target board 30.

[0075] Figure 9 is a top view showing an example of the position of a vehicle 20 in the imaging space by an imaging device 12 according to one embodiment. As shown in this figure, the imaging space may also be provided with a first mark 100 to indicate the position of the vehicle 20 and to confirm the parking position of the vehicle 20 from the image.

[0076] Furthermore, a second mark 200 can be applied to the vehicle 20 before imaging. This second mark 200 can be placed in a fixed position for each vehicle type, for example. The second mark 200 may be formed from a magnetic sheet or the like, for example. Using this second mark 200, the information processing device 10 can calculate a reference point for the vehicle 20.

[0077] The first mark 100 is arranged so that four of them form the vertices of a rectangle, but it is not limited to this arrangement. Any number of marks can be placed at positions where the vehicle 20 will stop appropriately, and / or at positions where the position of the vehicle 20 can be appropriately determined from the image, etc. Similarly, the second mark 200 can be placed in any number and at any position within a range that can appropriately indicate the reference point of the vehicle 20.

[0078] The star mark represents a reference position in the imaging space of the imaging device 12, set for each vehicle type of vehicle 20. Ideally, the position of the star mark is defined as the point at which proper calibration can be performed by stopping the vehicle 20 so that its position coincides with the position of the reference point of the vehicle 20 calculated from the second mark 200.

[0079] The information processing device 10 calculates the positional shift between the position of the star mark and the position of the reference point of the vehicle 20 calculated from the second mark 200, based on the image acquired by the imaging device 12. Simultaneously, it can calculate the yaw angle of the vehicle 20 relative to the imaging space of the imaging device 12 from the angle between the centerlines of the opposite sides of the rectangle formed by the first mark 100 and the centerline of the vehicle 20. As shown in the top view, the information processing device 10 can also estimate these positions and yaw angles from images captured from an overhead perspective.

[0080] Figure 10 shows an example of a space for achieving calibration. The information processing system 1 may optionally be configured as a system including a target board 30. With the target board 30 installed, the information processing system 1 may be configured to achieve proper calibration at the position specified by the first mark 100.

[0081] The target board 30, for example, displays a test pattern by printing, drawing, or other means. The target device 22 performs calibration using the test pattern. The target device 22 can acquire highly accurate calibration parameters based, for example, distance information to the test pattern (e.g., distance to a reference point) and attitude information.

[0082] In this configuration, taking into account the positional displacement and yaw angle of the vehicle 20 relative to the imaging space calculated as described above, the target device 22 can achieve more accurate calibration using parameters based on the position and orientation information of the target device 22 relative to the vehicle 20 calculated in the aforementioned embodiment.

[0083] Specifically, the target device 22 calculates parameters such as external parameters for the vehicle 20 from the position and orientation information of the vehicle 20, and performs calibration by imaging the target board 30 using these parameters, thereby correcting the parameters using the positional and orientational deviations of the vehicle 20 relative to space.

[0084] For example, the target device 22 can image the target board 30, correct the image using information about the displacement of the vehicle 20 relative to space that has already been acquired, and then perform highly accurate calibration of the corrected image based on the position and orientation information of the vehicle 20 calculated according to the respective embodiments described above.

[0085] As described above, according to the embodiments of this disclosure, even a slight deviation from the ideal placement in an environment where high-precision result calibration is performed can result in a large error. However, by using a LiDAR sensor that can accurately measure the position and orientation of the LiDAR sensor itself, the accuracy of the calibration can be greatly improved.

[0086] In any of the embodiments described above, vehicle-specific values ​​may be stored in the memory area of ​​the information processing device 10 or the memory area of ​​the target device 22. The information processing device 10 and / or the target device 22 can also adjust the parameters using these vehicle-specific values, thereby enabling proper calibration in the target device 22.

[0087] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and components may be omitted, substituted, or modified without departing from the spirit of the invention. The embodiments and variations thereof described herein are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0088] 1: Information processing system, 10: Information processing equipment, 12: Imaging device, 100: First mark, 20: Vehicles, 200: Second mark, 22: Target device, 24: Cover, 30: Target board

Claims

1. A device that acquires data for calibrating a target device which is a device equipped with a processor and having a LiDAR (Light Detection and Ranging) sensor mounted on a vehicle, The aforementioned processor, Based on images of the vehicle and the target device acquired by an imaging device having a LiDAR sensor positioned outside the vehicle to capture images from a position overlooking the vehicle and the target device, parameters related to calibration in the target device are calculated. Information processing device.

2. The aforementioned processor, From the image acquired from the aforementioned imaging device, the shape of the upper surface of the target device is extracted. The information processing apparatus according to claim 1.

3. The aforementioned processor, Based on the shape of the upper surface, information relating to the position and orientation of the target device is acquired. The information processing apparatus according to claim 2.

4. The aforementioned processor, Based on the shape of the upper surface of the target device, the position of the target device with respect to the vehicle, the yaw angle with respect to the reference axis, and the pitch angle with respect to the reference axis are obtained. The information processing apparatus according to claim 3.

5. The aforementioned processor, Based on the information relating to the position and orientation of the target device, the parameters are calculated. The information processing apparatus according to claim 3.

6. The aforementioned processor, From the image acquired from the imaging device, the shapes of the emission surface and light-receiving surface of the target device are extracted. The information processing apparatus according to claim 1.

7. The aforementioned processor, Based on the shapes of the injection surface and the light-receiving surface, information relating to the position and orientation of the target device relative to the vehicle is acquired. The information processing apparatus according to claim 6.

8. The aforementioned processor, Based on the shape of the light-emitting surface and light-receiving surface of the target device, the height at which the target device is installed and the roll angle with respect to the reference axis are obtained. The information processing apparatus according to claim 7.

9. The aforementioned processor, Information relating to the vehicle is obtained from the aforementioned image, Based on the information relating to the said vehicle, information relating to the position and orientation of the said vehicle is acquired. The information processing apparatus according to claim 1.

10. The aforementioned processor, Based on the information relating to the vehicle, the position of the vehicle from a reference point and the yaw angle of the vehicle with respect to the reference axis are obtained. The information processing apparatus according to claim 9.

11. The aforementioned processor, Based on the vehicle's position relative to a reference point and its yaw angle relative to the reference axis, the calibration parameters are corrected. The information processing apparatus according to claim 10.

12. The vehicle and the target device, which is a device having a LiDAR sensor mounted on the vehicle, are captured from a position overlooking the vehicle by an imaging device, which is a device having a LiDAR sensor located outside the vehicle. The processor obtains images of the vehicle and the target device from the image information acquired by the imaging device. The processor calculates parameters related to the calibration of the target device based on the image. Information processing methods.

13. The processor transmits the parameters to the target device. The information processing method according to claim 12.

14. A system for calibrating a target device which is a device having a LiDAR sensor mounted on a vehicle, An imaging device having a LiDAR sensor that takes images from a position overlooking the vehicle and the target device, A processor that calculates calibration parameters for the target device from images of the vehicle and the target device output from the imaging device, An information processing system equipped with the following features.

15. The processor transmits the parameters to the target device. The information processing system according to claim 14.

Images