Ranging device and control method

Abstract

This ranging device comprises: a LiDAR (300) that has a scanner and measures the distance to points within a scanning range; a camera (200) that captures an image of an imaging range at least partially overlapping the scanning range; and a control device (100) that determines a calibration parameter for calibrating the scanning range by the LiDAR. The control device acquires a distance to at least one point in a region of interest in the image by pre-ranging using the LiDAR, and determines a calibration parameter on the basis of the distance to the at least one point. The LiDAR uses the determined calibration parameter to measure the distance to points in the region of interest again.

Description

Distance measuring device and control method

[0001] The present disclosure relates to a distance measuring device and a control method.

[0002] Conventionally, there has been a need to use both an image and the distance to an object in the image. For example, Patent Document 1 discloses a technology that uses a ToF sensor and a camera to superimpose a movement trajectory of a person captured by the ToF sensor on an image captured by the camera. Patent Document 1 describes performing calibration so that a human detection frame captured by the camera image matches a human detection frame captured by the ToF sensor.

[0003] Japanese Patent Application Laid-Open No. 2022-101310

[0004] As in the calibration example described in Patent Literature 1, when using a camera for capturing an image and a sensor for measuring distance, there is a problem that the installation positions of the cameras and sensors are different, making it impossible to simply superimpose the image and the distance measurement results. Therefore, when a pixel in an image is specified, the measured distance corresponding to that pixel may not be accurate. Therefore, an object of the present disclosure is to provide a distance measuring device and the like that can more accurately measure the distance to an object in an image.

[0005] A ranging device according to one aspect of the present disclosure comprises a LiDAR having a scanner and measuring the distance to each point within a scan range, a camera capturing an image of a shooting range that at least partially overlaps with the scan range, and a control device that determines calibration parameters for calibrating the scan range by the LiDAR, wherein the control device obtains the distance to at least one point in a region of interest in the image by pre-ranging using the LiDAR and determines the calibration parameters based on the distance to the at least one point, and the LiDAR re-measures the distance to each point in the region of interest using the determined calibration parameters.

[0006] Furthermore, a control method according to one aspect of the present disclosure is a control method for controlling a LiDAR by outputting calibration parameters for calibrating a scan range by a LiDAR having a scanner that measures the distance to each point within the scan range, wherein the method obtains the distance of at least one point in a region of interest within an image of a shooting range that at least partially overlaps with the scan range from a pre-ranging result using the LiDAR, and determines the calibration parameters used to re-measure the distance to each point in the region of interest based on the distance of the at least one point and outputs the calibration parameters to the LiDAR.

[0007] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0008] According to the present disclosure, it is possible to more accurately measure the distance to an object in an image.

[0009] FIG. 1 is a schematic diagram showing an example of installation of a distance measuring device according to an embodiment. FIG. 2 is a block diagram showing the functional configuration of a control device according to an embodiment. FIG. 3 is a diagram for explaining differences in distance measurement results in an installation example according to an embodiment. FIG. 4 is a diagram for explaining differences in distance measurement results in an installation example according to an embodiment. FIG. 5 is a flowchart showing an example of operation of a distance measuring device according to an embodiment. FIG. 6 is a diagram showing an example of a display unit of a distance measuring device according to an embodiment. FIG. 7 is a diagram showing an example of a calibration table of a distance measuring device according to an embodiment.

[0010] (Knowledge that forms the basis of the disclosure) Conventionally, there has been a need to use both an image and the distance to an object in the image. For example, Patent Document 1 discloses a technology that uses a ToF sensor and a camera to superimpose a person's movement trajectory captured by the ToF sensor on an image captured by the camera. Patent Document 1 describes performing calibration so that the person detection frame captured by the camera image matches the person detection frame captured by the ToF sensor. As in this example, when using a camera for capturing an image and a sensor for measuring distance (hereinafter also referred to as a distance sensor), there is a problem in that the installation positions of the camera and the sensor for measuring distance are different, making it impossible to simply superimpose the image and the distance measurement result.

[0011] One example of a device that may suffer from the above-described problem is a distance measuring device that uses a camera and a distance measuring sensor to specify a portion of an image captured by the camera and measure the distance to that portion. With such a distance measuring device, when measuring the distance to an object in the image, the pixels of the object in the image must match the pixels of the object in the distance measurement results. For example, when a pixel in the image is specified, the measured distance corresponding to that pixel will not correspond to the specified pixel, resulting in an inaccurate distance. In other words, the distance measuring device cannot be used to specify an object in the image and measure the distance to that object.

[0012] As will be described in detail later, the closer an object is to the camera and the ranging sensor, the greater the impact of a mismatch between the pixels of the object in the image and the pixels of the object in the ranging results. Therefore, in this disclosure, in order to provide a ranging device or the like that can more accurately measure the distance to an object in an image, the ranging sensor pre-measures the distance to the object, and based on the results of the pre-measurement, outputs calibration parameters for calibrating the mismatch between the pixels in the image and the pixels in the ranging results for that distance. By resetting the ranging sensor based on the output calibration parameters, it becomes possible to measure the distance to the object more accurately.

[0013] A more specific outline of the present disclosure is as follows.

[0014] A ranging device according to a first aspect of the present disclosure comprises a LiDAR having a scanner that measures the distance to each point within a scan range, a camera that captures an image of a shooting range that at least partially overlaps with the scan range, and a control device that determines calibration parameters for calibrating the scan range by the LiDAR, wherein the control device obtains the distance to at least one point in a region of interest in the image by pre-ranging using the LiDAR and determines calibration parameters based on the distance to the at least one point, and the LiDAR re-measures the distance to each point in the region of interest using the determined calibration parameters.

[0015] With such a distance measuring device, the distance to the area of ​​interest can be used by pre-distance measurement, and calibration parameters for calibrating the scan range can be determined according to that distance. Then, by calibrating the scan range using the determined calibration parameters, the discrepancy between the area of ​​interest on the image and the area of ​​interest in the distance measurement results can be reduced in subsequent distance measurements. This makes it possible to use the distance measuring device in such a way that the area of ​​interest shown in the image can be specified and the distance to an object in that area of ​​interest can be measured. This makes it possible to more accurately measure the distance to an object in the image.

[0016] In addition, a distance measuring device according to a second aspect is the distance measuring device described in the first aspect, in which the control device determines the calibration parameter by selecting one calibration parameter from a calibration table containing multiple calibration parameter candidates prepared in advance for the distance of at least one point.

[0017] According to this, the calibration parameter can be determined by using the distance to the area of ​​interest obtained by pre-ranging and selecting one from a calibration table containing multiple calibration parameter candidates prepared in advance for each distance.

[0018] A distance measuring device according to a third aspect is the distance measuring device according to the first or second aspect, in which the control device displays the acquired image and accepts designation of a region of interest from the user for the displayed image.

[0019] This allows an image to be displayed and allows a user to specify a region of interest for the image.

[0020] A distance measuring device according to a fourth aspect is the distance measuring device according to any one of the first to third aspects, in which the pre-distance measuring has a lower resolution than the second distance measuring using the calibration parameters.

[0021] This has the advantage that it is easy to avoid the amount of data becoming large and the time required for pre-distance measurement being long, which would be caused by providing a resolution higher than necessary for pre-distance measurement.

[0022] Furthermore, a ranging device according to a fifth aspect is a ranging device according to the second aspect or the third or fourth aspect that cites the second aspect, in which each of the multiple calibration parameter candidates corresponds to each of multiple distance ranges that are separated according to the distance of at least one point.

[0023] According to this, as the plurality of calibration parameter candidates, it is possible to use calibration parameter candidates that correspond to a plurality of distance ranges according to the distance measurement results of the pre-distance measurement.

[0024] A distance measuring device according to a sixth aspect is the distance measuring device according to the fifth aspect, wherein the range width of the plurality of distance ranges is narrower as the distance from the camera is closer.

[0025] This allows the calibration parameter candidates to be associated with a plurality of distance ranges in greater detail as the distance from the camera becomes closer.

[0026] In addition, a ranging device according to a seventh aspect is a ranging device described in any one of the first to sixth aspects, in which the center of the camera's field of view and the center of the LiDAR scanning range intersect at a distance that is 50 times or more the installation distance between the camera and the LiDAR.

[0027] This makes it possible to more accurately measure the distance to objects in the image when the camera and LiDAR are installed such that the center of the camera's field of view and the center of the LiDAR's scanning range intersect at a distance that is 50 times or more the installation distance between the camera and LiDAR.

[0028] In addition, the ranging device of the eighth aspect is a ranging device described in any one of the first to seventh aspects, and at least one point is included in the half of the area of ​​interest that is closer to the LiDAR.

[0029] This allows the distance to at least one point included in the half of the area of ​​interest that is closer to the LiDAR to be used as the distance measurement result of the pre-ranging.

[0030] In addition, a control method according to a ninth aspect is a control method for controlling a LiDAR by outputting calibration parameters for calibrating a scan range by a LiDAR having a scanner that measures the distance to each point within the scan range. The control method obtains the distance of at least one point in a region of interest within an image of a shooting range that at least partially overlaps with the scan range from a pre-ranging result using the LiDAR, and determines calibration parameters to be used to re-measure the distance to each point in the region of interest based on the distance of the at least one point and outputs the calibration parameters to the LiDAR.

[0031] This can provide the same effects as the distance measuring device described above.

[0032] Furthermore, these comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0033] Hereinafter, embodiments will be described in detail with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not recited in independent claims will be described as optional components. Note that each figure is a schematic diagram and is not necessarily an exact illustration. Furthermore, in each figure, substantially identical components are assigned the same reference numerals, and duplicated descriptions may be omitted or simplified.

[0034] (Embodiment) [Configuration] First, an overview and configuration of a distance measuring device according to an embodiment will be described. FIG. 1 is a schematic diagram showing an example of installation of a distance measuring device according to an embodiment. FIG. 1 shows the layout of each component of the distance measuring device when viewed from a direction along the imaging direction of a camera 200 and a LiDAR (Light Detection and Ranging) 300 (a distance measuring sensor) included in the distance measuring device. As shown in the figure, the distance measuring device includes the camera 200, the LiDAR 300, and a control device. The camera 200 is a device that captures an image within an imaging range (in other words, within a field of view or angle of view). The camera 200 uses a photoelectric conversion element to receive light arriving from each position on a two-dimensional surface within the imaging range, accumulates electric charges, and generates an image corresponding to the two-dimensional surface. The image generated by the camera 200 may be a color image, a grayscale image, or a monochrome image. Camera 200 may be configured to receive any of visible light, infrared light, and ultraviolet light, as long as the received light can be visualized by the generated image.

[0035] The LiDAR 300 is a distance measurement sensor that measures the distance to each position in three-dimensional space within the scan range and generates a distance point cloud image using the distance measurement results. The LiDAR 300 emits laser light, measures the distance to an object based on the reflected light, and generates a distance point cloud image by converting the distance into a brightness value or color hierarchy of each pixel. The LiDAR 300 has a scanner to enable scanning. The scanner is a device that changes the direction of light emitted from the LiDAR 300. The scanner can be any existing scanner, such as a galvanometer scanner or a polygon scanner.

[0036] The control device 100 is a device used to control the LiDAR 300. The control device 100 is realized by executing a predetermined program using a processor, a memory, and various input / output hardware, and performs the functions of each functional unit described below. FIG. 2 is a block diagram showing the functional configuration of a control device according to an embodiment. As shown in FIG. 2, the control device 100 includes, as functional units, an image acquisition unit 101, a reception unit 102, a ranging result acquisition unit 103, a storage unit 104, an output unit 105, and a display unit 106.

[0037] The image acquisition unit 101 is a functional unit that communicates with the camera 200 and acquires images captured by the camera 200 .

[0038] The reception unit 102 is realized by, for example, an interface device such as a touch panel, a mouse, and a keyboard, and a signal processing function on software that processes input from the interface device. The reception unit 102 is a functional unit that receives a designation of a region of interest within an image. For example, when an image is displayed, the reception unit 102 receives a selection of a range of coordinates of at least a portion of the displayed image as the designation. As a result, the reception unit 102 outputs pixels within the image that correspond to the selected coordinates. The reception unit 102 is also a functional unit that receives other inputs from the user. In this case, the reception unit 102 receives an input that specifies the operation settings of the ranging device, etc., as other inputs from the user.

[0039] The distance measurement result acquisition unit 103 is a functional unit that communicates with the LiDAR 300 and acquires the distance measurement results by the LiDAR 300, i.e., the distance point cloud image.

[0040] The storage unit 104 is a functional unit realized by a semiconductor memory and a software memory controller. The storage unit 104 is a storage device for storing a calibration table including candidates for a plurality of calibration parameters, which will be described later.

[0041] The output unit 105 is a functional unit that determines and outputs one calibration parameter from the calibration table (i.e., from a plurality of calibration parameter candidates) based on the distance measurement result.

[0042] The display unit 106 is a functional unit that displays the acquired images and distance measurement results. The display unit 106 includes, for example, a display module that is overlaid on a touch panel that serves as the reception unit 102, and is also used to receive the displayed images and distance measurement results, as well as direct inputs from the user via various software buttons.

[0043] Returning to FIG. 1 , when arranging the camera 200 and the LiDAR 300, for example, as shown in FIG. 1A , the LiDAR 300 may be arranged at a position offset from the camera 200 in the horizontal direction (corresponding to the left-right direction on the paper) but not in the vertical direction (corresponding to the up-down direction on the paper) relative to the camera 200. More specifically, the position is as shown in FIGS. 3 and 4 . FIGS. 3 and 4 are diagrams for explaining the difference in distance measurement results in an installation example according to the embodiment. As shown in FIG. 3 , an example will be described in which the LiDAR 300 is arranged on the right side of the camera 200 in the horizontal direction (in other words, the camera 200 is arranged on the left side of the LiDAR 300). Assume that distance ranges A, B, C, D, and E are set in the direction away from the camera 200. In distance range E, the imaging range of camera 200, shown by the thick line in the figure, can be overlapped with the scanning range by tilting the scanning range of LiDAR 300, shown by the dashed line in the figure, slightly to the left.

[0044] In distance range D, the scanning range of LiDAR 300 can be tilted further to the left with respect to the imaging range of camera 200 than in distance range E, thereby allowing the imaging range and the scanning range to overlap. In distance range C, the scanning range of LiDAR 300 can be tilted further to the left with respect to the imaging range of camera 200 than in distance range D, thereby allowing the imaging range and the scanning range to overlap. In distance range B, the scanning range of LiDAR 300 can be tilted further to the left with respect to the imaging range of camera 200 than in distance range C, thereby allowing the imaging range and the scanning range to overlap. In distance range A, the scanning range of LiDAR 300 can be tilted further to the left with respect to the imaging range of camera 200 than in distance range B, thereby allowing the imaging range and the scanning range to overlap. More precisely, as shown in FIG. 4 , the angle of the right end of the scanning range, which corresponds to the right end of the imaging range, changes based on the distance to the object 99, as shown by the circle plot. Furthermore, the angle of the left end of the scanning range, which corresponds to the left end of the imaging range, changes based on the distance to the object 99, as shown by the plot of square marks.

[0045] In this way, the closer the distance (to the object 99) from the camera 200, the more the scanning range of the LiDAR 300 deviates toward the side (right side) where the LiDAR 300 is installed relative to the imaging range of the camera 200. Therefore, the closer the distance from the camera 200, the more calibration is required to tilt the scanning range of the LiDAR 300 toward the side (left side) where the camera 200 is installed relative to the LiDAR 300 so that the ranges overlap. In this case, if the camera 200 and the LiDAR 300 are housed in the same housing and the installation interval is fixed, the angle to be calibrated for each distance range can be set in advance based on the fixed installation interval.

[0046] Furthermore, even if the camera 200 and the LiDAR 300 are separate, as long as the installation interval can be input after installation, the angle to be calibrated for each distance range can be calculated based on the input installation interval. Based on this knowledge, in this embodiment, the preset angle to be calibrated for the LiDAR 300 is determined by identifying the distance range in which the object 99 to be measured exists. For example, in the example shown in the figure, the object 99 exists in distance range D. In other words, since the distance to the object 99 belongs to distance range D, the scan range of the LiDAR 300 is calibrated to an angle range that is to the left of distance range E and to the right of distance range C. Note that, as shown in the figure, the closer the distance from the camera 200, the narrower the range width for each distance range. This corresponds to the fact that the closer the distance from the camera 200, the greater the nonlinear range deviation. In other words, by narrowing the range width for each distance range as the distance from the camera 200 becomes shorter, the effect of easily suppressing the influence of nonlinear range deviation is achieved.

[0047] As shown in (b) of FIG. 1, LiDAR 300 may be disposed at a position that is not offset in the horizontal direction of camera 200 but is offset in the vertical direction of camera 200. In this case, the closer the distance from camera 200, the more the scanning range of LiDAR 300 is tilted toward the side where camera 200 is installed (upper side) relative to LiDAR 300 to perform calibration to overlap the ranges. Also, as shown in (c) of FIG. 1, LiDAR 300 may be disposed at a position that is offset in the horizontal direction of camera 200 but is offset in the vertical direction of camera 200. In this case, the closer the distance from camera 200, the more the scanning range of LiDAR 300 is tilted toward the side where camera 200 is installed (lower right side) relative to LiDAR 300 to perform calibration to overlap the ranges. Similarly, if the up / down / left / right orientations of camera 200 and LiDAR 300 are reversed, the direction of change in angle during calibration may be reversed.

[0048] However, the center of the imaging range of camera 200 and the center of the scanning range of LiDAR 300 intersect at a distance of 50 times or more the above-mentioned installation interval. Beyond this intersection point, the distance between their centers increases again, to be precise, but on the image or range image, the increase is within a few pixels, and it can be considered that the center of the imaging range of camera 200 and the center of the scanning range of LiDAR 300 essentially coincide beyond the intersection point. In this way, by making the center of the imaging range and the center of the scanning range coincide at a certain distance from camera 200 and LiDAR 300, there is an advantage in that it is easy to avoid placing camera 200 and LiDAR 300 at an installation interval that cannot be addressed by calibration.

[0049] [Operation] Next, the operation of the distance measuring device described above will be described with reference to Figures 5 to 7, focusing on the operation of the control device 100. Figure 5 is a flowchart showing an example of the operation of the distance measuring device according to the embodiment. Figure 6 is a diagram showing an example of the display unit of the distance measuring device according to the embodiment. Figure 7 is a diagram showing an example of a calibration table of the distance measuring device according to the embodiment.

[0050] First, the control device 100 initializes the distance measurement conditions based on user input or previous settings (e.g., settings at the time of shipment of the distance measurement device) (step S11). Then, the user presses the "pre-distance measurement" button shown in FIG. 6 to initiate pre-distance measurement, which involves acquiring an image by photographing the image capture range with the camera 200 and acquiring a distance image by measuring the scan range with the LiDAR 300 (step S12). Because this distance measurement is performed to determine the distance range based on the distance to the object 99, some distance deviation in the distance measurement results is acceptable. This pre-distance measurement is performed continuously, and multiple consecutive images and distance images are acquired in the time domain. The distance images acquired here only need to have a resolution sufficient to allow the user to specify a region of interest. Furthermore, because continuous distance measurement in the time domain is required, they are acquired at a relatively low resolution. Specifically, distance measurement is performed for every five pixels in both the vertical and horizontal directions to acquire a distance image with a resolution of 1 / 25.

[0051] Next, the acquired image and a low-resolution distance image as the distance measurement result are displayed (step S13). As shown in Fig. 6, the display unit 106 displays a "camera image" which is a distance image and a "point cloud image" which is a low-resolution distance image. Thereafter, it is determined whether the high-resolution mode has been selected by determining whether the user has pressed the "high-resolution" button shown in Fig. 6 (step S14). If the "high-resolution" button has not been pressed and it is determined that the high-resolution mode has not been selected (No in step S14), the process returns to step S13 and is repeated.

[0052] On the other hand, if it is determined that the "High Definition" button has been pressed and the high-definition mode has been selected (Yes in step S14), the display unit 106 freezes the displayed image at the image captured when the "High Definition" button was pressed and switches to a still image (step S15). Then, the user selects at least a portion of the still image, as shown by the selection range 99a in FIG. 6, and the reception unit 102 receives the user's designation of a region of interest (step S16). For example, if the user presses the "Region Selection" button shown in FIG. 6, the selected region is confirmed as the designated region of interest. In other words, by determining whether the "Region Selection" button has been pressed, it is determined whether reception has been completed (step S17). If the "Region Selection" button has not been pressed and it is determined that reception has not been completed (No in step S17), the process returns to step S16 and is repeated.

[0053] On the other hand, if the "Select Area" button is pressed and it is determined that the acceptance is complete (Yes in step S17), a distance image of the area of ​​interest in the pre-ranging is acquired (step S18). The output unit 105 acquires the distance of at least one point in the area of ​​interest. For example, the output unit 105 may acquire the average value of the distances of all pixels in the area of ​​interest as the distance of the at least one point, or may acquire the distance of a representative point, such as the pixel closest to all pixels in the area of ​​interest, as the distance of the at least one point. Furthermore, since calibration has not yet been performed in the pre-ranging, the distance of the area of ​​interest on the camera 200 side relative to the LiDAR 300 may not be accurate. Conversely, the output unit 105 may narrow down the area of ​​interest in the image to the area closer to the LiDAR 300, determine the average value of the distances of all pixels, or determine the representative point, and acquire the distance.

[0054] Next, the output unit 105 accesses the storage unit 104 and refers to the calibration table (step S19). The output unit 105 determines and outputs calibration parameters using the distance of at least one point acquired above (step S20). As shown in FIG. 7, the calibration table includes multiple calibration parameter candidates. Specifically, one calibration parameter is prepared in advance for each distance range. As shown in the figure, the calibration parameters include scan angles defined by the left and right ends of the scan range (because the camera 200 and the LiDAR 300 are offset left and right in this case) for the distance range.

[0055] The calibration parameters also include the magnitude of the scan angle corresponding to one pixel in the left-right direction of the scan range relative to the distance range. For example, as shown in FIG. 3, when the output unit 105 acquires a distance corresponding to distance range D from the region of interest where the object 99 is present, it references the calibration table and determines and outputs calibration parameters for scanning a scan angle range of -33.172 to 26.417 (deg) while changing the angle by 0.03104 deg per pixel. The LiDAR 300 then resets the ranging conditions using the output calibration parameters (step S21). Specifically, in this resetting, the output calibration parameters correspond to the left and right ends of the camera's field of view, and a portion of the scan range corresponding to the region of interest is determined as the scan range for the region of interest. In this way, the control device 100 outputs the determined scan range for the region of interest to the LiDAR 300 and controls the scan range of the LiDAR 300.

[0056] Thereafter, the LiDAR 300 scans the determined scan range of the region of interest to again measure the distance to each point in this region of interest (step S22). The second ranging here is not low-resolution like the pre-ranging, but rather a distance image with a number of pixels corresponding to the number of pixels in the image is acquired. When scanning the scan range of the region of interest in the second ranging, the LiDAR 300 scans only the scan range of the region of interest and outputs a ranging result corresponding to the scan range of the region of interest. Alternatively, the second ranging may scan a wider scan range that includes the scan range of the region of interest, and extract and output a ranging result corresponding to the scan range of the region of interest.

[0057] The display unit 106 displays the image thus acquired and the distance measurement results (step S23).

[0058] Other Embodiments Although the embodiments have been described above, the present disclosure is not limited to the above-described embodiments.

[0059] For example, the distance measuring device described in the above embodiment may be realized as a single device including all of the components, or may be realized by allocating each function to multiple devices and coordinating these multiple devices. In the latter case, the device corresponding to the control device may be an information processing device such as a smartphone, a tablet terminal, or a PC. Also, the distance measuring device may include a camera incorporating the function of the control device, or a LiDAR incorporating the function of the control device.

[0060] In the above-described embodiment, the processing performed by a specific processing unit may be performed by another processing unit. The order of multiple processing operations may be changed, or multiple processing operations may be performed in parallel.

[0061] In the above-described embodiments, each component may be realized by executing a software program suitable for that component, or by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0062] Furthermore, each component may be realized by hardware. For example, each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0063] Furthermore, the general or specific aspects of the present disclosure may be realized as an apparatus, a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, etc. Furthermore, the general or specific aspects of the present disclosure may be realized as any combination of an apparatus, a system, a method, an integrated circuit, a computer program, and a recording medium.

[0064] For example, the present disclosure may be realized as a control method executed by a computer, or as a program for causing a computer to execute the control method. The present disclosure may also be realized as a computer-readable non-transitory recording medium on which such a program is recorded.

[0065] In addition, this disclosure also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art would think of, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope that does not deviate from the intent of this disclosure.

[0066] The present disclosure is useful in measuring distances across a two-dimensional surface accompanied by an image.

[0067] 99 Object 99a Selection range 100 Control device 101 Image acquisition unit 102 Reception unit 103 Distance measurement result acquisition unit 104 Storage unit 105 Output unit 106 Display unit 200 Camera 300 LiDAR

Claims

1. A distance measuring device having a scanner, a LiDAR that measures the distance to each point within a scanning range, a camera that captures an image of a shooting range that at least partially overlaps with the scanning range, and a control device that determines calibration parameters for calibrating the scanning range by the LiDAR, wherein the control device acquires the distance of at least one point in a region of interest in the image by pre-distance measurement using the LiDAR, determines the calibration parameters based on the distance of the at least one point, and the LiDAR re-measures the distance to each point in the region of interest using the determined calibration parameters.

2. The distance measuring device according to claim 1, wherein the control device determines by selecting one calibration parameter from a calibration table including a plurality of candidates for the calibration parameters prepared in advance for each distance of the at least one point.

3. The distance measuring device according to claim 1 or 2, wherein the control device displays the acquired image and accepts designation of the region of interest from the user for the displayed image.

4. The distance measuring device according to claim 1 or 2, wherein the pre-distance measurement has a lower resolution than the re-distance measurement using the calibration parameters.

5. The distance measuring device according to claim 2, wherein each of the plurality of candidates for the calibration parameters corresponds to each of a plurality of distance ranges delimited according to the distance of the at least one point.

6. The distance measuring device according to claim 5, wherein the plurality of distance ranges have a narrower range width as the distance from the camera is closer.

7. The distance measuring device according to claim 1 or 2, wherein the center of the field of view of the camera and the center of the scanning range of the LiDAR intersect at a distance of 50 times or more the installation interval between the camera and the LiDAR.

8. The distance measuring device according to claim 1 or 2, wherein the at least one point is included in the region of the region of interest that is on the side closer to the LiDAR.

9. A control method for controlling the LiDAR by outputting calibration parameters for calibrating the scan range by the LiDAR that has a scanner and measures the distance to each point within the scan range, the method comprising: obtaining the distance of at least one point in a region of interest in an image of a shooting range that at least partially overlaps with the scan range from a pre-measurement result using the LiDAR; determining the calibration parameters used to re-measure the distance to each point in the region of interest based on the distance of the at least one point and outputting the calibration parameters to the LiDAR.

Images