Laser scanning device, laser scanning method, and laser scanning program

Abstract

To enable setting of appropriate scanning conditions according to a scanning target in laser scanning.SOLUTION: A laser scanner 100 for performing laser scanning is provided, the laser scanner comprising a camera 107 serving as image capturing means, an optical axis deflection unit 208 serving as means of performing laser scanning, a depth information acquisition unit 202 configured to use an image captured by the camera 107 to acquire depth information of a target object in the captured image, and a scanning condition setting unit for setting laser scanning conditions for the target object based on the depth information.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to laser scanning. [Background technology]

[0002] A laser scanning device is known (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent No. 6877946 Summary of the Invention [Problem to be solved by the invention]

[0004] When performing laser scanning of a specific target, it is necessary to set scanning conditions according to the shape and distance of the target. Specifically, it is desirable to adjust the scanning range and scanning density according to the shape of the target and the distance to the target. In this context, an object of the present invention is to obtain a laser scanning technology that can set appropriate scanning conditions according to the scanning target. [Means for solving the problem]

[0005] The present invention is a laser scanning device that includes an imaging means, a laser scanning means, a means for obtaining depth information of an object depicted in an image based on the image captured by the imaging means, and a means for setting conditions for the laser scan of the object based on the depth information.

[0006] In the present invention, the objects include a first object and a second object, and a means for correcting the depth information obtained from an image of the second object based on the result of a laser scan of the first object is provided. In this embodiment, it is preferable that the second object has an appearance similar to that of the first object. It is also preferable that the second object is an object having depth information closest to that of the first object.

[0007] In the present invention, a preferred aspect is that a laser scan is performed on the shooting range of the shooting means, the laser scan is performed at a scan density smaller than the scan density of the laser scan on the target, and the depth information is corrected based on the results of the laser scan on the shooting range of the shooting means.

[0008] The present invention is a method for performing laser scanning, which obtains depth information of an object shown in a captured image based on the captured image, and sets conditions for laser scanning for the object based on the depth information.The present invention is also a program for laser scanning for performing laser scanning, which causes a computer to obtain depth information of an object shown in the captured image based on the captured image, and sets conditions for laser scanning for the object based on the depth information. Effect of the Invention

[0009] According to the present invention, a laser scanning technique is obtained that allows appropriate setting of scanning conditions according to a scanning target. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is an external view of a laser scanner. [Diagram 2] FIG. 1 is a conceptual diagram illustrating the principle of laser scanning. [Diagram 3] FIG. 1 is a block diagram of a laser scanner. [Figure 4]FIG. 13 is an image diagram showing an example of a captured image. [Diagram 5] 11 is a flowchart illustrating an example of a processing procedure. [Figure 6] 11 is a flowchart illustrating an example of a processing procedure. [Figure 7] FIG. 13 is an image diagram showing an example of a scan pattern. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] 1. First embodiment (Laser scanner overview) 1 shows the appearance of a laser scanner (laser scanning device) 100. Laser scanner 100 includes a tripod 101 serving as a leg, a base unit 102 fixed to the upper part of tripod 101, a horizontal rotation unit 103 serving as a rotating body capable of horizontal rotation (rotation about a vertical axis) on base unit 102, and a support unit 104 fixed onto horizontal rotation unit 103.

[0012] The support unit 104 has a pair of vertical parts that stand up vertically and are arranged facing each other with a space between them in the horizontal direction, and the vertical rotation unit 105 is held between the pair of vertical parts in a state where it can rotate vertically. Here, the vertical rotation is rotation about an axis that extends in the horizontal direction. The vertical rotation is synonymous with the elevation angle rotation and depression angle rotation.

[0013] The horizontal rotation of the horizontal rotation unit 103 and the vertical rotation of the vertical rotation unit 105 are driven by a motor, and the rotation angle is measured by an encoder. It is also possible to perform the horizontal rotation of the horizontal rotation unit 103 and the vertical rotation of the vertical rotation unit 105 manually.

[0014] An operation panel (touch panel display) (not shown) for operating the laser scanner 100 is disposed on the back of the vertical rotation unit 105. The laser scanner 100 can also be operated using a terminal (remote control). In this case, wireless communication is performed between the terminal and the laser scanner 100. A smartphone or a tablet computer can also be used as the terminal.

[0015] The vertical rotation unit 105 includes an optical unit 106, which is an optical system that emits and receives laser scanning light. The laser scanner 100 can perform laser scanning of a specific range without moving the horizontal rotation unit 103 and the vertical rotation unit 105.

[0016] The principle of this laser scanning will be explained below. This laser scanning is described in Japanese Patent No. 6616077 and Japanese Patent No. 6877946.

[0017] 2 is a diagram showing the principle of laser scanning. An optical axis deflection unit 208 is disposed inside the optical unit 106 in FIG.

[0018] Optical axis deflection unit 208 has prisms A and B, and prisms A' and B'. Prisms A and B can rotate about the direction of the optical axis (optical axis of light-emitting unit 206) of the scanning light (pulse-like distance measuring light) emitted from light-emitting unit 206. The rotation is performed by a motor, and the rotation angle is precisely measured. The direction of the optical axis of light-emitting unit 206 is determined to coincide with the directions of the optical axes of optical unit 106 and light-receiving unit 207.

[0019] Prism A' and Prism B' are capable of rotation in the same manner as Prism A and Prism B. The rotation is performed by a motor, and the rotation angle is precisely measured.

[0020] Prism A and Prism A' are set to deflect light incident from the same direction by the same deflection angle, and when they rotate, they rotate synchronously in the same way. The same is true for Prism B and Prism B', which are set to deflect light incident from the same direction by the same deflection angle, and when they rotate, they rotate synchronously in the same way.

[0021] A pulse of scanning light is emitted from the light emitting unit 206. The direction of the optical axis of this scanning light is deflected by the prisms B and A, and the scanning light is emitted from the optical unit 106 in a specific direction to the outside.

[0022] This scanning light is reflected from the reflection point of the target, travels a path in the reverse direction of the optical path from optical unit 106 to the reflection point, and enters optical unit 106. This scanning light that has entered optical unit 106 from the outside passes through prisms A' and B', during which the direction of the optical axis is deflected, and enters light receiving unit 207.

[0023] Prisms A and A', and prisms B and B' are set to deflect light incident from the same direction at the same deflection angle, so that the scanning light reflected and returned from the reflection point passes through prisms A' and B', and the direction of its optical axis is returned to the direction of the optical axis of the light receiving unit 207, and the scanning light is incident on the light receiving unit 207.

[0024] Here, when prism A and prism A' are rotated synchronously, the direction of the optical axis in the scanning direction emitted to the outside from optical unit 106 changes. This is the same when prism B and prism B' are rotated synchronously. Also, when prism A and prism A' are rotated synchronously and at the same time prism B and prism B' are rotated synchronously, the direction of the optical axis in the scanning direction emitted to the outside from optical unit 106 changes.

[0025] Therefore, by rotating prisms A and A' and / or prisms B and B' while emitting pulses of scanning light, the direction of the scanning light changes from moment to moment, making it possible to perform laser scanning within the range of deflection by the prisms.

[0026] A variety of laser scanning patterns and scanning ranges can be realized by combining the rotation speed, rotation direction, and rotation range of the prism. Figure 7 shows an example of a scanning pattern.

[0027] For example, if prism A and prism A' are rotated and prism B and prism B' are not rotated, a circular scan pattern is obtained. Next, if prism B and prism B' are rotated slightly and then stopped while prism A and prism A' are rotated and a scan is performed, a scan pattern in which two circles of different diameters are arranged concentrically is obtained. By repeating this process, the scan pattern in Figure 7(A) is obtained. Furthermore, by combining the above operation with the rotation of prism B and prism B', the scan pattern in Figure 7(B) is obtained.

[0028] In addition, if scanning is performed while rotating prisms A and A' in a first direction and rotating prisms B and B' in the opposite direction to the first direction at the same speed, a linear scan is performed. Using this, the scan pattern in Figure 7(C) can be obtained.

[0029] Furthermore, if scanning is performed while rotating prisms A and A' in a first direction and rotating prisms B and B' in the opposite direction to the first direction at different speeds, the scan pattern shown in Figure 7(D) is obtained.

[0030] As shown in FIG. 7, laser scanning using the optical axis deflection unit 208 in FIG. 2 can achieve a variety of scan patterns and scan ranges by combining the rotation of two prism systems.

[0031] Furthermore, by adjusting the rotation speed of the prism and / or the light emission frequency (pulse frequency) of the light emitting unit 206, the scan density (density of scan points) can be adjusted.

[0032] Furthermore, according to this embodiment, scanning is also possible on a surface that does not directly face the laser scanner 100 such that the intervals between scan points are constant in the depth direction. For example, when a laser scanner is installed on a road surface and a laser scan of the road surface is performed from near to far, if scanning is performed with scanning light at a normal equally spaced angle, the intervals between scan points will become wider as the distance increases. In this embodiment, by varying the rotation speed of the prism during scanning, it is possible to make the intervals between scan points in the near and far direction constant or close to constant (equal intervals) in the laser scan of the above road surface.

[0033] The irradiation direction of the scanning light (the direction of emission from the optical unit 106) can be calculated from the rotation angle of each prism. That is, the direction of the scan point as seen by the laser scanner 100 is calculated from the rotation angles of prisms A and A', and the rotation angles of prisms B and B'.

[0034] The prism characteristics are selected so that the deflection angle is 0° when the rotation angles of the pair of prisms A and A' and the pair of prisms B and B' are different by 180°. In this case, the deflection angle is 0°, and the scanning light is emitted from the optical unit 106 in the front direction.

[0035] 2, in order to explain the principle, a structure is shown in which prisms A and A' are prepared separately, and prisms B and B' are prepared separately. Here, it is also possible to use a single prism as both prisms A and A', and to use a single prism as both prisms B and B'. In this case, an optical system that separates and combines the optical axis of light-emitting unit 206 and the optical axis of light-receiving unit 207 is placed in front of light-emitting unit 206 or light-receiving unit 207.

[0036] Returning to Fig. 1, the vertical rotation unit 105 is equipped with a camera 107. The camera 107 is a digital camera that captures RGB images, and captures an area that can be laser scanned by the optical unit 106. By analyzing the image data of the image captured by the camera 107, depth information (distance information) of an object captured in the captured image can be obtained.

[0037] (Laser scanner block diagram) Fig. 3 is a functional block diagram of laser scanner 100. Laser scanner 100 includes camera 107, light emitting unit 206, light receiving unit 207, and optical axis deflection unit 208. Camera 107 is a digital color camera shown in Fig. 1, and captures an image of a laser scan range whose principle is shown in Fig. 2. The relationship between the position and attitude of camera 107 and optical axis deflection unit 208 is known, and the position of a scan point in the shooting range (shooting screen) of camera 107 can be specified. In other words, the corresponding relationship between a specific range (specific part) in an image captured by camera 107 and the laser scan range can be specified.

[0038] The light emitting unit 206 includes a light emitting element (e.g., a laser diode element) and an optical system for emitting scanning light. The scanning light is emitted in pulses from the light emitting unit 206. The light receiving unit 207 includes an optical system and a light receiving element (e.g., a photodiode) for receiving the scanning light reflected and returned by the target object.

[0039] The optical axis deflection unit 208 deflects (changes the direction of) the optical axis of the scanning light using a prism whose principle is shown in Fig. 2. The specific structure of the optical axis deflection unit 208 is described in, for example, Japanese Patent No. 6616077 and Japanese Patent No. 6877946.

[0040] The laser scanner 100 comprises the following functional units: a subject extraction unit 201, a depth information acquisition unit 202, a scan subject designation receiving unit 203, a scan density designation receiving unit 204, a scan condition setting unit 205, a deflection control unit 209, a distance calculation unit 210, an optical axis direction calculation unit 211, a point cloud data generation unit 212, a depth information correction unit 213, and a depth information learning unit 214.

[0041] The laser scanner 100 includes a computer. This computer functions as a control computer that controls the operation of the laser scanner 100. This computer executes the processes shown in Fig. 5 and Fig. 6, which will be described later. In addition, this computer realizes each of the above-mentioned functional units.

[0042] Each of the above-mentioned functional units is realized by executing software for realizing each functional unit by a computer built into the laser scanner 100. One or more of these functional units may be configured with dedicated hardware. Also, one or more of these functional units may be configured with hardware prepared separately from the laser scanner 100 (for example, a separately prepared computer or data processing server).

[0043] The photographed object extraction unit 201 extracts various objects captured in the image captured by the camera 107. For example, individual objects such as a road surface, a wall, a roof, a handrail, a pipe, a pillar, an overhead line, a road surface, and a vehicle are identified and extracted. This process is performed using image processing software. The extracted object images are classified into types such as a road surface, a wall, a roof, a handrail, a pipe, a pillar, an overhead line, a road surface, and a vehicle. FIG. 4 is an image diagram showing an example of a captured image.

[0044] The depth information acquisition unit 202 acquires depth information (distance information) of an object captured in an image (RGB image) captured by the camera 107. Here, the depth information is information on the distance from the optical origin (projection center) of the camera 107 to the object.

[0045] The image captured by the camera 107 is a monocular image. The depth information acquisition unit 202 predicts distance information of objects captured in the captured image, which is a monocular image, using a neural network (depth estimation AI model) that utilizes deep learning, and creates a depth map of the captured image. In this depth map, distance information (distance from the optical origin of the camera 107) of the objects captured in the captured image is expressed by shading and color tone. Various depth estimation AI models have been developed and are available on the Internet.

[0046] By combining information relating to each extracted target object extracted by extraction unit 201 for capturing images with the above-mentioned depth information (distance information), it is possible to obtain depth information (distance information) for each target object.

[0047] The scan target designation receiving unit 203 receives a target designated from among the images captured by the camera 107. The designation is performed using the operation panel of the laser scanner 100. Specifically, the designation is performed as follows. First, the image captured by the camera 107 is displayed on the touch panel display of the operation panel. In the displayed image, the user touches a portion that the user wishes to designate as the target of the laser scan. This designates the target of the laser scan, and the information about the designation is acquired by the scan target designation receiving unit 203.

[0048] When a scan target is selected, a scan pattern suitable for the scan target is selected. A desired scan pattern can also be selected from a plurality of scan patterns.

[0049] The scan density designation receiving unit 204 receives a designation of the scan density (density of scan points) for a designated scan target. The scan density is designated using an operation panel (not shown) provided on the laser scanner 100. The scan density is the number of scan points (reflection points) per unit area of ​​the scan target surface.

[0050] The scan condition setting unit 205 sets the scan conditions for the laser scan performed by the laser scanner 100. The scan conditions set the light emission conditions of the light emitting unit for performing a scan at a specified density for a range covering a specified target, and / or the conditions for controlling the rotation of the prism in the optical axis deflection unit 208.

[0051] For example, suppose that the utility pole 151 in Fig. 4 is selected as the target of laser scanning. In this case, a vertically long rectangular scanning range is set according to the shape of the utility pole 151, and scanning conditions are set to realize laser scanning at a specified scanning density in the scanning range. At this time, the scanning density is set using the depth information of the utility pole 151 estimated by the depth estimation AI model based on the captured image.

[0052] For example, the distance to a scan point is L, the angle of the adjacent scan light is θ, and Lθ is the approximate distance between the scan points. If the scan points are not directly facing each other, the distance will be longer than Lθ depending on the angle, so in that case, the direction of the scan surface must be taken into consideration. Here, L is estimated in the depth estimation AI model, and the interval Lθ between the scan points can be set by adjusting θ. θ is adjusted by setting the emission frequency of the scan light and the rotation speed of the prism. The scan density is set by setting this Lθ.

[0053] The deflection control unit 209 generates a control signal for controlling a motor that rotates the prism in the optical axis deflection unit 208. The deflection control unit 209 controls the laser scan. The deflection control unit 209 functions as a scan control unit.

[0054] Based on the principle of optical distance measurement, the distance calculation unit 210 calculates the distance from the optical origin to the scan point (the reflection point of the scanning light) of the laser scanner 100. The optical origin is, for example, the position of the light emitting point of the light emitting element of the light emitting unit 206.

[0055] The optical axis direction calculation unit 211 calculates the direction of the optical axis of each scanning light (each of one scanning light (one pulse)) from information on the rotation angle of the prism in the optical axis deflection unit 208. This process obtains information on the direction of each scanning point as seen by the laser scanner 100. The point cloud data generation unit 212 generates point cloud data, which is data summarizing data on the distance and direction of each scanning point from the optical origin of the laser scanner 100. The point cloud data also includes the intensity of reflected light from each scanning point.

[0056] The depth information correction unit 213 corrects the depth information (distance information) of the image acquired by the depth information acquisition unit 203 based on the point cloud data generated by the point cloud data generation unit. The accuracy of the depth information of the image acquired by the depth information acquisition unit 203 has a large error compared to stereo photography measurement and optical distance measurement, and is easily affected by how the target image is captured (color, effect of shadow, brightness, resolution, relationship with the background image, etc.).

[0057] Therefore, the following process is carried out to reduce the influence of the above-mentioned errors: First, assume that multiple objects are extracted from a captured image, and depth information (distance information) is obtained for each of them.

[0058] Here, a laser scan is performed on the first extraction target with a narrower range, and depth information (distance information) based on the laser scan is obtained. Next, the depth information of the extraction target obtained from the captured image (first depth information) is compared with the depth information obtained from the laser scan (second depth information).

[0059] Here, the first depth information is information estimated from a captured image using a depth estimation AI model, and has a larger error than the second depth information, i.e., the second depth information is more accurate than the first depth information.

[0060] Here, the difference or ratio between the first depth information and the second depth information is obtained as error correction information, and this error correction information is used to correct the depth information obtained by the depth estimation AI model for extraction targets other than the first extraction target (e.g., the second extraction target, the third extraction target, ...).

[0061] For example, in the case where the second depth information / first depth information=k for the first extraction target, k is acquired as an error coefficient, and the depth information (first depth information) obtained from the image of the second extraction target is multiplied by k. This corrects the error in the depth information obtained from the image related to the second extraction target. The same process is performed for the third extraction target and subsequent extraction targets.

[0062] Also, for the second extraction target, "k=second depth information / first depth information" may be obtained, and the k obtained there may be used to correct the depth information obtained from the image of the third extraction target. In other words, assuming n to be a natural number not including 0, for the nth extraction target, "kn=second depth information / first depth information" may be obtained, and the kn obtained there may be used to correct the depth information obtained from the image of the (n+1)th extraction target.

[0063] The following aspects are also possible as the process performed in the depth information correction unit 213. First, the error coefficient k for the first extraction target is obtained by the above method. Next, an extraction target having an appearance similar to that of the first extraction target is selected as the second extraction target. For example, if the first extraction target is a pipe, an object having a similar appearance of the pipe or an appearance close to that is selected as the next extraction target. Then, depth information is obtained from the image of the next selected extraction target, and this depth information is corrected using the error coefficient k obtained for the first extraction target. Specifically, the depth information obtained from the image of the next selected extraction target is multiplied by k.

[0064] This method is based on the finding that objects with similar appearances may be prone to similar errors in depth estimation (depth information estimation) by a depth estimation AI model.

[0065] The following aspect is also possible as the process performed in the depth information correction unit 213. First, the error coefficient k for the first extraction target is acquired by the above method. Next, the extraction target for which the first depth information closest to the first depth information for the first extraction target (depth information estimated by the depth estimation AI model based on the image) is obtained is selected as the next extraction target. Then, depth information is obtained from the image of the next selected extraction target, and this depth information is corrected using the error coefficient k obtained for the first extraction target. Specifically, the depth information obtained from the image of the next selected extraction target is multiplied by k.

[0066] This method is based on the finding that if the depth estimates made by a depth estimation AI model are close, the error is likely to be close as well.

[0067] The depth information learning unit 214 uses the depth information (distance information) obtained by laser scanning as teacher data to re-learn the depth estimation AI model used in the depth information acquisition unit 202. This can improve the estimation accuracy of the depth estimation AI model.

[0068] (An example of a processing procedure) An example of the operation of the laser scanner 100 will be described below. Fig. 5 is a flow chart showing an example of the processing procedure. A program for executing the processing of Fig. 5 is stored in a storage unit of a computer included in the laser scanner 100 or in an appropriate storage medium, and is read and executed by the computer. It is also possible to store the program in a server and download it from there for use.

[0069] First, the camera 107 is used to capture an image of an area including a laser scan target, and image data is obtained (step S101). Next, based on the image data obtained in step S101, individual targets are extracted from the captured image (step S102). This process is performed by the target extraction unit 201.

[0070] Next, depth information of each of the objects extracted in step S102 is acquired (step S103). This process is performed by the depth information acquisition unit 202. Next, designation of an object to be laser scanned is accepted from among the multiple shooting objects extracted in step S102 (step S104). This designation is performed by the user. The number of objects designated may be one or more.

[0071] An example of step S104 will be described below. For example, suppose that utility pole 151 is extracted from the captured image in Fig. 4, and this utility pole 151 is selected as the scanning target. In this case, the screen in Fig. 4 is displayed on the display of the operation unit of laser scanner 100 or the display of the terminal. The user selects the part of utility pole 151 from the displayed image. This selection information is received by scanning target designation receiving unit 203 in Fig. 3.

[0072] Next, a scan target is selected (step S105). For example, if multiple targets are specified in step S103, one of them is selected as the scan target (step S105). The selection can be automatic or user-selected.

[0073] Next, a designation of the scan density (density of scan points) for the scan target selected in step S105 is accepted (step S106). For example, 1 point / cm 2 The scan density can be specified by the user.

[0074] Next, the conditions for the laser scan for the range narrowed down to the scan target selected in step S105 and the scan conditions for obtaining the scan density specified in step S106 are set (step S107). This process is performed by the scan condition setting unit 205 in Fig. 3. A detailed example of the process performed in step S107 will be described below.

[0075] For example, suppose that utility pole 151 is selected as the scanning target in the display image of Fig. 4. In this case, conditions are set for scanning limited to utility pole 151. These scanning conditions are conditions that determine the range of the scan. Specifically, conditions are set that determine the rotation range of the prisms (prism A, prism A', prism B, and prism B' in Fig. 2) for scanning limited to utility pole 151.

[0076] The scan density is set by the following procedure. First, in step S103, depth information (distance information) of the utility pole 151 is obtained. Therefore, the scan density at utility pole 151 is uniquely determined by the emission frequency of the scan light and the rotation speed of the prism in deflection control unit 209. Therefore, when the scan density at utility pole 151 is specified, the emission frequency of the scan light and / or the rotation speed of the prism in deflection control unit 209 is adjusted to obtain the specified scan density.

[0077] After the laser scan conditions have been set, a laser scan is performed on the designated target (site) to obtain point cloud data of the target (step S108).

[0078] Next, based on the point cloud data obtained in step S108, the depth information (distance information) obtained in step S103 is corrected (step S109).

[0079] For example, suppose that the utility pole 151 in Fig. 4 is selected as the scanning target. In this case, the depth information (distance information) obtained in step S103 for the utility pole 151 is compared with the depth information (distance information) obtained as a result of step S108 for the utility pole 151 to determine the difference, and a correction parameter for correcting the difference is obtained. Using this correction parameter, the depth information of the next scanning target to be selected (depth information estimated by a depth estimation AI model based on a captured image) is corrected.

[0080] Next, it is determined whether or not there is an unselected object at this point (step S109). If there is an unselected object, the unselected object is selected, and the processing from step S105 onwards is executed again.

[0081] (Advantages) FIG. 3 shows laser scanner 100, which is a device for performing laser scanning and includes camera 107 as an imaging means, optical axis deflection unit 208 as a means for performing laser scanning, depth information acquisition unit 202 as a means for obtaining depth information of an object shown in an image based on the image captured by camera 107, and a scan condition setting unit as a means for setting conditions for the laser scan of the object based on the depth information.

[0082] According to this configuration, by setting the conditions for laser scanning using distance information of the object to be photographed obtained from the photographed image, it is possible to set appropriate scanning conditions according to the object to be scanned.

[0083] (Relearning process) By repeating the process in Figure 5, data on the target image and its distance information obtained by laser scanning are accumulated. By performing machine learning using the distance information obtained by this laser scanning as training data, the estimation accuracy of the depth estimation AI model used can be improved.

[0084] 2. Second embodiment An example of the process is shown in Fig. 6. A program for executing the process in Fig. 6 is stored in a storage unit or an appropriate storage medium of a computer included in the laser scanner 100, and is executed by the computer.

[0085] First, an image of an area including a measurement target is captured (step S201). Next, a laser scan is performed on the captured area in step S201 (step S202). This laser scan may be a coarse (sparse) laser scan, and is set to, for example, 1 / 100 or less of the scan density in the laser scan (laser scan in step S108) for each object extracted from the captured image. This is because the purpose of the laser scan in step S202 is to obtain point cloud data for calibrating the depth information estimated by the depth estimation AI model, and it is not necessary to obtain depth information (distance information) at a density as high as that of point cloud data for creating a 3D model.

[0086] Next, an object to be photographed is extracted from the photographed image obtained in step S201 (step S203), and depth information is acquired based on the photographed image of the extracted object (step S204).

[0087] Next, the depth information obtained in step S204 is corrected using the laser scan data obtained in step S202 (step S205).

[0088] A detailed example of this process will be described below. The depth information obtained in step S204 is information estimated by the depth estimation AI model based on the captured image and contains errors. On the other hand, the laser scan data obtained in step S202 has a sparse scan density, but the distance measurement information of each scan point has higher accuracy than the depth information (distance information) estimated by the depth estimation AI model. In addition, the captured image obtained in step S201 and the laser scan data obtained in step S202 can be superimposed, and the correspondence between the captured image and the scan points can be identified.

[0089] Therefore, in relation to the depth information obtained in step S204, the depth information estimated by the depth estimation AI model at the corresponding laser scan point is compared with the depth information of the laser scan point to evaluate the error. Although the density of the laser scan points is sparse, the error of the depth information estimated by the depth estimation AI model is evaluated at the intervals of the sparse laser scan points, and calibration is performed.

[0090] For example, let's look at a certain scan point Pi. In this case, the depth information (distance information) obtained by the laser scan at the scan point Pi is compared with the depth information estimated by the depth estimation AI model at that position. Then, based on the difference, the depth information estimated by the depth estimation AI model at the point Pi is corrected. This process is performed for each scan point.

[0091] The depth information estimated by the depth estimation AI model between two scan points is corrected by interpolation. For example, the correction value at scan point Pi is Δf1, and the correction value at scan point Pi+1 is Δf2. In this case, it is assumed that the correction value changes linearly between scan point Pi and scan point Pi+1, and the depth information estimated by the depth estimation AI model between scan point Pi and scan point Pi+1 is corrected.

[0092] After step S205, the process proceeds to step S104 and subsequent steps described with reference to Fig. 5. In this process, the process of step S109 in Fig. 5 is omitted (although it may be executed, of course).

[0093] According to this embodiment, the depth information obtained from the image is calibrated using the result of the coarse laser scan in step S202. This makes it possible to improve the accuracy of the depth information obtained from the image and set appropriate scanning conditions.

[0094] In this embodiment, the laser scan in step S202 may be a coarse scan density laser scan, and the time required for scanning is short. Therefore, it is possible to suppress the decrease in accuracy of depth information (distance information) obtained from the image while suppressing the increase in work time.

[0095] 3. Other embodiments The means for performing laser scanning of a specific range is not limited to the principle shown in Fig. 2. For example, a laser scanner that performs scanning electronically, or a laser scanner that performs scanning light by rotating or reciprocating a mirror can also be used. [Industrial Applicability]

[0096] The present invention can be used in technologies relating to laser scanning. [Explanation of symbols]

[0097] 100...laser scanner, 101...tripod, 102...base portion, 103...horizontal rotation portion, 104...support portion, 105...vertical rotation portion, 106...optical portion, 107...camera, 151...electric pole.

Claims

1. A laser scanning device, comprising: A photographing means; A means for performing laser scanning; a means for obtaining depth information of an object shown in an image based on the image captured by the imaging means; means for setting conditions for the laser scan of the object based on the depth information; A laser scanning device comprising:

2. The objects include a first object and a second object, 2. The laser scanning device of claim 1, further comprising means for modifying the depth information obtained from the image of the second object based on the results of a laser scan of the first object.

3. The laser scanning device of claim 2 , wherein the second object is similar in appearance to the first object.

4. The laser scanning device according to claim 2 , wherein the second object is an object having depth information closest to the first object.

5. A laser scan is performed on the imaging range of the imaging means, the laser scanning is performed at a scan density less than a scan density of the laser scanning of the object; 2. The laser scanning device according to claim 1, wherein the depth information is corrected based on the result of the laser scanning of the imaging range of the imaging means.

6. 1. A method for performing laser scanning, comprising: Obtaining depth information of an object shown in a photographed image based on the photographed image; A laser scanning method for setting conditions for laser scanning the object based on the depth information.

7. A laser scanning program for performing a laser scan, To your computer acquiring depth information of an object shown in a photographed image based on the photographed image; A laser scanning program that sets conditions for laser scanning the object based on the depth information.

Images