Information processing device, control device, information processing method, and program

Abstract

To achieve more suitable calibration for targeting flying pests and other targets with high-power lasers.SOLUTION: An information processing device comprises: an image acquisition unit that acquires a plurality of images obtained by capturing images of irradiation points where a laser is irradiated multiple times from a laser irradiation device on a specified structure while changing a scanning angle of a laser scanning device which is a component of the laser irradiation device; an error calculation unit that calculates a distance as an error between coordinates of the irradiation points captured in the multiple images and a laser straight line defined by a parameter group that determine a positional relation between the laser scanning device and the camera; and a parameter group optimization unit that optimizes the parameter group so as to minimize the error.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a control device, an information processing method, and a program. [Background technology]

[0002] A technology for targeting flying pests with a high-power laser is known. To target flying pests with a high-power laser, it is necessary to specify (calibrate) in advance the positional relationship between the coordinate system of the stereo camera that measures the three-dimensional position of the flying pest and the coordinate system of the galvanometer mirror that controls the angle of the laser beam.

[0003] In this regard, Patent Document 1 describes a technology for calibrating an analog signal with an amplitude proportional to the rotational speed of a conveyor to determine the relative position of a workpiece to be irradiated with a laser. Patent Document 2 describes a technology for creating a calibration table for a camera that captures an image of a target surface and a galvanometer scanner that scans the target surface with a laser beam. However, both of these conventional technologies irradiate a laser onto a flat surface and do not specify the positional relationship between the camera and the galvanometer mirror. In other words, these conventional technologies do not achieve calibration suitable for targeting flying pests that move in three dimensions with a high-power laser. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 8-323486 [Patent Document 2] Japanese Patent Application Laid-Open No. 2008-264789 Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention has been made in consideration of these circumstances, and one of its objects is to provide an information processing device, a control device, an information processing method, and a program that can achieve more suitable calibration for targeting flying pests and other targets with a high-power laser. [Means for solving the problem]

[0006] An information processing device according to one aspect of the present invention includes an image acquisition unit that acquires multiple images obtained by capturing images of irradiation points where a laser is irradiated multiple times from a laser irradiation device onto a predetermined structure while changing the scanning angle of a laser scanning device, which is a component of a laser irradiation device; an error calculation unit that calculates an error as the distance between the coordinates of the irradiation points captured in the multiple images and a laser straight line defined by a parameter group that determines the positional relationship between the laser scanning device and the camera; and a parameter group optimization unit that optimizes the parameter group so as to minimize the error.

[0007] The parameter group optimization unit may optimize the parameter group by changing a parameter included in the parameter group to change the laser straight line and bring it closer to the coordinates of the irradiation point.

[0008] The camera is a 3D camera, and when a stereo camera, which is one type of 3D camera, is used, the coordinates of the illuminated point may be three-dimensional coordinates calculated from a visible image and a parallax image included in the multiple images.

[0009] The group of parameters may include at least one of the amount of parallel movement from a camera coordinate system based on the camera to a laser scanning device coordinate system based on the laser scanning device, the amount of rotational movement from the camera coordinate system to the laser scanning device coordinate system, an arbitrary point on a laser line irradiated by the laser irradiation device, the direction of the laser line before being reflected by the laser scanning device, and the deviation between the command value of the scanning angle of the laser scanning device and the actual angle.

[0010] Another aspect of the present invention is a control device that controls the laser irradiation device based on a group of parameters optimized by the information processing device, and includes an image acquisition unit that acquires an image obtained by capturing an image of a target with a camera, a scan angle determination unit that determines the scan angle that minimizes the distance between the coordinates of the target captured in the image and a laser line defined by the group of parameters and the scan angle of the laser scanning device, and a laser irradiation control unit that provides the determined scan angle as a command value to the laser irradiation device to cause it to irradiate a laser.

[0011] Another aspect of the present invention is an information processing method in which a computer acquires multiple images obtained by using a camera to capture irradiation points where a laser is irradiated multiple times from a laser from a laser irradiation device on a specified structure while changing the scanning angle of the laser scanning device, which is a component of the laser irradiation device; calculates an error as the distance between the coordinates of the irradiation points captured in the multiple images and a laser line defined by a group of parameters that determine the positional relationship between the laser scanning device and the camera; and optimizes the group of parameters so as to minimize the error.

[0012] Another aspect of the present invention is a program that causes a computer to acquire multiple images obtained by using a camera to capture irradiation points where a laser is irradiated multiple times from a laser from a laser irradiation device on a specified structure while changing the scanning angle of the laser scanning device, which is a component of the laser irradiation device; calculate the distance between the coordinates of the irradiation points captured in the multiple images and a laser line defined by a group of parameters that determine the positional relationship between the laser scanning device and the camera as an error; and optimize the group of parameters so as to minimize the error. [Effects of the Invention]

[0013] According to the present invention, it is possible to achieve more suitable calibration for targeting flying pests and other targets with a high-power laser. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a diagram showing an example of a usage environment and configuration of an information processing device 100 according to a first embodiment. [Figure 2] FIG. 2 is a diagram for explaining an outline of the optimization process executed by the parameter set optimization unit 130. [Figure 3] FIG. 10 is a diagram showing details of a parameter group 144 to be optimized. [Figure 4] 10 is a diagram for explaining a method for calculating a laser straight line by the error calculation unit 120. FIG. [Figure 5] FIG. 10 is a diagram for explaining a parameter optimization method performed by the parameter set optimization unit 130. [Figure 6] 1 is a diagram showing an outline of the results of performing calibration according to the present invention on a non-planar structure S. FIG. [Figure 7] 4 is a flowchart showing an example of the flow of processing executed by the information processing device 100 according to the first embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a usage environment and configuration of a control device 200 according to a second embodiment. [Figure 9] 10 is a diagram for explaining a method for determining a mirror angle by a mirror angle determination unit 220. FIG. [Figure 10] 10 is a flowchart showing an example of the flow of processing executed by a control device 200 according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of an information processing device, a control device, an information processing method, and a program according to the present invention will be described with reference to the drawings. [First embodiment] [overview] 1 is a diagram showing an example of a usage environment and configuration of an information processing device 100 according to the first embodiment. The information processing device 100 operates in cooperation with, for example, a camera 10 and a laser irradiation device 50 including a galvanometer mirror as a component.

[0016] The camera 10 is, for example, a stereo camera that can acquire depth information of an object in addition to two-dimensional information in length and width by capturing images synchronously using multiple cameras, or a 3D camera such as a ToF (Time of Flight) camera that measures distance from the time it takes for irradiated light to reflect and return. The camera 10 may also be a 3D laser scanner such as a LIDAR (light detection and ranging) scanner. The camera 10 is installed facing in a direction that allows it to capture an image of the surface of a predetermined structure S, such as a wall. The camera 10 is communicably connected to the information processing device 100 via a wired cable or a wireless network.

[0017] The laser irradiation device 50 is, for example, a high-power laser device that outputs a laser capable of killing flying pests, and is communicatively connected to the information processing device 100 via a wired cable or a wireless network. The laser irradiation device 50 irradiates the surface of a predetermined structure S with lasers L1, L2, ... multiple times while changing the mirror angle of the galvanometer mirror (i.e., the laser irradiation angle), and the camera 10 captures images of the structure S showing irradiation points P1, P2, ... on the structure S irradiated with these lasers. In this case, the laser irradiation may be at regular intervals (in a grid pattern) or not. The image acquired by the camera 10 is a set of a visible image and a parallax image, and the camera 10 transmits the set of the visible image and the parallax image acquired for each irradiation point P to the information processing device 100. The direction of the laser emitted from the laser irradiation device 50 can be changed by a galvanometer mirror, but other laser scanning devices such as a MEMS (Micro-Electro-Mechanical Systems) mirror or a deformable mirror may also be used instead of a galvanometer mirror.

[0018] The information processing device 100 is a computer device such as a personal computer, a tablet terminal, or a server. The information processing device 100 includes, for example, an image acquisition unit 110, an error calculation unit 120, a parameter set optimization unit 130, and a storage unit 140. Each of the image acquisition unit 110, the error calculation unit 120, and the parameter set optimization unit 130 is realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or an SOC (System On Chip), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transitory storage medium) such as a hard disk drive (HDD) or flash memory, or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM, and installed by inserting the storage medium into a drive device. The storage unit 140 is realized by a storage device such as a HDD, flash memory, or random access memory (RAM). The storage unit 140 stores, for example, irradiation point image data 142 and a parameter set 144.

[0019] The image acquisition unit 110 acquires a set of visible images and parallax images of the illuminated point P captured by the camera 10 (hereinafter, it is assumed that N sets have been acquired, where N is a positive integer), and stores them as illuminated point image data 142. The parallax image includes depth information of the illuminated point P as pixel values, along with the two-dimensional position of the illuminated point P based on the camera coordinate system. Therefore, the image acquisition unit 110 calculates the three-dimensional coordinates X i(i=1, 2, . . . N) are calculated and stored in the storage unit 140 in association with the irradiation point image data 142.

[0020] The error calculation unit 120 calculates the error between the three-dimensional coordinates of the irradiation point P in the camera coordinate system and the laser straight line of the laser irradiation device 50 defined by the parameter group 144. Here, the parameter group 144 is a plurality of parameters that determine the positional relationship between the camera coordinate system and the galvanometer mirror coordinate system. In other words, the error calculation unit 120 converts the laser straight line of the laser irradiation device 50 in the galvanometer mirror coordinate system in accordance with the parameter group 144, thereby deriving the laser straight line of the laser irradiation device 50 in the camera coordinate system and being able to calculate the error between the three-dimensional coordinates of the irradiation point P. Details of the parameter group 144 and the processing executed by the error calculation unit 120 will be described later.

[0021] The parameter set optimization unit 130 optimizes the parameter set 144 so as to minimize the error calculated by the error calculation unit 120. FIG. 2 is a diagram for explaining an overview of the optimization process executed by the parameter set optimization unit 130. The upper part of FIG. 2 shows the state before the optimization process is executed by the parameter set optimization unit 130, and the lower part of FIG. 2 shows the state after the optimization process is executed by the parameter set optimization unit 130. Before the optimization process is executed, the parameter set 144 is given an arbitrary initial value, so the distance (error) between the irradiation point P and the laser straight line L is large. Therefore, the parameter set optimization unit 130 reduces the error between the irradiation point P and the laser straight line L by repeatedly adjusting the values ​​of the parameter set 144 and recalculating the laser straight line L. The value of the parameter set 144 at the point when the error is minimized becomes the optimized (calibrated) parameter set 144. Details of the optimization process will be described below.

[0022] [Optimization process details] 3 is a diagram showing details of the parameter group 144 to be optimized. In FIG. 3, the coordinate system defined by a combination of the X-axis, Y-axis, and Z-axis represents the camera coordinate system based on the camera 10. G Axis, Y G axis, Z G The coordinate system defined by the combination of axes represents a galvanometer mirror coordinate system with the galvanometer mirror as the reference. In FIG. 3, as an example, the camera coordinate system is defined as an orthogonal right-handed system with the right direction of the camera 10 body as the X axis, the downward direction as the Y axis, and the origin as the focal position of the left camera. Furthermore, the galvanometer mirror coordinate system is defined as follows: G axis, downward Y G The axis and origin are defined as an orthogonal right-handed system with the center position on the rotation axis of the mirror. The symbol T represents the amount of parallel translation from the camera coordinate system to the galvanometer mirror coordinate system (in other words, the position of the galvanometer mirror coordinate system as seen from the camera coordinate system), and T = [T x ,T y ,T z ] T The symbol R(φ, θ, Ψ) represents the amount of rotational movement from the camera coordinate system to the galvanometer mirror coordinate system, with rotation around the Z axis (the rotation direction from the X axis to the Y axis is positive) represented as Ψ, rotation around the Y axis (the rotation direction from the Z axis to the X axis is positive) represented as θ, and rotation around the X axis (the rotation direction from the Y axis to the Z axis is positive) represented as φ. The symbol q0 represents an arbitrary point on the laser line irradiated by the laser irradiation device 50, and the symbol X G Set the axis components to appropriate values, e.g., q0=[-0.01,γ Y ,γ Z ] T The symbol v0 represents the direction of the laser beam before it is reflected by the galvanometer mirror, and X G Set the axis components to appropriate values, e.g., v0=[1.0,μ Y ,μ Z ] T The ten parameters u = [φ, θ, Ψ, T x ,T y ,T z ,γ Y ,γ Z ,μ Y ,μ Z ]T are the parameter group 144 to be optimized. In addition, the deviation of the initial angle of the galvanometer mirror (Δα0, Δα1) may be included in the parameter group 144. This means the deviation between the command value of the mirror angle and the actual angle.

[0023] FIG. 4 is a diagram for explaining a method for calculating the laser line by the error calculation unit 120. The coordinates of each point shown in FIG. 4 are hereinafter given on the galvanometer mirror coordinate system. In FIG. 4, symbol q1 represents the coordinate of the reflection point of the first mirror M1 of the galvanometer mirror that first reflects the laser irradiated by the laser irradiation device 50, symbol q2 represents the coordinate of the reflection point of the second mirror M2 of the galvanometer mirror that next reflects the laser irradiated by the laser irradiation device 50 and reflected by the first mirror M1, and symbol q3 represents the coordinate of the irradiation point P on the structure S irradiated with the laser. These coordinates are referred to as q k (Hereafter, k=0,1,2,3) q k is expressed by the following equation (1).

[0024]

number

[0025] In formula (1), a k (k=0, 1, 2) represent the coordinates of the first mirror M1, the second mirror M2, and the reference point on the structure S, respectively, and n k (k=0,1,2) is a k represents the normal vector of k (k=0,1,2) represents the vector that indicates the direction of the laser line. k , n k , v k are expressed by the following equations (2) to (4), respectively.

[0026]

number

[0027]

number

[0028]

number

[0029] In formulas (2) to (4), b k , d k (k=0, 1) represent the position vector of the reference point on the rotation axis of the first mirror M1 and the second mirror M2 and the direction vector of the rotation axis, respectively. By applying the mirror angles α0 and α1 of the galvanometer mirror to the above equations (1) to (4), the coordinate of the reflection point q2 of the second mirror M2 and the vector v2 are obtained. Here, the mirror angle α0 represents the mirror angle of the first mirror M1, and the mirror angle α1 represents the mirror angle of the second mirror M2. Furthermore, by applying the information a2 and n2 on the surface of the structure S to be irradiated with the laser, the coordinate q3 of the laser irradiation point P is obtained. In other words, this allows the error calculation unit 120 to calculate the laser straight line in the galvanometer mirror coordinate system. Note that when the deviation (Δα0, Δα1) of the initial angle of the galvanometer mirror is taken into consideration, α in the above equations (1) to (4) is calculated. k α k +Δα k Replace with and calculate.

[0030] After calculating the laser straight line in the galvanometer mirror coordinate system, the error calculation unit 120 calculates the error between the laser straight line and the irradiation point P in the camera coordinate system stored in advance as follows: More specifically, first, the two mirror angles are set as α i =(α 0(i) ,α 1(i) ), where parameter u is the explanatory variable and r i The function Fα with (i) is defined by the following equation (5).

[0031]

number

[0032] r i is the three-dimensional coordinate X of the illuminated point P in the camera coordinate system. i represents the foot of the perpendicular line dropped from the center of the laser beam to the straight line of the laser beam, and is also a coordinate defined in the camera coordinate system. C (u) represents the coordinates of the irradiation point P in the galvanometer mirror coordinate system converted to the camera coordinate system. Similarly, v2 C (u) is also the vector v2 in the galvanometer mirror coordinate system converted to the camera coordinate system, and q2 C and v2 C are all functions of u.

[0033] FIG. 5 is a diagram for explaining a method for calculating an error by the error calculation unit 120. In FIG. 5, X i represents the three-dimensional coordinates of the illuminated point P calculated from the parallax image captured by the camera 10, and r i is the three-dimensional coordinate X i The laser straight line L when the image corresponding to i These perpendicular feet have the same parameter u, but have different coordinates because the mirror angles α0 and α1 of the galvanometer mirror when the laser is irradiated are different. The error calculation unit 120 calculates the N measured three-dimensional coordinates X=[X1 T ,X2 T ,···,X N T ] and the laser line L i The foot of the perpendicular line r=[r1 T ,r2 T ,···,r N T The sum of squares S of the distances to [ ] is defined as the error according to the following equation (6).

[0034]

number

[0035] The parameter group optimization unit 130 finds a parameter u that minimizes the sum of squares S. First, the foot of the perpendicular line r=[r1 T ,r2 T ,···,r N T ] is expressed according to the following equation (7).

[0036]

number

[0037] Here, we consider making r approach X by slightly changing u by δu, and find δu that minimizes the following equation (8).

[0038]

number

[0039] In equation (8), J B represents the Jacobian matrix of F(u). δu, which minimizes equation (8), can be transformed to δS / δu=0, to obtain the following equation (9).

[0040]

number

[0041] In equation (9), J + B is J B represents the pseudo-inverse matrix of the above. The parameter set optimization unit 130 repeats updating according to the update equation u=u+δu until the sum of squares S becomes equal to or less than a predetermined value, or until it sufficiently converges to a minimum value and δu becomes equal to or less than a predetermined value. As another method for determining δu, the Levenberg-Marquard method may be used, and calculation is performed according to the following equation (10) by adding β, which changes the degree of steepest descent method.

[0042]

number

[0043] As explained above, the parameter u is updated by adding δu. Regarding the initial value of the parameter u, for example, the translation amount T=[T x ,T y ,T z ] may be set to a value manually measured by the person performing the calibration using a ruler or the like. The rotational movement amount R(φ,θ,Ψ) from the camera coordinate system to the galvanometer mirror coordinate system may be set to zero, since the camera coordinate system and the galvanometer mirror coordinate system are usually set to be parallel. The position of the laser passing point q0=[-0.01,γ Y ,γ Z ] T and the direction of the laser line before being reflected by the galvanometer mirror is v0=[1.0,μ Y ,μ Z ] T As for the realistic values, q0=[-0.01,0,0] T and v0=[1.0,0,0] T You can also set it as follows.

[0044] When the parameter u is optimized by the parameter group optimization unit 130, the optimized parameter u is used to map the point p in the galvanometer mirror coordinate system to the point p in the camera coordinate system according to the following equations (11) to (13): C This allows for more suitable calibration for targeting flying pests and other targets with high-power lasers.

[0045]

number

[0046]

number

[0047]

number

[0048] In the above description, the structure S onto which the laser is irradiated during calibration is described as being flat. However, the present invention is not limited to such a configuration, and the principles of the present invention can be applied as is even if the structure S has any shape. FIG. 6 is a diagram showing an overview of the results of performing calibration according to the present invention on a non-flat structure S. Specifically, when the structure S is a corner of a wall, the inventor of the present invention caused the laser irradiation device 50 to irradiate the corner of the wall with a laser multiple times, obtained the coordinates of the irradiation point P, and performed calibration with the laser line defined by the parameter u. As a result, it was confirmed that calibration between the galvanometer mirror coordinate system and the camera coordinate system can be performed even when the present invention is applied to a non-flat structure S.

[0049] [Processing flow] Next, a flow of processing executed by the information processing device 100 will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the flow of processing executed by the information processing device 100 according to the first embodiment. The processing of the flowchart shown in Fig. 7 is executed, for example, when the camera 10 and the laser irradiation device 50 are installed for the first time or when the positional relationship between the camera 10 and the laser irradiation device 50 is changed.

[0050] First, the image acquisition unit 110 acquires multiple images obtained by photographing irradiation points where a laser is irradiated multiple times on a predetermined structure (step S100). Next, the error calculation unit 120 calculates the sum of squares S of the distances between the coordinates of the multiple irradiation points and multiple laser lines based on the initial value of the parameter u as an error (step S102). Next, the parameter set optimization unit 130 executes an optimization process for the parameter u until the sum of squares S becomes equal to or less than a predetermined value (step S104). The parameter set optimization unit 130 acquires the parameter u for which the sum of squares S becomes equal to or less than the predetermined value as a calibrated parameter (step S106). This completes the processing of this flowchart.

[0051] In the first embodiment, since the target is a flying pest moving in a three-dimensional position, the camera 10 is a 3D camera such as a stereo camera, and is calibrated to minimize the distance between the three-dimensional coordinates of the irradiation point P and the laser line. However, the present invention is not limited to such a configuration. For example, if the target pest moves on a two-dimensional plane such as a wall, the camera 10 does not need to be a 3D camera, but may be a digital camera using a solid-state image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). In this case, the optimization process minimizes the distance between the two-dimensional coordinates of the irradiation point P and the laser line of the laser irradiation device 50 defined by the parameter set 144, further simplifying the calculation.

[0052] According to the first embodiment described above, unlike conventional techniques in which the structure onto which the laser is irradiated for calibration is limited to a flat surface, the positional relationship between the camera and the galvanometer mirror can be determined by irradiating a three-dimensional structure of any shape with a laser and minimizing the distance between the laser irradiation point and the laser line. This allows for more suitable calibration for targeting flying pests that move in three dimensions with a high-power laser.

[0053] [Second embodiment] In the second embodiment, based on the parameter u optimized by the calibration according to the first embodiment, an appropriate mirror angle of the galvanometer mirror for irradiating a laser onto a target T such as a flying pest insect is determined, and the laser irradiation device 50 is controlled. The control device according to the second embodiment will be described below.

[0054] 8 is a diagram showing an example of the usage environment and configuration of a control device 200 according to the second embodiment. The control device 200 operates in cooperation with, for example, a camera 10 and a laser irradiation device 50 including a galvanometer mirror as a component. The configurations of the camera 10 and the laser irradiation device 50 are the same as those in the first embodiment, and therefore a description thereof will be omitted.

[0055] The control device 200 is a computer device such as a personal computer, a tablet terminal, or a server. The control device 200 includes, for example, an image acquisition unit 210, a mirror angle determination unit 220, a laser irradiation control unit 230, and a storage unit 240. The image acquisition unit 210, the mirror angle determination unit 220, and the laser irradiation control unit 230 are each realized by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI, ASIC, FPGA, GPU, or SOC, or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device such as a HDD or flash memory (a storage device having a non-transitory storage medium), or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM and installed by inserting the storage medium into a drive device. The storage unit 240 is realized by a storage device such as a HDD, flash memory, or RAM. The storage unit 240 stores, for example, target image data 242 and an optimization parameter set 244 .

[0056] The image acquisition unit 210 acquires a set of a visible image and a parallax image of the target T captured by the camera 10, and stores them as target image data 242. The parallax image includes depth information of the target T as pixel values, along with the two-dimensional position of the target T based on the camera coordinate system. Therefore, the image acquisition unit 210 calculates a three-dimensional coordinate Z in the camera coordinate system for the target T in the parallax image, associates it with the target image data 242, and stores it in the storage unit 240. Unlike the first embodiment, the acquired visible image and parallax image of the target T are one set, and the calculated three-dimensional coordinate Z is also one. The optimization parameter set 244 is a parameter u optimized by the information processing device 100 according to the first embodiment.

[0057] FIG. 9 is a diagram illustrating a method for determining a mirror angle by the mirror angle determination unit 220. Based on the optimization parameter group 244, the mirror angle determination unit 220 determines the mirror angle α, i.e., the mirror angle α0 of the first mirror M1 and the mirror angle α1 of the second mirror M2, so that the laser line is irradiated onto the target T. In FIG. 9, s represents the foot of a perpendicular line dropped from the three-dimensional position Z of the target in the camera coordinate system to the laser line, and is also a coordinate defined in the camera coordinate system. More specifically, s is expressed according to the following equation (14):

[0058]

number

[0059] In equation (14), q2 C (α) represents the coordinates of the irradiation point P in the galvanometer mirror coordinate system converted to the camera coordinate system. Similarly, v2 C (α) is also the vector v2 in the galvanometer mirror coordinate system converted to the camera coordinate system, and q2 C and v2 C are both functions of the two mirror angles α = (α0, α1).

[0060] Unlike during calibration, the three-dimensional position Z of the target and the foot s of the perpendicular line are the coordinate values ​​of a single point, so the square of the distance Q between Z and s is defined as the error according to the following equation (15).

[0061]

number

[0062] Here, we consider changing α by a small amount δα to bring s closer to Z, and find δα that minimizes the following equation (16).

[0063]

number

[0064] In equation (16), J B is F u represents the Jacobian matrix of (α). δα, which minimizes equation (16), can be transformed to δQ / δα=0, yielding the following equation (17).

[0065]

number

[0066] In equation (17), J + B is J B The mirror angle determination unit 220 repeats updating according to the update equation α=α+δα until Q becomes equal to or less than a predetermined value, or until it sufficiently converges to a minimum value and δα becomes equal to or less than a predetermined value.

[0067] Once the mirror angle α=(α0, α1) is determined by the mirror angle determination unit 220, the laser irradiation control unit 230 provides the determined mirror angle α as a command value to the laser irradiation device 50, and the laser irradiation device 50 adjusts the angles of the first mirror M1 and the second mirror M2 to the determined mirror angle α and irradiates the laser. This allows the laser to be accurately irradiated onto the target.

[0068] [Processing flow] Next, the flow of processing executed by the control device 200 according to the second embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the flow of processing executed by the control device 200 according to the second embodiment.

[0069] First, the image acquisition unit 210 acquires an image of the target T (step S200). Next, the mirror angle determination unit 220 calculates the square of the distance Q between the coordinates of the target T and the laser line based on the optimization parameter u and the mirror angle α as an error (step S202). Next, the mirror angle determination unit 220 executes optimization processing for the mirror angle α until the error Q becomes equal to or less than a predetermined value (step S204). The laser irradiation control unit 230 provides the mirror angle α at which the error Q becomes equal to or less than the predetermined value as a command value to the laser irradiation device 50, causing the laser to irradiate (step S206). This completes the processing of this flowchart.

[0070] According to the second embodiment described above, the mirror angle for irradiating the laser is set by minimizing the distance between the position of the target T and the laser line, based on the parameters optimized by the information processing device 100 according to the first embodiment. This allows the laser to be accurately irradiated onto the target.

[0071] According to the above-described embodiment, it is possible to realize a more suitable calibration for targeting flying pests, etc. with a high-power laser. Furthermore, according to the embodiment, even if the stereo camera or galvanometer mirror is reinstalled, or if the device is misaligned due to some external impact, the above-described calibration allows the positional relationship to be quickly identified.

[0072] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0073] 10 Camera 50 Laser irradiation device 100 Information processing device 110 Image acquisition unit 120 Error calculation section 130 Parameter Swarm Optimization Unit 142 Irradiation point image data 200 control device 210 Image acquisition unit 220 Mirror angle determination unit 230 Laser irradiation control unit 240 Storage section 242 Target Image Data 244 Optimization Parameter Set

Claims

1. an image acquisition unit that acquires a plurality of images by capturing images of irradiation points of a predetermined structure irradiated with a laser from the laser irradiation device by a camera while changing the scanning angle of a laser scanning device that is a component of the laser irradiation device; an error calculation unit that calculates, as an error, a distance between the coordinates of the irradiation point captured in the plurality of images and a laser line defined by a group of parameters that determine a positional relationship between the laser scanning device and the camera; a parameter set optimization unit that optimizes the parameter set so as to minimize the error, Information processing device.

2. the parameter group optimization unit changes the parameters included in the parameter group to change the laser straight line, and optimizes the parameter group by bringing the laser straight line closer to the coordinates of the irradiation point. The information processing device according to claim 1 .

3. the camera is a 3D camera; When a stereo camera, which is a type of the 3D camera, is used, the coordinates of the irradiation point are three-dimensional coordinates calculated from the visible image and the parallax image included in the plurality of images. The information processing device according to claim 1 .

4. The group of parameters includes at least one of a translation amount from a camera coordinate system based on the camera to a laser scanning device coordinate system based on the laser scanning device, a rotation amount from the camera coordinate system to the laser scanning device coordinate system, an arbitrary point on a laser line irradiated by the laser irradiation device, a direction of the laser line before being reflected by the laser scanning device, and a deviation between a command value of a scanning angle of the laser scanning device and an actual angle. The information processing device according to claim 1 .

5. A control device that controls the laser irradiation device based on a parameter group optimized by the information processing device according to claim 1, an image acquisition unit that acquires an image obtained by capturing an image of the target with a camera; a scan angle determination unit that determines a scan angle that minimizes a distance between the coordinates of the target captured in the image and a laser line defined by the parameter group and a scan angle of the laser scanning device; a laser irradiation control unit that gives the determined scanning angle as a command value to the laser irradiation device to irradiate the laser, Control device.

6. The computer While changing the scanning angle of a laser scanning device, which is a component of the laser irradiation device, a predetermined structure is irradiated with a laser from the laser irradiation device multiple times, and a plurality of images are obtained by capturing images of the irradiation points with a camera; calculating, as an error, a distance between the coordinates of the irradiation point captured in the plurality of images and a laser line defined by a group of parameters that determine a positional relationship between the laser scanning device and the camera; optimizing the set of parameters to minimize the error; Information processing methods.

7. On the computer, While changing the scanning angle of a laser scanning device, which is a component of the laser irradiation device, a predetermined structure is irradiated with a laser from the laser irradiation device multiple times, and a plurality of images are obtained by capturing images of the irradiation points with a camera; calculating, as an error, a distance between the coordinates of the irradiation point captured in the plurality of images and a laser straight line defined by a group of parameters that determine a positional relationship between the laser scanning device and the camera; optimizing the set of parameters to minimize the error; program.

Images