Automatic calibration method of visual positioning system, visual positioning system and spraying robot

Abstract

The invention discloses an automatic calibration method of a visual positioning system, the visual positioning system and a spraying robot, the visual positioning system comprises an image acquisition device and a laser projection device, the image acquisition device is used for acquiring a ground image, and the laser projection device projects a laser pattern of a preset geometric structure to the ground; the automatic calibration method comprises the steps that after the visual positioning system is powered on, the laser projection device is started, and image coordinates of a laser pattern on the ground are acquired through the image acquisition device; obtaining pose information of the image acquisition device according to the image coordinates and pre-stored design position information of the laser pattern; a carrier carrying the visual positioning system is controlled to start moving, and a moving image of the laser pattern is acquired through an image acquisition device; obtaining a metric scale of the visual positioning system according to the motion image; and calibrating an external parameter matrix of the image acquisition device according to the pose information, the metric scale and the internal parameter matrix of the image acquisition device.

Description

Technical Field

[0001] This application relates to the field of visual positioning technology, specifically to an automatic calibration method for a visual positioning system, a visual positioning system, and a spraying robot. Background Technology

[0002] Currently, existing high-precision positioning systems typically rely on the complex fusion of multiple sensors, such as inertial measurement units, wheel speed encoders, and real-time dynamic differential positioning technology, in order to achieve positioning accuracy. While these systems can achieve high precision, they have significant technical limitations.

[0003] High-precision sensors are expensive, especially real-time dynamic differential positioning modules, which significantly increase the overall material cost of the system. The fusion of multiple sensors requires complex hardware synchronization and software algorithms, increasing the difficulty of system development and integration. Furthermore, before leaving the factory or after deployment, each sensor needs precise offline calibration. This process typically requires specialized calibration tools and facilities, and is manually performed by technicians, making it extremely tedious and time-consuming. Summary of the Invention

[0004] In view of this, this application provides an automatic calibration method for a visual positioning system and a spraying robot, which reduces the development and integration difficulty of the positioning system, realizes automatic calibration of the visual positioning system, simplifies the calibration process, and reduces time consumption. The technical solution of this application is as follows: The first aspect of this application provides an automatic calibration method for a visual positioning system. The visual positioning system includes an image acquisition device and a laser projection device. The image acquisition device is used to acquire ground images, and the laser projection device projects a laser pattern with a preset geometric structure onto the ground. The automatic calibration method includes: (a) after the visual positioning system is powered on, starting the laser projection device and acquiring the image coordinates of the laser pattern on the ground through the image acquisition device; (b) obtaining the pose information of the image acquisition device based on the image coordinates and the pre-stored design position information of the laser pattern; (c) controlling the vehicle carrying the visual positioning system to start moving and acquiring a motion image of the laser pattern through the image acquisition device; (d) obtaining the metric scale of the visual positioning system based on the motion image; and (e) calibrating the extrinsic parameter matrix of the image acquisition device based on the pose information, the metric scale, and the intrinsic parameter matrix of the image acquisition device.

[0005] In one embodiment of this application, obtaining the image coordinates of the laser pattern on the ground using the image acquisition device includes: obtaining a ground image of the laser pattern using the image acquisition device; obtaining laser point features of the target point of the laser pattern from the ground image; and obtaining the laser point feature coordinates of the laser point features as the image coordinates.

[0006] In one embodiment of this application, the design location information includes the world coordinates of the target point in the laser pattern, and the image coordinates include the laser point feature coordinates of the target point; obtaining the pose information of the image acquisition device based on the image coordinates and the pre-stored design location information of the laser pattern includes: solving a PnP problem based on the intrinsic parameter matrix of the image acquisition device, the world coordinates of the target point, and the laser point feature coordinates to obtain the altitude and heading angle of the image acquisition device as the pose information.

[0007] In one embodiment of this application, the laser pattern includes a laser grid; obtaining the metric scale of the visual positioning system based on the motion image includes: obtaining the physical size of the laser grid based on the design location information; and using the physical size as a geometric scale constraint factor of the motion image to obtain the metric scale of the visual positioning system.

[0008] In one embodiment of this application, calibrating the extrinsic matrix of the image acquisition device based on the pose information, the metric scale, and the intrinsic parameter matrix of the image acquisition device includes: calibrating the extrinsic matrix using bundle adjustment based on the pose information, the metric scale, and the intrinsic parameter matrix of the image acquisition device.

[0009] In one embodiment of this application, the method further includes: when the vehicle is performing a moving operation, acquiring the image coordinates of the laser pattern on the ground every preset time interval, and performing steps (b) to (e).

[0010] A second aspect of this application provides a visual positioning system, including an image acquisition device, a laser projection device, and a control device, wherein the control device is connected to the image acquisition device and the laser projection device; the image acquisition device is used to acquire ground images; the laser projection device projects a laser pattern with a preset geometric structure onto the ground; and the control device is used to execute the automatic calibration method.

[0011] In one embodiment of this application, the control device is further configured to acquire ground texture features through the image acquisition device, and acquire the attitude information and position information of the vehicle equipped with the visual positioning system based on the ground texture features, the pre-stored feature map, the extrinsic parameter matrix and the intrinsic parameter matrix of the image acquisition device.

[0012] A third aspect of this application provides a spraying robot, including the aforementioned visual positioning system.

[0013] In one embodiment of this application, the laser projection device includes at least two cross laser emitting modules, which are evenly arranged on both sides of the spraying array of the spraying robot; the multiple cross laser emitting modules are used to project cross laser patterns onto the spraying area of ​​the spraying array, and the multiple cross laser patterns together form a laser pattern marking the spraying area.

[0014] It is understood that the visual positioning system of this application includes an image acquisition device and a laser projection device. After the visual positioning system is powered on, the laser projection device is activated and the image coordinates of the laser pattern on the ground are acquired through the image acquisition device. The pose information of the image acquisition device is obtained through the image coordinates and the design position information of the laser pattern. Then, the motion image of the laser pattern during the movement is acquired. The metric scale of the visual positioning system is obtained based on the motion image. Finally, automatic calibration is achieved based on the pose information, the metric scale, and the intrinsic parameter matrix of the image acquisition device. This can reduce the development and integration difficulty of the positioning system, realize the automatic calibration of the visual positioning system, simplify the calibration process of the positioning system, and reduce the time consumption. Attached Figure Description

[0015] Figure 1 This is a schematic block diagram of a visual positioning system provided in an embodiment of this application.

[0016] Figure 2 This is a flowchart illustrating an automatic calibration method for a visual positioning system provided in an embodiment of this application.

[0017] Figure 3 This is a flowchart illustrating an image coordinate acquisition method provided in an embodiment of this application.

[0018] Figure 4 This is a flowchart illustrating a method for obtaining metric dimensions provided in an embodiment of this application.

[0019] Figure 5 This is a flowchart illustrating another automatic calibration method for a visual positioning system provided in an embodiment of this application.

[0020] Figure 6 This is a schematic block diagram of a visual positioning system provided in an embodiment of this application.

[0021] Figure 7 This is a schematic block diagram of a spraying robot provided in an embodiment of this application. Detailed Implementation

[0022] It should be noted that in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects, not to describe a specific order or sequence.

[0023] It should also be noted that the methods disclosed in the embodiments of this application or the methods shown in the flowcharts include one or more steps for implementing the method. Without departing from the scope of the claims, the execution order of multiple steps can be interchanged, and some steps can also be deleted.

[0024] Currently, existing high-precision positioning systems typically rely on the complex fusion of multiple sensors, such as inertial measurement units, wheel speed encoders, and real-time dynamic differential positioning technology, in order to achieve positioning accuracy. While these systems can achieve high precision, they have significant technical limitations.

[0025] High-precision sensors are expensive, especially real-time dynamic differential positioning modules, which significantly increase the overall material cost of the system. The fusion of multiple sensors requires complex hardware synchronization and software algorithms, increasing the difficulty of system development and integration. Furthermore, before leaving the factory or after deployment, each sensor needs precise offline calibration. This process typically requires specialized calibration tools and facilities, and is manually performed by technicians, making it extremely tedious and time-consuming.

[0026] This application provides an automatic calibration method for a visual positioning system and a spraying robot, which reduces the difficulty of developing and integrating the positioning system, and realizes automatic calibration of the visual positioning system, simplifies the calibration process of the positioning system, and reduces the time consumption.

[0027] Please refer to Figure 1 , Figure 1 This is a schematic block diagram of a visual positioning system provided in an embodiment of this application. The visual positioning system 100 includes an image acquisition device 110 and a laser projection device 120. The image acquisition device 110 is used to acquire ground images, and the laser projection device 120 projects a laser pattern with a preset geometric structure onto the ground.

[0028] In this embodiment, the visual positioning system 100 is applicable to various mobile vehicles. For example, the visual positioning system 100 can be installed in a work robot. The image acquisition device 110 of the visual positioning system 100 acquires ground images for visual navigation, guiding the work robot to perform operations in the target work area. The work robot includes spraying robots, weeding robots, and snow removal robots, etc., and is not limited here.

[0029] The image acquisition device 110 can be fixed on a mobile vehicle, with the image acquisition angle facing the ground of the work area, so that the image acquisition device 110 can acquire ground images of the work area. The laser projection device 120 can also be fixed on a mobile vehicle, with its projection angle also facing the ground of the work area, so that the laser pattern projected by the laser projection device 120 is within the work area.

[0030] In some embodiments, the visual positioning system 100 further includes a control device connected to the image acquisition device 110 and the laser projection device 120. The control device is used to control the image acquisition device 110 to acquire ground images and to control the laser projection device 120 to project a laser pattern with a preset geometric structure onto the ground. The image acquisition device 110 may include a camera, a monocular camera, etc., and is not limited thereto.

[0031] Next, combined Figure 1 This application introduces an automatic calibration method for a visual positioning system, as described in its embodiments. Please refer to [link / reference]. Figure 2 Specifically, it includes the following steps: Step S21: After the visual positioning system is powered on, start the laser projection device and obtain the image coordinates of the laser pattern on the ground through the image acquisition device.

[0032] In this embodiment, when the visual positioning system is applied to a spraying robot, the system powers on after the robot is started. Upon power-on, the control device within the system initializes and begins a static calibration process. During this process, the control device activates the laser projection device, projecting a laser pattern with a preset geometric structure onto the ground in the work area. Simultaneously, the control device also activates the image acquisition device to acquire images of the laser pattern onto the ground in the work area. After obtaining the image of the laser pattern, the control device determines its image coordinates.

[0033] Step S22: Obtain the pose information of the image acquisition device based on the image coordinates and the pre-stored laser pattern design position information.

[0034] In this embodiment, the aforementioned design position information includes the world coordinates of the laser pattern, which can be pre-stored in the control device as reference coordinates for automatic calibration. The control device compares the world coordinates of the laser pattern with the image coordinates to obtain the pose information of the image acquisition device. The pose information may include, for example, the height and viewing angle of the image acquisition device.

[0035] In some embodiments, the above-mentioned design location information can be obtained through a limited number of trials. For example, after installing the laser projection device on the spraying robot, the manufacturer can use the center point of the spraying robot's tool as the origin, and through a limited number of trials, obtain the world coordinates of the laser pattern and store it in the control device.

[0036] Step S23: Control the vehicle equipped with the visual positioning system to start moving, and acquire the motion image of the laser pattern through the image acquisition device.

[0037] In this embodiment, the vehicle is a spraying robot. After obtaining the pose information from the image acquisition device, the control device can send a notification to the controller of the spraying robot, so that the controller can control the spraying robot to start moving and perform a movement calibration process. After the spraying robot starts moving, the control device of the vision positioning system then acquires the motion image of the laser pattern through the image acquisition device.

[0038] In some embodiments, during the motion calibration process, the controller controls the spraying robot to travel at a preset speed for a preset time, after which the spraying robot stops. The control device of the visual positioning system obtains motion images for the preset time and performs the motion calibration process based on the motion images for the preset time.

[0039] Step S24: Obtain the metric scale of the visual positioning system based on the motion image.

[0040] In this embodiment, after acquiring a moving image, the control device of the visual positioning system can obtain the metric dimensions of the visual positioning system based on the moving image. For example, the control device can pre-store the size information of the laser pattern, and obtain the metric dimensions based on the size information and the moving image. The metric dimensions include millimeters, centimeters, and decimeters, etc., and are not limited here.

[0041] Step S25: Based on the pose information, metric scale, and intrinsic parameter matrix of the image acquisition device, calibrate the extrinsic parameter matrix of the image acquisition device.

[0042] In this embodiment, after obtaining the pose information and metric scale of the image acquisition device, the control device of the visual positioning system calibrates the extrinsic matrix of the image acquisition device by combining the intrinsic parameter matrix of the image acquisition device. For example, global optimization algorithms such as bundle adjustment can be used to calibrate the extrinsic matrix, thereby eliminating errors caused by the offset of the installation parameters of the image acquisition device due to changes in the on-site environment or vibration.

[0043] It is understood that the visual positioning system of this application includes an image acquisition device and a laser projection device. After the visual positioning system is powered on, the laser projection device is activated and the image coordinates of the laser pattern on the ground are acquired through the image acquisition device. The pose information of the image acquisition device is obtained through the image coordinates and the design position information of the laser pattern. Then, the motion image of the laser pattern during the movement is acquired. The metric scale of the visual positioning system is obtained based on the motion image. Finally, automatic calibration is achieved based on the pose information, the metric scale, and the intrinsic parameter matrix of the image acquisition device. This can reduce the development and integration difficulty of the positioning system, realize the automatic calibration of the visual positioning system, simplify the calibration process of the positioning system, and reduce the time consumption.

[0044] Please refer to Figure 3 , Figure 3 This application provides a flowchart illustrating an image coordinate acquisition method, which specifically includes the following steps: Step S31: Acquire a ground image of the laser pattern using an image acquisition device.

[0045] In this embodiment of the application, during the static calibration process, the control device of the visual positioning system first acquires a ground image of the laser pattern in the work area through an image acquisition device.

[0046] Step S32: Obtain the laser point features of the target points of the laser pattern from the ground image.

[0047] In this embodiment of the application, after obtaining a ground image, the control device can extract the laser point features of the target points of the laser pattern from the ground image. The target points include the corner points of the laser pattern; for example, if the laser pattern is a grid pattern, the target points can be the corner points within the grid pattern.

[0048] Step S33: Obtain the laser point feature coordinates as image coordinates.

[0049] In this embodiment of the application, after obtaining the laser point features, the control device can extract the laser point feature coordinates from the ground image as the aforementioned image coordinates.

[0050] In some embodiments, the design location information includes the world coordinates of the target point in the laser pattern, and the image coordinates include the laser point feature coordinates of the target point. The above-mentioned method of obtaining the pose information of the image acquisition device based on the image coordinates and the pre-stored design location information of the laser pattern includes: solving a PnP problem based on the intrinsic parameter matrix of the image acquisition device, the world coordinates of the target point, and the laser point feature coordinates to obtain the altitude and heading angle of the image acquisition device as pose information.

[0051] In some embodiments, the laser pattern described above includes a laser grid. Please refer to... Figure 4 The above step S24 may further include the following steps: Step S241: Obtain the physical dimensions of the laser grid based on the design location information.

[0052] In this embodiment of the application, the design location information may include the ground clearance of the laser projection device and the height-related grid magnification factor. The control device of the visual positioning system can obtain the physical size of the laser grid based on the ground clearance of the laser projection device and the grid magnification factor.

[0053] Step S242: Use physical dimensions as geometric scale constraint factors for motion images to obtain the metric scale of the visual positioning system.

[0054] In this embodiment, after obtaining the physical dimensions of the laser grid, the control device of the visual positioning system uses these physical dimensions as geometric scale constraint factors for the moving image to obtain the metric scale of the visual positioning system through a reverse calculation process.

[0055] In some embodiments, the above-mentioned calibration of the extrinsic matrix of the image acquisition device based on pose information, metric scale, and intrinsic parameter matrix of the image acquisition device includes: calibrating the extrinsic matrix using bundle adjustment based on pose information, metric scale, and intrinsic parameter matrix of the image acquisition device.

[0056] Please refer to Figure 5 , Figure 5 For another automatic calibration method of the visual positioning system provided in the embodiments of this application, please refer to Figure 2 Specifically, it includes the following steps: Step S51: After the visual positioning system is powered on, start the laser projection device and obtain the image coordinates of the laser pattern on the ground through the image acquisition device.

[0057] Step S52: Obtain the pose information of the image acquisition device based on the image coordinates and the pre-stored laser pattern design position information.

[0058] Step S53: Control the vehicle equipped with the visual positioning system to start moving, and acquire the motion image of the laser pattern through the image acquisition device.

[0059] Step S54: Obtain the metric scale of the visual positioning system based on the motion image.

[0060] Step S55: Based on the pose information, metric scale, and intrinsic parameter matrix of the image acquisition device, calibrate the extrinsic parameter matrix of the image acquisition device.

[0061] Step S56: When the vehicle is moving, the image coordinates of the laser pattern on the ground are acquired every preset time interval, and steps S52 to S55 are executed.

[0062] In this embodiment of the application, when the visual positioning system is applied to a spraying robot, after the spraying robot has been moving and operating for a period of time, the laser projection device will be activated again to obtain the image coordinates of the laser pattern on the ground and perform an automatic calibration process again to compensate for the slight parameter shifts caused by temperature changes or long-term vibration, and to avoid errors in visual positioning.

[0063] Please refer to Figure 6 , Figure 6 This is a schematic block diagram of a visual positioning system provided in an embodiment of this application. The visual positioning system 100 includes an image acquisition device 110, a laser projection device 120, and a control device 130. The control device 130 is connected to the image acquisition device 110 and the laser projection device 120. The image acquisition device 110 is used to acquire ground images. The laser projection device 120 projects a laser pattern with a preset geometric structure onto the ground. The control device 130 is used to execute the automatic calibration method of any of the above embodiments.

[0064] The control device 130 is also used to acquire ground texture features through the image acquisition device 110, and acquire the attitude information and position information of the vehicle equipped with the visual positioning system 100 based on the ground texture features, the pre-stored feature map, the extrinsic parameter matrix and the intrinsic parameter matrix of the image acquisition device 110.

[0065] This application also provides a spraying robot, such as... Figure 7 As shown, the spraying robot 10 includes a spraying array 101 and a visual positioning system 100 of any of the above embodiments.

[0066] The laser projection device 120 includes at least two cross-shaped laser emitting modules, which are evenly arranged on both sides of the spraying array 101 of the spraying robot 10. Each cross-shaped laser emitting module 121 is used to project a cross-shaped laser pattern onto the spraying area of ​​the spraying array 101, and the cross-shaped laser patterns projected by multiple cross-shaped laser emitting modules together form a laser pattern marking the spraying area.

[0067] It is understood that the beneficial effects of the spraying robot 10 can be referred to the beneficial effects of the automatic calibration method in the foregoing embodiments, and will not be repeated here.

[0068] This application also provides a computer storage medium storing a computer program, which, when executed by a control device, causes the control device to perform the above-described automatic calibration method.

[0069] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer storage medium or transmitted through the computer storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0070] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation can be combined arbitrarily.

[0071] The embodiments described above are merely preferred embodiments of this application and are not intended to limit the scope of this application. Any modifications and improvements made by those skilled in the art to the technical solutions of this application without departing from the spirit of this application should fall within the protection scope defined by the claims of this application.

Claims

1. An automatic calibration method for a visual positioning system, characterized in that, The visual positioning system includes an image acquisition device and a laser projection device. The image acquisition device is used to acquire ground images, and the laser projection device projects a laser pattern with a preset geometric structure onto the ground. The automatic calibration method includes: (a) After the visual positioning system is powered on, the laser projection device is activated, and the image coordinates of the laser pattern on the ground are acquired through the image acquisition device; (b) Obtain the pose information of the image acquisition device based on the image coordinates and the pre-stored design position information of the laser pattern; (c) Control the vehicle equipped with the visual positioning system to start moving, and acquire the motion image of the laser pattern through the image acquisition device; (d) Obtain the metric scale of the visual positioning system based on the motion image; (e) The extrinsic parameter matrix of the image acquisition device is calibrated based on the pose information, the metric scale, and the intrinsic parameter matrix of the image acquisition device.

2. The automatic calibration method as described in claim 1, characterized in that, The step of acquiring the image coordinates of the laser pattern on the ground through the image acquisition device includes: The ground image of the laser pattern is acquired using the image acquisition device; The laser point features of the target points of the laser pattern are obtained from the ground image; The coordinates of the laser point feature are obtained as the image coordinates.

3. The automatic calibration method as described in claim 1, characterized in that, The design location information includes the world coordinates of the target point in the laser pattern, and the image coordinates include the laser point feature coordinates of the target point; The step of obtaining the pose information of the image acquisition device based on the image coordinates and the pre-stored design position information of the laser pattern includes: Based on the intrinsic parameter matrix of the image acquisition device, the world coordinates of the target point, and the characteristic coordinates of the laser point, the PnP problem is solved to obtain the altitude and heading angle of the image acquisition device as the pose information.

4. The automatic calibration method as described in claim 1, characterized in that, The laser pattern includes a laser grid; obtaining the metric scale of the visual positioning system based on the motion image includes: The physical dimensions of the laser grid are obtained based on the design location information; The physical dimensions are used as geometric scale constraint factors for the moving image to obtain the metric scale of the visual positioning system.

5. The automatic calibration method as described in claim 1, characterized in that, The step of calibrating the extrinsic parameter matrix of the image acquisition device based on the pose information, the metric scale, and the intrinsic parameter matrix of the image acquisition device includes: The extrinsic parameter matrix is ​​calibrated using the bundle adjustment method based on the pose information, the metric scale, and the intrinsic parameter matrix of the image acquisition device.

6. The automatic calibration method as described in claim 1, characterized in that, Also includes: When the vehicle is moving, the image coordinates of the laser pattern on the ground are acquired every preset time interval, and steps (b) to (e) are executed.

7. A visual positioning system, characterized in that, It includes an image acquisition device, a laser projection device, and a control device, wherein the control device is connected to the image acquisition device and the laser projection device; The image acquisition device is used to acquire ground images; The laser projection device projects a laser pattern with a preset geometric structure onto the ground; The control device is used to perform the automatic calibration method as described in any one of claims 1 to 6.

8. The visual positioning system as described in claim 7, characterized in that, The control device is also used to acquire ground texture features through the image acquisition device, and acquire the attitude information and position information of the vehicle equipped with the visual positioning system based on the ground texture features, the pre-stored feature map, the extrinsic parameter matrix and the intrinsic parameter matrix of the image acquisition device.

9. A spraying robot, characterized in that, Including the visual positioning system as described in claim 7 or 8.

10. The spraying robot as described in claim 9, characterized in that, The laser projection device includes at least two cross-shaped laser emitting modules, which are evenly arranged on both sides of the spraying array of the spraying robot; Multiple cross-shaped laser emission modules are used to project cross-shaped laser patterns onto the spraying area of ​​the spray array, and the multiple cross-shaped laser patterns together form a laser pattern marking the spraying area.

Images