Calibration method, device and drone

Abstract

The present application provides a calibration method, apparatus, and drone. The method comprises: obtaining a first three-dimensional coordinate system and a pose trajectory of a camera deployed on a drone over a first time period; projecting the pose trajectory onto a first plane to obtain a plurality of projection points; fitting the plurality of projection points on the first plane to obtain a first straight line; determining a second three-dimensional coordinate system for the camera based on the first straight line; and determining extrinsic parameters of the camera relative to the drone based on the first and second three-dimensional coordinate systems. The above method can improve the efficiency of camera extrinsic calibration and reduce calibration costs.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of drones, and more specifically, to a calibration method, device, and drone. Background Art

[0002] Camera calibration is the process of obtaining camera parameters. Camera parameters include intrinsic parameters and extrinsic parameters. Intrinsic parameters are the camera's own parameters, while extrinsic parameters are parameters related to the camera's installation position, such as pitch, roll, and yaw angles.

[0003] Traditional production line camera extrinsic calibration typically uses a hand-eye calibration method. This involves using a target (e.g., a checkerboard calibration plate or a QR code calibration plate) to adjust the target's angle, either manually or through a robotic arm, so that the camera can capture images of the checkerboard calibration plate at various angles. This approach is complex and time-consuming, and it relies on a reference object for calibration. Furthermore, the positional relationship between the reference object and the camera must be clearly defined during the calibration process, resulting in high implementation costs and low calibration efficiency.

[0004] Therefore, how to improve the efficiency of camera extrinsic calibration and reduce the calibration cost is an urgent problem to be solved. Summary of the Invention

[0005] The embodiments of the present application provide a calibration method, device, and drone, which can improve the efficiency of camera external parameter calibration, reduce calibration costs, and thereby improve the performance of the drone in agricultural operations.

[0006] In a first aspect, a calibration method is provided, which includes: obtaining a first three-dimensional coordinate system of a camera and a posture trajectory of the camera within a first time period, wherein the camera is deployed on a drone; projecting the posture trajectory onto a first plane to obtain a plurality of projection points; fitting the plurality of projection points on the first plane to obtain a first straight line; determining a second three-dimensional coordinate system of the camera based on the first straight line; and determining external parameters of the camera relative to the drone based on the first three-dimensional coordinate system and the second three-dimensional coordinate system.

[0007] Optionally, the first three-dimensional coordinate system of the camera can be the initial coordinate system of the camera deployed on the drone, for example, the x-axis of the first three-dimensional coordinate system of the camera points to the right of the drone, the y-axis points to the ground, and the z-axis points to the front of the drone.

[0008] Alternatively, the camera's trajectory during the first time period may be a combination of multiple straight lines, or a combination of straight lines and curves. For example, if a camera is deployed on a drone and the drone travels 50 meters forward and 30 meters to the right, the camera's trajectory during the first time period may be a combination of two straight lines.

[0009] Optionally, a least square method or a random sample consensus (RANSAC) algorithm may be used to fit the multiple projection points on the first plane to obtain the first straight line.

[0010] In the embodiment of the present application, the camera's position trajectory over a first time period can be projected onto a first plane to obtain a first straight line. Based on the second three-dimensional coordinate system obtained from the first straight line and the first three-dimensional coordinate system, the camera's external parameters relative to the drone can be determined. This method eliminates the need for specific calibration objects, improves camera calibration efficiency, and reduces calibration costs. When agricultural drones are calibrated using this method, their performance in agricultural operations can be enhanced.

[0011] In combination with the first aspect, in certain implementations of the first aspect, determining the second three-dimensional coordinate system of the camera based on the first straight line includes: using the first straight line as the x-axis of the second three-dimensional coordinate system; using a second straight line perpendicular to the first straight line as the y-axis on the second three-dimensional coordinate system, and the second straight line is located on the first plane; using a third straight line perpendicular to the first plane as the z-axis of the second three-dimensional coordinate system, and the third straight line intersects with the first straight line and the second straight line.

[0012] In an embodiment of the present application, the first straight line can be used as the x-axis of the second three-dimensional coordinate system, and the second straight line and the third straight line perpendicular to the first straight line can be used as the y-axis and z-axis of the second three-dimensional coordinate system. In this way, the second three-dimensional coordinate system can be constructed relatively quickly, thereby facilitating the drone to complete the calibration of the camera based on the relative position relationship between the first three-dimensional coordinate system and the second three-dimensional coordinate system.

[0013] In combination with the first aspect, in certain implementations of the first aspect, determining the external parameters of the camera relative to the drone based on the first three-dimensional coordinate system and the second three-dimensional coordinate system includes: determining the rotation matrix of the camera relative to the drone based on the relative position relationship between the coordinate axes of the first three-dimensional coordinate system and the coordinate axes of the second three-dimensional coordinate system.

[0014] In an embodiment of the present application, the rotation matrix of the camera relative to the drone can be determined based on the relative position relationship between the coordinate axes of the first three-dimensional coordinate system and the coordinate axes of the second three-dimensional coordinate system. In this way, the drone can achieve accurate attitude estimation, coordinate transformation, calibration and camera calibration based on the rotation matrix, thereby enhancing the navigation, perception and control capabilities of the drone and improving the efficiency and accuracy of agricultural operations.

[0015] In combination with the first aspect, in certain implementations of the first aspect, obtaining the pose trajectory of the camera within the first time length includes: obtaining image sequence information taken by the camera within the first time length and the pose information of the drone within the first time length; using a visual simultaneous localization and mapping (SLAM) algorithm to process the image sequence information and the pose information of the drone within the first time length to obtain the pose trajectory of the camera.

[0016] In the embodiment of the present application, the camera's image sequence and the drone's posture information can be processed in real time through the visual SLAM algorithm. This can provide a more comprehensive and accurate posture estimation, which is conducive to the rapid calibration of the camera and drone, thereby providing a more reliable foundation for the drone's agricultural operations.

[0017] In combination with the first aspect, in certain implementations of the first aspect, the first plane is the plane where the x-axis and the y-axis are located in the drone coordinate system, and the second plane where the posture trajectory is located is parallel to the first plane.

[0018] Optionally, when the method is applied in a vehicle, the first plane may be the ground.

[0019] In an embodiment of the present application, the plane where the x-axis and y-axis in the drone coordinate system are located can be used as the first plane, and the first plane is parallel to the second plane where the camera's posture trajectory is located. In this way, the camera's posture trajectory can represent the drone's posture trajectory, thereby making the fitted first straight line more accurate and the determined external parameters of the camera relative to the drone more accurate.

[0020] In a second aspect, a calibration device is provided, which includes: an acquisition unit for acquiring a first three-dimensional coordinate system of a camera and a posture trajectory of the camera within a first time period, wherein the camera is deployed on a drone; a processing unit for: projecting the posture trajectory onto a first plane to obtain a plurality of projection points; fitting the plurality of projection points on the first plane to obtain a first straight line; determining a second three-dimensional coordinate system of the camera based on the first straight line; and determining the external parameters of the camera relative to the drone based on the first three-dimensional coordinate system and the second three-dimensional coordinate system.

[0021] In combination with the second aspect, in certain implementations of the second aspect, the processing unit is specifically used to: use the first straight line as the x-axis of the second three-dimensional coordinate system; use the second straight line perpendicular to the first straight line as the y-axis on the second three-dimensional coordinate system, and the second straight line is located on the first plane; use the third straight line perpendicular to the first plane as the z-axis of the second three-dimensional coordinate system, and the third straight line intersects with the first straight line and the second straight line.

[0022] In combination with the second aspect, in certain implementations of the second aspect, the processing unit is specifically used to determine the rotation matrix of the camera relative to the drone based on the relative position relationship between the coordinate axes of the first three-dimensional coordinate system and the coordinate axes of the second three-dimensional coordinate system.

[0023] In combination with the second aspect, in certain implementations of the second aspect, the acquisition unit is further used to obtain image sequence information taken by the camera within the first time length and the posture information of the drone within the first time length; the processing unit is further used to use a visual simultaneous localization and mapping (SLAM) algorithm to process the image sequence information and the posture information of the drone within the first time length to obtain the posture trajectory of the camera.

[0024] In combination with the second aspect, in certain implementations of the second aspect, the first plane is the plane where the x-axis and y-axis in the drone coordinate system are located, and the second plane where the posture trajectory is located is parallel to the first plane.

[0025] In a third aspect, a calibration device is provided, which includes: at least one processor and a memory, wherein the at least one processor is coupled to the memory and is used to read and execute instructions in the memory, so that the device implements the method in any one of the implementation methods of the above-mentioned first aspect.

[0026] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a program code, and when the computer program code is run on a computer, the computer executes the method in any one of the implementation modes of the above-mentioned first aspect.

[0027] In a fifth aspect, a chip is provided, which includes a circuit for executing the method in any one of the implementations of the first aspect.

[0028] In a sixth aspect, a computer program product is provided, which includes a computer program. When the computer program is run, the computer executes the method in any one of the implementation modes of the first aspect.

[0029] In a seventh aspect, a smart device is provided, comprising: a calibration device in any one of the implementation methods of the second aspect above.

[0030] In combination with the seventh aspect, in some implementations of the seventh aspect, the smart device is a drone or a vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0032] Figure 2is a schematic flow chart of a calibration method provided in an embodiment of the present application;

[0033] Figure 3 is a schematic flow chart of another calibration method provided in an embodiment of the present application;

[0034] Figure 4 is a schematic diagram of the calibration method provided in an embodiment of the present application;

[0035] Figure 5 This is a calibration device provided in an embodiment of the present application;

[0036] Figure 6 This is another calibration device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The technical solution in this application will be described below with reference to the accompanying drawings.

[0038] The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of this application and the appended claims, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one, two or more. The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0039] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0040] The solutions of the embodiments of the present application can be applied to smart devices. Smart devices refer to any device, instrument, or machine with computing and processing capabilities. The smart devices in the embodiments of the present application can be robots, autonomous vehicles, smart assisted driving vehicles, unmanned aerial vehicles, smart assisted aircraft, smart home devices, etc. This application does not impose any restrictions on the smart devices. Any device that can be equipped with a camera can be included in the scope of the smart devices of this application.

[0041] The methods provided in the embodiments of this application can be applied to autonomous driving, drone navigation, agricultural drone operations, robot navigation, industrial non-contact inspection, 3D reconstruction, virtual reality, and other scenarios requiring camera calibration. Specifically, the methods provided in the embodiments of this application can be applied to agricultural drone operations, which are briefly described below.

[0042] like Figure 1 As shown, the drone 110 can be used in operation scenarios such as farmland monitoring, spraying and sowing, and plant disease and pest detection.

[0043] The drone 110 may include a sensor system. The sensor system includes a number of sensors that sense information about the environment surrounding the drone 110. For example, the sensor system may include a positioning system (the positioning system may be a GPS system, a BeiDou system, or other positioning systems), an inertial measurement unit (IMU), a radar, a laser rangefinder, and a camera 111. It should be understood that although Figure 1 While only camera 111 is shown, drone 110 may also include other sensors. To achieve better perception, information from multiple sensors can be fused. Specifically, extrinsic parameters between the sensors can be used to unify the different sensors into the same coordinate system, thereby achieving fusion of multiple sensor information.

[0044] The calibration module 112 is used to determine the external parameters of the sensor. In the embodiment of the present application, the sensor system includes a camera 111, and the calibration module 112 is used to determine the external parameters of the camera 111. Figure 1 As shown, the calibration module 112 can calibrate the camera extrinsic parameters based on the images captured by the camera 111.

[0045] The upper-layer functional module 113 can implement corresponding functions based on the external parameters of camera 111. In other words, the external parameter calibration results of camera 111 can be provided to the upper-layer services of the drone. For example, the ranging functional module can determine the distance between an obstacle and the drone from the image captured by camera 111 based on the external parameters of camera 111. For another example, the obstacle avoidance functional module can identify, evaluate, avoid, or otherwise navigate potential obstacles in the drone's environment based on the external parameters of camera 111 from the image captured by camera 111.

[0046] Furthermore, after the drone 110 is calibrated, the calibration status can be sent to the display device 114, and the calibration personnel can view the current calibration status of the drone on the display device 114. Figure 1 It is only a schematic diagram of a system architecture provided in an embodiment of the present application. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation.

[0047] The following introduces the technical problems to be solved by this application and the technical solutions adopted.

[0048] Camera calibration is the process of obtaining camera parameters. Camera parameters include intrinsic parameters and extrinsic parameters. Intrinsic parameters are the camera's own parameters, while extrinsic parameters are parameters related to the camera's installation position, such as pitch, roll, and yaw angles.

[0049] Traditional production line camera extrinsic calibration typically uses a hand-eye calibration method. This involves using a target (e.g., a checkerboard calibration plate or a QR code calibration plate) to adjust the target's angle, either manually or through a robotic arm, so that the camera can capture images of the checkerboard calibration plate at various angles. This approach is complex and time-consuming, and it relies on a reference object for calibration. Furthermore, the positional relationship between the reference object and the camera must be clearly defined during the calibration process, resulting in high implementation costs and low calibration efficiency.

[0050] In another external parameter calibration method, it is necessary to install an inertial measurement unit (IMU) on a smart device (e.g., a drone) to provide rotation information, and then estimate the translation information by filtering. This processing method can calibrate the external parameters of the camera and IMU at the same time, which can improve the efficiency and robustness of the calibration. However, this calibration method requires the installation of other sensors besides the camera on the drone, and the positional relationship between the installed sensors and the drone must be known, which has high implementation costs and low calibration efficiency.

[0051] Therefore, how to improve the efficiency of camera extrinsic calibration and reduce the calibration cost is an urgent problem to be solved.

[0052] The embodiments of the present application provide a calibration method, device, and drone, which can improve the efficiency of camera external parameter calibration, reduce calibration costs, and thereby improve the performance of drones in agricultural operations.

[0053] Figure 2 This is a calibration method provided in an embodiment of the present application. Figure 2 The calibration method shown can be applied to Figure 1 In the drone, specifically, the method 200 can be executed by the calibration module 112, and the method 200 can include steps S201 to S205.

[0054] S201, obtaining a first three-dimensional coordinate system of a camera and a position trajectory of the camera within a first time period.

[0055] Optionally, the first three-dimensional coordinate system of the camera can be the initial coordinate system of the camera deployed on the drone, for example, the x-axis of the first three-dimensional coordinate system of the camera points to the right of the drone, the y-axis points to the ground, and the z-axis points to the front of the drone.

[0056] Alternatively, the camera's trajectory during the first time period may be a combination of multiple straight lines, or a combination of straight lines and curves. For example, if a camera is deployed on a drone and the drone travels 50 meters forward and 30 meters to the right, the camera's trajectory during the first time period may be a combination of two straight lines.

[0057] In one embodiment, in step S201, obtaining the camera's pose trajectory over a first duration may include: obtaining image sequence information captured by the camera over the first duration and pose information of the drone over the first duration; and processing the image sequence information and the drone's pose information over the first duration using a visual SLAM algorithm to obtain the camera's pose trajectory. This can provide a more comprehensive and accurate pose estimation, facilitate rapid calibration of the camera and drone, and thus provide a more reliable foundation for drone agricultural operations.

[0058] S202: Project the posture trajectory onto the first plane to obtain multiple projection points.

[0059] In one embodiment, the first plane is the plane containing the x-axis and y-axis in the drone's coordinate system, and the second plane containing the pose trajectory is parallel to the first plane. In this way, the camera's pose trajectory can represent the drone's pose trajectory, making the fitted first straight line more accurate and the determined extrinsic parameters of the camera relative to the drone more accurate.

[0060] Optionally, in the above embodiment, the drone may fly at a fixed altitude, and the plane in which the drone actually flies may be parallel to the first plane and the second plane, so that the calibration result can be ensured to be more accurate.

[0061] In one embodiment, when the method is applied in a vehicle, the first plane may be the ground.

[0062] Optionally, in the above embodiment, the vehicle can travel on a relatively flat ground, and the planes where the x-axis and y-axis of the vehicle body's coordinate system are located can be parallel to the first plane and the second plane, so that the calibration result can be ensured to be more accurate.

[0063] S203: Fit the multiple projection points on the first plane to obtain a first straight line.

[0064] Optionally, a least square method or a RANSAC algorithm may be used to fit multiple projection points on the first plane to obtain a first straight line.

[0065] S204: Determine a second three-dimensional coordinate system of the camera according to the first straight line.

[0066] In one embodiment, step S204 includes using the first straight line as the x-axis of the second three-dimensional coordinate system; using a second straight line perpendicular to the first straight line as the y-axis of the second three-dimensional coordinate system, the second straight line being located on the first plane; and using a third straight line perpendicular to the first plane as the z-axis of the second three-dimensional coordinate system, the third straight line intersecting the first and second straight lines. In this way, the second three-dimensional coordinate system can be constructed relatively quickly, facilitating the drone to calibrate its camera based on the relative positional relationship between the first and second three-dimensional coordinate systems.

[0067] S205 : Determine the external parameters of the camera relative to the drone according to the first three-dimensional coordinate system and the second three-dimensional coordinate system.

[0068] In one embodiment, step S205 specifically includes determining a rotation matrix of the camera relative to the drone based on the relative positional relationship between the coordinate axes of the first three-dimensional coordinate system and the coordinate axes of the second three-dimensional coordinate system. This allows the drone to accurately perform attitude estimation, coordinate transformation, calibration, and camera calibration based on the rotation matrix, thereby enhancing the drone's navigation, perception, and control capabilities and improving the efficiency and accuracy of agricultural operations.

[0069] In the embodiment of the present application, the camera's pose trajectory over a first time period can be projected onto a first plane to obtain a first straight line. Based on the second three-dimensional coordinate system obtained from the first straight line and the first three-dimensional coordinate system, the camera's external parameters relative to the drone can be determined. This method eliminates the need for specific calibration objects to determine the camera's external parameters, improving calibration efficiency and reducing calibration costs. When agricultural drones are calibrated using this method, their performance in agricultural operations can be enhanced.

[0070] Figure 3It is a schematic flow chart of another calibration method provided in an embodiment of the present application. Method 300 may be a specific implementation of steps S201 to S205 in method 200. Method 300 may include the following steps.

[0071] S301, controlling the UAV to travel a path within a first time period.

[0072] Among them, controlling the drone to travel a path within the first time period can be controlling the drone to fly 20 meters forward first and then 10 meters to the left.

[0073] Exemplarily, the first duration may be 10 seconds.

[0074] S302: Acquire a first three-dimensional coordinate system of the camera.

[0075] Exemplarily, the first three-dimensional coordinate system of the camera may be an initial coordinate system of the camera fixed on the drone.

[0076] S303, using the visual SLAM algorithm to calculate the camera's position trajectory.

[0077] Optionally, in this step, the image sequence information captured by the camera during the first time period and the position and posture information of the drone during the first time period can be obtained first, and then the above information can be processed by the visual SLAM algorithm to obtain the position and posture trajectory of the camera.

[0078] For example, when the drone flies 20 meters forward and then 10 meters to the left, the camera's position trajectory can be as follows: Figure 4 The first three-dimensional coordinate system of the camera is reflected in the pose trajectory as shown in (a). Figure 4 As shown in (b) in .

[0079] It should be understood that when the drone flies for a period of time, the forward direction of the drone can be fitted into the x-axis direction of the drone coordinate system. Since the camera and the drone are in a rigid connection state, the posture trajectory of the camera can also represent the posture trajectory of the drone.

[0080] S304: Project the points in the camera's posture trajectory within a preset range onto a first plane, fit the projected points to obtain a first straight line, and use the first straight line as the x-axis of the second three-dimensional coordinate system.

[0081] For example, the point where the camera pose is within the preset range can be the camera pose trajectory point corresponding to the first 50 meters of the UAV flight.

[0082] S305 : Use a second straight line located in the first plane and perpendicular to the first straight line as the y-axis of the second three-dimensional coordinate system.

[0083] S306 : Generate a z-axis of a second three-dimensional coordinate system according to the x-axis and the y-axis according to the right-handed coordinate system.

[0084] For example, the established second three-dimensional coordinate system can be as follows Figure 4 As shown in (c) in .

[0085] S307 : Determine the external parameters of the camera relative to the drone based on the positional relationship between the first three-dimensional coordinate system and the second three-dimensional coordinate system.

[0086] For example, Figure 4 The first three-dimensional coordinate system shown in (b) is rotated to Figure 4 The position of the second three-dimensional coordinate system shown in (c) in the figure is used to determine the rotation matrix of the camera relative to the drone (a type of camera external parameter).

[0087] In the embodiments of the present application, the positional relationship between the first three-dimensional coordinate system and the second three-dimensional coordinate system can be used to determine the external parameters of the camera relative to the drone. This method eliminates the need for specific calibration objects, thereby improving calibration efficiency and reducing calibration costs. When agricultural drones are calibrated using this method, their performance in agricultural operations can be enhanced.

[0088] Figure 5 is a schematic diagram of a calibration device 500 provided in an embodiment of the present application. The device 500 may include an acquisition unit 510, a storage unit 520, and a processing unit 530. The acquisition unit 510 is used to acquire instructions and / or data. The acquisition unit 510 may also be referred to as a communication interface or communication unit. The storage unit 520 is used to implement corresponding storage functions and store corresponding instructions and / or data. The processing unit 530 can read the instructions and / or data in the storage unit to enable the device 500 to implement the aforementioned calibration method.

[0089] The device 500 includes: an acquisition unit 510, used to acquire a first three-dimensional coordinate system of a camera and a posture trajectory of the camera within a first time period, wherein the camera is deployed on a drone; a processing unit 530, used to: project the posture trajectory onto a first plane to obtain a plurality of projection points; fit the plurality of projection points on the first plane to obtain a first straight line; determine a second three-dimensional coordinate system of the camera based on the first straight line; and determine external parameters of the camera relative to the drone based on the first three-dimensional coordinate system and the second three-dimensional coordinate system.

[0090] In one possible implementation, the processing unit 530 is specifically used to: use the first straight line as the x-axis of the second three-dimensional coordinate system; use the second straight line perpendicular to the first straight line as the y-axis on the second three-dimensional coordinate system, and the second straight line is located on the first plane; use the third straight line perpendicular to the first plane as the z-axis of the second three-dimensional coordinate system, and the third straight line intersects with the first straight line and the second straight line.

[0091] In one possible implementation, the processing unit 530 is specifically configured to determine a rotation matrix of the camera relative to the drone based on a relative position relationship between coordinate axes of the first three-dimensional coordinate system and coordinate axes of the second three-dimensional coordinate system.

[0092] In one possible implementation, the acquisition unit 510 is also used to obtain image sequence information taken by the camera within the first time period and the posture information of the drone within the first time period; the processing unit 530 is also used to use the visual simultaneous localization and mapping (SLAM) algorithm to process the image sequence information and the posture information of the drone within the first time period to obtain the camera's posture trajectory.

[0093] In one possible implementation, the first plane is the plane where the x-axis and y-axis in the drone coordinate system are located, and the second plane where the posture trajectory is located is parallel to the first plane.

[0094] Alternatively, if the device 500 is located in the drone 100, the processing unit 530 may be Figure 1 The calibration module 112 is shown.

[0095] Figure 6 It is a schematic diagram of another calibration device 600 provided in an embodiment of the present application.

[0096] The device 600 includes a memory 610, a processor 620, and a communication interface 630. The memory 610, processor 620, and communication interface 630 are connected via an internal connection path. The memory 610 is used to store instructions, and the processor 620 is used to execute the instructions stored in the memory 610 to control the communication interface 630 to obtain information, so that the device 600 implements the aforementioned calibration method. Optionally, the memory 610 can be coupled to the processor 620 via an interface or integrated with the processor 620.

[0097] It should be noted that the communication interface 630 may be a transceiver, for example but not limited to a transceiver. The communication interface 630 may also include an input / output interface.

[0098] The processor 620 stores one or more computer programs, which include instructions. When the instructions are executed by the processor 620, the apparatus 600 executes the calibration method in each of the above embodiments.

[0099] It should be understood that in the embodiments of the present application, the processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0100] It should also be understood that in the embodiments of the present application, the memory may include read-only memory and random access memory, and provide instructions and data to the processor. A portion of the processor may also include non-volatile random access memory. For example, the processor may also store device type information.

[0101] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor 620 or by instructions in the form of software. The method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 610, and the processor 620 reads the information in the memory 610 and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0102] Optionally, Figure 6 The communication interface 630 can be implemented Figure 5 The acquisition unit 510 in Figure 6 The memory 610 in the embodiment can be implemented Figure 5 The storage unit 520, Figure 6 The processor 620 in the embodiment can implement Figure 5 The processing unit 530 in .

[0103] Optionally, the device 500 or the device 600 may be located at Figure 1 Among the drones.

[0104] The embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program code, and when the computer program code is run on a computer, the computer executes the above Figure 2 or Figure 3 Any of the methods in .

[0105] The present application also provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above Figure 2 or Figure 3 Any of the methods in .

[0106] The embodiment of the present application also provides a chip, including: a circuit, which is used to execute the above Figure 2 or Figure 3 Any of the methods in .

[0107] The present application also provides a smart device, including: Figure 5 or Figure 6 Any of the calibration devices shown.

[0108] Optionally, the above-mentioned intelligent device is an agricultural drone or vehicle.

[0109] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0110] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0111] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0112] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0113] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0114] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0115] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A calibration method, characterized in that: The method comprises: Obtaining a first three-dimensional coordinate system of a camera and a pose trajectory of the camera within a first time period, wherein the camera is deployed on a smart device; Projecting the posture trajectory onto a first plane to obtain a plurality of projection points, where the first plane is a plane where the x-axis and the y-axis are located in the coordinate system of the smart device, and a second plane where the posture trajectory is located is parallel to the first plane; Fitting the multiple projection points on the first plane to obtain a first straight line; determining a second three-dimensional coordinate system of the camera according to the first straight line; Determining an extrinsic parameter of the camera relative to the smart device according to the first three-dimensional coordinate system and the second three-dimensional coordinate system; Determining a second three-dimensional coordinate system of the camera according to the first straight line includes: Taking the first straight line as the x-axis of the second three-dimensional coordinate system; A second straight line perpendicular to the first straight line is used as the y-axis of the second three-dimensional coordinate system, and the second straight line is located on the first plane; A third straight line perpendicular to the first plane is used as the z-axis of the second three-dimensional coordinate system, wherein the third straight line intersects the first straight line and the second straight line; The determining, based on the first three-dimensional coordinate system and the second three-dimensional coordinate system, external parameters of the camera relative to the smart device includes: A rotation matrix of the camera relative to the smart device is determined according to a relative positional relationship between coordinate axes of the first three-dimensional coordinate system and coordinate axes of the second three-dimensional coordinate system.

2. The method according to claim 1, wherein The obtaining of the position trajectory of the camera within the first time period includes: Acquire image sequence information captured by a camera within the first time period and position information of the smart device within the first time period; The image sequence information and the position information of the smart device within the first time period are processed using a visual simultaneous localization and mapping (SLAM) algorithm to obtain a position trajectory of the camera.

3. A calibration device, characterized in that: The device comprises: an acquiring unit, configured to acquire a first three-dimensional coordinate system of a camera and a pose trajectory of the camera within a first time period, wherein the camera is deployed on a smart device; A processing unit for: Projecting the posture trajectory onto a first plane to obtain a plurality of projection points, where the first plane is a plane where the x-axis and the y-axis are located in the coordinate system of the smart device, and a second plane where the posture trajectory is located is parallel to the first plane; Fitting the multiple projection points on the first plane to obtain a first straight line; determining a second three-dimensional coordinate system of the camera according to the first straight line; Determining an extrinsic parameter of the camera relative to the smart device according to the first three-dimensional coordinate system and the second three-dimensional coordinate system; The processing unit is specifically configured to: Taking the first straight line as the x-axis of the second three-dimensional coordinate system; A second straight line perpendicular to the first straight line is used as the y-axis of the second three-dimensional coordinate system, and the second straight line is located on the first plane; A third straight line perpendicular to the first plane is used as the z-axis of the second three-dimensional coordinate system, wherein the third straight line intersects the first straight line and the second straight line; The processing unit is specifically configured to determine a rotation matrix of the camera relative to the smart device based on a relative positional relationship between coordinate axes of the first three-dimensional coordinate system and coordinate axes of the second three-dimensional coordinate system.

4. The device according to claim 3, characterized in that The acquisition unit is further configured to acquire image sequence information captured by the camera within the first time period and position information of the smart device within the first time period; The processing unit is further configured to process the image sequence information and the posture information of the smart device within the first time period using a visual simultaneous localization and mapping (SLAM) algorithm to obtain a posture trajectory of the camera.

5. A calibration device, characterized in that: include: The method comprises a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store computer programs or instructions, and the processor is used to execute the computer program or instructions in the memory, so that the method according to claim 1 or 2 is performed.

6. A smart device, characterized in that: The smart device is a drone, and the drone includes: a calibration device according to any one of claims 3 to 5.

7. A smart device, characterized in that: The smart device is a vehicle, and the vehicle includes: a calibration device according to any one of claims 3 to 5.

Images