Multi-lens calibration device and system

Abstract

The utility model provides a multi-lens calibration device and system, comprising a fixing structure, a calibration structure and a control terminal, the control terminal is respectively in communication connection with the fixing structure and the calibration structure; the fixing structure is used for fixing the unmanned aerial vehicle and establishing a communication link between the unmanned aerial vehicle and the control terminal, and the unmanned aerial vehicle carries a multi-lens; the calibration structure is deployed at a specified position, so that the calibration structure is located in an acquisition range of a specified lens in the multi-lens lens; and the control terminal is used for sending a control instruction to the calibration structure and the fixing structure, triggering the calibration structure to execute a posture adjustment action, and triggering the specified lens to execute an image acquisition action on the calibration structure. The multi-lens calibration device can be suitable for a calibration scene in which the unmanned aerial vehicle carries the multi-lens, and the calibration effect of the multi-lens is remarkably improved.

Description

Technical Field

[0001] The utility model relates to the technical field of camera calibration, in particular to a multi-eye lens calibration device and system. Background Art

[0002] In the image measurement process and machine vision applications, in order to determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image, a geometric model of camera imaging must be established. The process of solving these geometric model parameters is camera calibration. Whether in image measurement or machine vision applications, camera calibration is a very critical link. The calibration accuracy and algorithm stability directly affect the results of subsequent camera work.

[0003] During the drone production process, the lens assembly process of each drone is different, so the drone lens needs to be calibrated. However, the calibration structure provided by the relevant technology cannot be applied to the calibration scenario where the drone is equipped with multiple (such as four) lenses. Utility Model Content

[0004] In view of this, the purpose of the utility model is to provide a multi-eye lens calibration device and system, which can be applicable to the calibration scenario of a drone equipped with multiple eyes, and significantly improve the calibration effect of the multiple eyes.

[0005] In a first aspect, an embodiment of the utility model provides a multi-eye lens calibration device, comprising: a fixed structure, a calibration structure and a control terminal, wherein the control terminal is respectively connected to the fixed structure and the calibration structure in communication;

[0006] The fixing structure is used to fix the drone and establish a communication link between the drone and the control terminal, and the drone is equipped with multiple cameras;

[0007] The calibration structure is deployed at a specified position so that the calibration structure is located within the acquisition range of a specified lens in the multi-eye lens;

[0008] The control terminal is used to send control instructions to the calibration structure and the fixed structure, trigger the calibration structure to perform a posture adjustment action, and trigger the designated lens to perform an image acquisition action for the calibration structure.

[0009] In one implementation, the relative position relationship between the calibration structure and the drone is adjusted so that the calibration structure is located within the acquisition range of another designated lens in the multi-eye lens.

[0010] In one embodiment, the fixing structure includes a support frame and a holder tool installed on the top of the support frame, and the drone is fixed to the support frame via the holder tool.

[0011] In one embodiment, the drone is provided with a first communication component, and the card holder tooling is provided with a second communication component matching the first communication component;

[0012] The first communication component is inserted into the second communication component, or the second communication component is inserted into the first communication component to establish a communication link between the drone and the control terminal.

[0013] In one embodiment, it also includes a first network port transfer cable, and the card holder tooling is also provided with a third communication component;

[0014] One end of the first network port transfer cable is connected to the third communication component, and the other end of the first network port transfer cable is connected to the control terminal.

[0015] In one embodiment, the calibration structure includes a mechanical arm, and a calibration plate fixed at a first end of the mechanical arm.

[0016] In one embodiment, it also includes a second Ethernet port cable, one end of the second Ethernet port cable is connected to the robotic arm, and the other end of the second Ethernet port cable is connected to the control terminal.

[0017] In one embodiment, a power supply component adapted to the drone is also included, and the power supply component is used to provide electrical energy to the drone.

[0018] In one embodiment, the multi-eye lens carried by the drone is a four-eye fisheye lens.

[0019] In a second aspect, an embodiment of the utility model further provides a multi-eye lens calibration system, comprising any multi-eye lens calibration device provided in the first aspect, and a drone fixed on the multi-eye lens calibration device, wherein the multi-eye lens carried by the drone is a lens of at least four eyes.

[0020] A multi-eye lens calibration device and system provided by an embodiment of the utility model include: a fixed structure, a calibration structure and a control terminal, wherein the control terminal is respectively communicatively connected with the fixed structure and the calibration structure; the fixed structure is used to fix an unmanned aerial vehicle (UAV) and establish a communication link between the UAV and the control terminal, wherein the UAV is equipped with a multi-eye lens; the calibration structure is deployed at a designated position so that the calibration structure is located within the acquisition range of a designated lens in the multi-eye lens; the control terminal is used to send control instructions to the calibration structure and the fixed structure, trigger the calibration structure to perform a posture adjustment action, and trigger the designated lens to perform an image acquisition action for the calibration structure. The control terminal in the above-mentioned device is respectively communicatively connected to the fixed structure and the calibration structure, and a communication link is established between the control terminal and the drone using the fixed structure. When the calibration structure is within the acquisition range of the designated lens, the control terminal will respectively trigger the designated lens to perform image acquisition actions for the calibration structure, and trigger the calibration structure to perform attitude adjustment actions. After the designated lens completes the image acquisition action, the next lens can be used as the designated lens and the above-mentioned operation can be repeated until all lenses carried by the drone complete the image acquisition action. The lens calibration can be completed using the collected images. The device provided by the embodiment of the utility model can be applicable to calibration scenarios in which drones are equipped with multiple lenses, significantly improving the calibration effect of the multiple lenses.

[0021] Other features and advantages of the utility model will be described in the following description, and partly become apparent from the description, or understood by practicing the utility model. The purpose and other advantages of the utility model are realized and obtained by the structures specifically pointed out in the description, claims and drawings.

[0022] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the specific implementation methods of the utility model or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the utility model. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A schematic diagram of the structure of a multi-lens calibration device provided by an embodiment of the utility model;

[0025] Figure 2 A schematic plan view of a multi-lens calibration device provided in an embodiment of the utility model;

[0026] Figure 3 A schematic diagram of the specific structure of a multi-lens calibration device provided by an embodiment of the utility model;

[0027] Figure 4 A schematic diagram of the structure of a multi-eye lens calibration system provided in an embodiment of the utility model.

[0028] Icons: 100-multi-lens calibration device; 200-UAV; 1-fixed structure; 2-calibration structure; 3-control terminal; 4-tripod; 5-card holder tooling; 6-pogoping to network port cable; 7-robotic arm; 8-QR code recognition block; 9-power supply component. DETAILED DESCRIPTION

[0029] The technical solution of the utility model will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the utility model, not all of the embodiments. Based on the embodiments of the utility model, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the utility model.

[0030] At present, the calibration structure provided by the relevant technology cannot be applied to the calibration scenario of the drone equipped with multiple lenses. Based on this, the utility model implements a multi-lens calibration device and system, which can be applied to the calibration scenario of the drone equipped with multiple lenses, and significantly improve the calibration effect of the multi-lens.

[0031] To facilitate understanding of this embodiment, firstly, a multi-lens calibration device disclosed in the embodiment of the utility model is described in detail, see Figure 1 The schematic diagram of the structure of a multi-eye lens calibration device is shown, the device comprises: a fixed structure 1, a calibration structure 2 and a control terminal 3, the control terminal 3 is respectively connected to the fixed structure 1 and the calibration structure 2 for communication. Optionally, the control terminal 3 and the fixed structure 1 and the calibration structure 2 can be connected by wire or wirelessly.

[0032] In one example, the fixed structure 1 is used to fix the drone and establish a communication link between the drone and the control terminal 3, and the drone is equipped with multiple cameras. In one embodiment, the drone and the fixed structure 1 are provided with matching communication components. When the drone is fixed on the fixed structure 1, the two communication components are successfully connected, and the control command sent by the control terminal 3 will be transmitted to the drone via the fixed structure 1, or the image collected by the drone will be transmitted to the control terminal 3 via the fixed structure 1.

[0033] In one example, the calibration structure 2 is deployed at a specified position so that the calibration structure 2 is located within the acquisition range of a specified lens in the multi-eye lens. The specified lens can be understood as the lens for the current image acquisition action to be performed, and each lens carried by the drone will be used as a specified lens in turn. In one embodiment, the calibration structure 2 includes a calibration plate, and the height difference between the calibration plate and the specified lens is less than a preset threshold, and the calibration plate and the specified lens are at a preset distance, so that the calibration plate is located within the acquisition range of the specified lens.

[0034] In one example, the control terminal 3 is used to send control instructions to the calibration structure 2 and the fixed structure 1, triggering the calibration structure 2 to perform a posture adjustment action, and triggering the designated lens to perform an image acquisition action for the calibration structure 2. In one embodiment, the control terminal 3 sends a first control instruction to the fixed structure 1, and the first control instruction is transmitted to the drone via the fixed structure 1, and the drone controls the designated lens to perform an image acquisition action for the calibration structure; when the image acquisition action is completed, the control terminal 3 sends a second control instruction to the calibration structure 2 to trigger the calibration structure 2 to perform a posture adjustment action; after the posture adjustment action is completed, the control terminal 3 continues to send the first control instruction to the fixed structure 1, repeating this process until the designated lens captures a preset number of images; adjust the relative position relationship between the calibration structure 2 and the fixed structure 1 so that the calibration structure 2 is located within the acquisition range of another designated lens in the multi-eye lens, and continue to repeat the above operations, so that each lens carried by the drone captures a preset number of images, and the multi-eye lens of the drone is calibrated using the images.

[0035] The embodiment of the utility model also provides an implementation method for adjusting the relative position relationship between the calibration structure 2 and the fixed structure 1. The position of the fixed structure 1 can be kept unchanged, and the position of the calibration structure 2 can be adjusted to change the relative position relationship between the calibration structure 2 and the fixed structure 1; or the position of the calibration structure 2 can be kept unchanged, and the position of the fixed structure 1 can be adjusted to adjust the relative position relationship between the calibration structure 2 and the fixed structure 1. Considering that the position of the moving fixed structure 1 may cause the parameters that affect the calibration effect in the drone to change, thereby having a negative impact on the calibration effect, the embodiment of the utility model preferably keeps the position of the fixed structure 1 unchanged and adjusts the position of the calibration structure 2.

[0036] The embodiment of the utility model also provides another implementation method for adjusting the relative position relationship between the calibration structure 2 and the fixed structure 1. For example, the relative position relationship between the calibration structure 2 and the fixed structure 1 can be changed manually; for another example, the calibration structure 2 can be moved by a robot to change the relative position relationship between the calibration structure 2 and the fixed structure 1. Specifically: the control terminal 3 is connected to the robot for communication, and the control terminal 3 sends instructions to the robot. The robot will perform actions such as lifting the calibration structure 2, moving along a preset path, and placing the calibration structure 2 according to a predetermined program; for another example, a track can be deployed in the calibration scene, and a structure matching the track is provided at the bottom of the calibration structure 2. The control terminal 3 sends instructions to the calibration structure 2 so that the calibration structure 2 moves along the track to the next designated position.

[0037] The control terminal in the multi-eye lens calibration device provided by the embodiment of the utility model is respectively communicated with the fixed structure and the calibration structure, and a communication link between the control terminal and the drone is established by using the fixed structure. When the calibration structure is within the acquisition range of the designated lens, the control terminal will respectively trigger the designated lens to perform an image acquisition action for the calibration structure, and trigger the calibration structure to perform a posture adjustment action. After the designated lens completes the image acquisition action, the next lens can be used as the designated lens and the above operation can be repeated until all lenses carried by the drone complete the image acquisition action. The lens calibration can be completed using the collected images. The device provided by the embodiment of the utility model can be applicable to calibration scenarios in which drones are equipped with multi-eye lenses, and the calibration effect of the multi-eye lenses can be significantly improved.

[0038] For ease of understanding, the present utility model embodiment takes a drone equipped with a four-eye fisheye lens as an example, and the present utility model embodiment provides the following Figure 2 A schematic plan view of a multi-lens calibration device is shown in FIG. Figure 2 It shows that four fisheye lenses are deployed in front of, behind, on the left and on the right of the drone.

[0039] The four-eye fisheye lens, with high-performance resolution, can perform all-round intelligent perception of unknown environments and enable safe flight and exploration of the unknown with panoramic images. It can accurately and timely avoid obstacles without blind spots, and adjust the flight course and altitude in real time according to the distance and direction of obstacles. For example, it can easily pass through complex areas such as jungles, mountains, and cities. It also has a built-in end-side AI module to complete standard collection and intelligent analysis of targets, making picture shooting more accurate, data collection more standardized, and report analysis more intelligent. A fisheye camera is a wide-angle camera with a large field of view that can capture large areas of scenes in a small space. However, due to the special structure of the fisheye lens, the distortion it produces is relatively serious and requires calibration for accurate image processing and measurement.

[0040] Based on this, the embodiment of the utility model also provides a specific structure of a multi-lens calibration device, see Figure 3 A schematic diagram of the specific structure of a multi-eye lens calibration device is shown.

[0041] like Figure 3 As shown, the fixing structure 1 includes a support frame, which can be a tripod 4, and a clamping fixture 5 installed on the top of the tripod 4, and the drone is fixed on the tripod 4 via the clamping fixture 5. Optionally, the clamping fixture 5 can be fixed on the top of the support frame by a variety of methods such as nuts, metal hinges, and a clamping structure.

[0042] Furthermore, the drone is provided with a first communication component, and the card holder tooling 5 is provided with a second communication component matching the first communication component; the first communication component is inserted into the second communication component, or the second communication component is inserted into the first communication component, so that the drone is fixed on the card holder tooling 5, and a communication link between the drone and the control terminal 3 can be established. For example, the first communication component is a plug (such as a USB plug or a Type-C plug), and the second communication component is an interface (such as a USB interface or a Type-C structure). By inserting the plug provided on the drone into the interface provided on the card holder tooling 5, the drone can be fixed and a communication link between the drone and the control terminal 3 can be established. For another example, the first communication component is an interface, and the second communication component is a plug. The specific setting can be based on actual needs, and the embodiment of the utility model does not limit this.

[0043] Furthermore, the multi-eye lens calibration device also includes a first network port cable, the first network port cable 6 can adopt a pogoping network port cable 6, the card holder tooling 5 is also provided with a third communication component, the third communication component can adopt a pogoping interface board, one end of the pogoping network port cable 6 is connected to the pogoping interface board, and the other end of the pogoping network port cable 6 is connected to the control terminal 3 to realize the communication connection between the card holder tooling 5 and the control terminal 3.

[0044] Please continue to see Figure 3 The calibration structure includes a mechanical arm 7 and a calibration plate fixed at the first end of the mechanical arm 7. The calibration plate can adopt a two-dimensional code recognition block 8.

[0045] Furthermore, the multi-eye lens calibration device also includes a second network port transfer cable, one end of the second network port transfer cable is connected to the mechanical arm, and the other end of the second network port transfer cable is connected to the control terminal. Optionally, the second network port transfer cable can also be a pogoping network port transfer cable, and a pogoping interface board can be provided on the mechanical arm to achieve communication connection between the mechanical arm 7 and the control terminal 3.

[0046] Please continue to see Figure 2, Figure 2 It is shown that the multi-eye lens calibration device also includes a power supply component 9 adapted to the drone, and the power supply component 9 is used to provide power to the drone.

[0047] In summary, the main components involved in the multi-eye lens calibration device provided by the embodiment of the utility model are shown in Table 1 below:

[0048] Table 1

[0049]

[0050]

[0051] Based on the multi-lens calibration structure provided in the above-mentioned embodiment, the embodiment of the utility model continues to take the drone equipped with a four-eye fisheye lens as an example, and provides a calibration process of a four-eye fisheye lens, including:

[0052] (1) unfolding the tripod 4;

[0053] (2) A card holder tooling 5 is installed on the top of the tripod 4, and the card holder tooling 5 is connected to the control terminal 3;

[0054] (3) Install the robotic arm 7 and connect to the control terminal 3. The fisheye camera on the front side of the drone is positioned directly opposite the zero-point center of the QR code recognition block 8 fixed at the end of the robotic arm 7 and the height is adjusted to be parallel. The distance between the QR code recognition block 8 and the drone is 50 cm;

[0055] (4) The Pogoping to network port cable 6 is connected to the control terminal 3 and installed to the card holder tooling 5 on the tripod 4;

[0056] (5) The UAV is loaded with the power supply component 9 and powered on;

[0057] (6) Inserting the drone into the card holder tool 5;

[0058] (7) Click on the control terminal 3 to confirm the start of the image acquisition program;

[0059] (8) The control terminal 3 determines whether the power supply assembly 9 is locked, otherwise an error call is reported, and if yes, the next step is entered;

[0060] (9) Notifying the control terminal 3 that the drone is ready to take photos;

[0061] (10) The control terminal 3 sends a photo-taking command to the drone, and the four fisheye cameras execute the photo-taking action (i.e., the aforementioned image acquisition action). The drone notifies the control terminal 3 when the photo-taking is completed;

[0062] (11) The control terminal 3 sends a photo transmission command to the drone, and at the same time sends a next photo posture command to the robotic arm 7. When the four photos of the drone are transmitted back to the computer, the angle of the drone and the angle of the QR code recognition block 8 are adjusted;

[0063] (12) After the drone and the robotic arm 7 have completed their actions, the control terminal 3 is notified;

[0064] (13) Control terminal 3 to repeatedly execute instructions (5) to (7) until all 600 photos are taken;

[0065] (14) The control terminal 3 uses the photo to calibrate the four-eye fisheye image carried by the drone. The calibration process involves executing the parameter transmission program, detecting whether there is an abnormality in the panoramic image and outputting the result, and writing the configuration file. The configuration file includes: device number, device sn, device vpn account, and device Qianxun account;

[0066] (15) The drone pulls out the card holder tooling 5;

[0067] (16) The drone unloads the power supply component 9 and cuts off the power.

[0068] In summary, the results of fisheye camera calibration are very important for image processing and computer vision applications. It can improve the accuracy and stability of the image, thereby achieving more sophisticated and efficient image processing and measurement. The traditional calibration procedure is complicated and takes 30 minutes. The embodiment of the utility model optimizes the aircraft four-eye fisheye calibration process, saves calibration time, formulates an automated calibration process, takes 10 minutes, avoids the calibration error problem caused by the traditional calibration method, facilitates the operation of the aircraft production line, and reduces the line stop problem caused by calibration errors.

[0069] Based on the above embodiments, the present invention provides a multi-lens calibration system. Figure 4 A structural schematic diagram of a multi-objective calibration system is shown, which includes the multi-eye lens calibration device 100 provided in the aforementioned embodiment, and a drone 200 fixed on the multi-eye lens calibration device 100, and the multi-eye lens carried by the drone 200 is a lens with at least four eyes.

[0070] The multi-eye lens calibration system provided by the embodiment of the utility model includes the multi-eye lens calibration device provided by the aforementioned embodiment. The control terminal in the device is respectively connected to the fixed structure and the calibration structure for communication, and uses the fixed structure to establish a communication link between the control terminal and the drone. When the calibration structure is located within the acquisition range of the designated lens, the control terminal will respectively trigger the designated lens to perform an image acquisition action for the calibration structure, and trigger the calibration structure to perform a posture adjustment action. After the designated lens completes the image acquisition action, the next lens can be used as the designated lens and the above operation can be repeated until all lenses carried by the drone complete the image acquisition action, and the lens calibration can be completed using the collected images. The device provided by the embodiment of the utility model can be applicable to calibration scenarios in which drones are equipped with multi-eye lenses, and significantly improves the calibration effect of the multi-eye lenses.

[0071] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the multi-eye lens calibration system described above can refer to the corresponding process in the aforementioned embodiment, and will not be repeated here.

[0072] In the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0073] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0074] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the utility model, which are used to illustrate the technical solution of the utility model, rather than to limit it. The protection scope of the utility model is not limited thereto. Although the utility model is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solution recorded in the above-mentioned embodiments within the technical scope disclosed by the utility model, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solution of the embodiment of the utility model, and should be included in the protection scope of the utility model. Therefore, the protection scope of the utility model should be based on the protection scope of the claims.

Claims

1. A multi-lens calibration device, characterized in that: include: A fixed structure, a calibration structure and a control terminal, wherein the control terminal is communicatively connected with the fixed structure and the calibration structure respectively; The fixing structure is used to fix the drone and establish a communication link between the drone and the control terminal, and the drone is equipped with multiple cameras; The calibration structure is deployed at a specified position so that the calibration structure is located within the acquisition range of a specified lens in the multi-eye lens; The control terminal is used to send control instructions to the calibration structure and the fixed structure, trigger the calibration structure to perform a posture adjustment action, and trigger the designated lens to perform an image acquisition action for the calibration structure.

2. The multi-lens calibration device according to claim 1, characterized in that: The relative position relationship between the calibration structure and the drone is adjusted so that the calibration structure is located within the acquisition range of another designated lens in the multi-eye lens.

3. The multi-lens calibration device according to claim 1, characterized in that: The fixing structure includes a support frame and a holder tool installed on the top of the support frame, and the UAV is fixed on the support frame via the holder tool.

4. The multi-lens calibration device according to claim 3, characterized in that: The drone is provided with a first communication component, and the card holder tooling is provided with a second communication component matching the first communication component; The first communication component is inserted into the second communication component, or the second communication component is inserted into the first communication component to establish a communication link between the drone and the control terminal.

5. The multi-lens calibration device according to claim 3, characterized in that: It also includes a first network port transfer cable, and the card holder tooling is also provided with a third communication component; One end of the first network port transfer cable is connected to the third communication component, and the other end of the first network port transfer cable is connected to the control terminal.

6. The multi-lens calibration device according to claim 1, characterized in that: The calibration structure includes a mechanical arm and a calibration plate fixed at a first end of the mechanical arm.

7. The multi-lens calibration device according to claim 6, characterized in that: It also includes a second network port cable, one end of which is connected to the robotic arm, and the other end of which is connected to the control terminal.

8. The multi-lens calibration device according to claim 1, characterized in that: It also includes a power supply component adapted to the drone, and the power supply component is used to provide electrical energy to the drone.

9. The multi-lens calibration device according to claim 1, characterized in that: The multi-eye lens carried by the drone is a four-eye fisheye lens.

10. A multi-lens calibration system, characterized in that: The invention comprises the multi-eye lens calibration device as described in any one of claims 1 to 9, and a drone fixed on the multi-eye lens calibration device, wherein the multi-eye lens carried by the drone is a lens of at least four eyes.

Images