Mobile calibration device and method suitable for large-scale scene monitoring network

Abstract

The invention relates to the technical field of camera calibration, in particular to a mobile calibration device and method suitable for a large-scene monitoring network, the device comprises a monitoring camera end and a mobile platform end, the monitoring camera end comprises a monitoring camera and a monitoring server, the monitoring server is used for controlling and processing image data of the monitoring camera, and the mobile platform end is used for receiving the image data of the monitoring camera. The mobile platform end comprises a mobile chassis, a calibration plate, an LED synchronous signal lamp, a sensor module and a fixed support, the fixed support is fixedly arranged on the mobile chassis, the calibration plate surrounds the periphery of the fixed support, the LED synchronous signal lamp is fixedly arranged on the fixed support, and the sensor module is fixedly arranged at the top of the fixed support and is in a relatively static state; the mobile platform end drives the calibration plate to walk in the whole field range, so that the monitoring camera end collects images on the calibration plate for internal reference calibration, meanwhile, the LED synchronous signal lamp and the sensor module are used for resolving external parameters of the monitoring camera in a scene, productivity is liberated, and calibration efficiency and calibration precision are improved at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a mobile calibration device and method suitable for a large-scene monitoring network. Background Art

[0002] In the field of camera calibration, accurate calibration of camera parameters is crucial for subsequent image processing, target tracking, 3D reconstruction and other tasks. However, traditional camera calibration methods generally rely on manually holding a calibration plate and changing the position of the calibration plate within the camera's field of view to collect multiple images, and then complete the calibration of camera parameters through these image data.

[0003] Specifically, the traditional calibration method requires the operator to accurately control the posture of the calibration plate to ensure that the feature points on the calibration plate (such as corner points, circular marks, etc.) are always completely and clearly within the field of view of the camera. In large-scene applications, due to the large size of the scene and the wide field of view, the operator needs to constantly move the calibration plate and adjust its posture in order to capture enough feature point information at different viewing angles and positions of the camera. This process is not only time-consuming and labor-intensive, but also easily affected by human factors, such as the operator's hand stability, the speed of the calibration plate movement, etc., which may greatly affect the accuracy and reliability of the calibration results.

[0004] In addition, with the rapid development of computer vision technology, the demand for camera calibration in large scenes is increasing, and traditional calibration methods can no longer meet the needs of efficient and accurate calibration in practical applications. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the present application provides a mobile calibration device and calibration method suitable for large-scale monitoring networks, which liberates productivity while improving calibration efficiency and calibration accuracy.

[0006] The technical solution adopted by the present invention to solve its technical problems is: a mobile calibration device suitable for a large-scene monitoring network, comprising: a monitoring camera end and a mobile platform end, the monitoring camera end comprising a monitoring camera and a monitoring server, the monitoring server being used to control and process monitoring camera image data, the mobile platform end comprising a mobile chassis, a calibration board, an LED synchronization signal light, a sensor module and a fixed bracket, the fixed bracket being fixedly arranged on the mobile chassis, the calibration board being arranged around the fixed bracket, the LED synchronization signal light being fixedly arranged on the fixed bracket, the sensor module being fixedly arranged on the top of the fixed bracket and being in a relatively static state; the mobile platform end drives the calibration board to move around the entire site, so that the monitoring camera end collects the image on the calibration board for internal parameter calibration, and at the same time uses the LED synchronization signal light and the sensor module to solve the external parameters of the monitoring camera under the scene.

[0007] The fixed bracket in the above technical solution includes a bottom plate, a support column, a bracket and a top plate, and the bottom plate, the bracket and the top plate are combined into a rigid body through the support column.

[0008] In the above technical solution, the bottom plate and the top plate are provided with through holes for screws to pass through, and the fixed bracket is connected to the mobile chassis through the bottom plate by screws; the fixed bracket is connected to the sensor module through the top plate by screws; the bracket is arranged around the support column for connecting the calibration plate.

[0009] The calibration plate in the above technical solution is in the shape of an isosceles trapezoid, and together with the fixed bracket forms a pyramid-shaped three-dimensional calibration device.

[0010] In the above technical solution, the calibration plate is provided with a wide black border and a square mark; the square mark is used to determine the identifier of the black border.

[0011] The identifier of the black border in the above technical solution is composed of a binary matrix.

[0012] The working state of the LED synchronous signal light in the above technical solution is controlled by the sensor module.

[0013] The sensor module in the above technical solution is composed of at least one camera or radar sensor and at least one computer device, which is used to run the SLAM algorithm.

[0014] In the above technical solution, the positional relationship between the calibration plate and the sensor and computing unit module is determined by the fixing bracket and the installation method, and the positional relationship between the two remains unchanged after installation.

[0015] The technical solution adopted by the present invention to solve its technical problem is: a mobile calibration method suitable for a large-scene monitoring network, applied to the mobile calibration device as described above, including the mobile platform end using the sensor module to run the SLAM algorithm before starting in the scene, and outputting the posture relationship sensor_T_origin between the mobile platform end (the sensor module is used as the coordinate origin of the mobile platform end) and the starting position in real time, which is composed of a rotation matrix and a translation vector.

[0016] The rotation matrix is ​​in the form:

[0017]

[0018] The translation vector is of the form:

[0019]

[0020] Taking the start position of the mobile terminal device as the scene origin, according to the external reference relationship tag_T_sensor between the calibration plate and the sensor module and the relationship sensor_T_origin between the sensor module and the scene origin, the real-time position and posture relationship tag_T_origin between the calibration plate and the scene origin is obtained;

[0021] The mobile platform moves in a calibration scene, and the monitoring camera collects image data of the monitoring camera in real time and detects whether there is a calibration plate therein. If it is detected that there is a calibration plate, the corresponding image data is saved for calibration of internal parameters.

[0022] The beneficial effects of the present invention are: in large scenes, there is no need to manually hold a calibration plate to collect data and calibrate each camera; the designed pyramid-shaped three-dimensional calibration module does not need to adjust the angle of the calibration object so that the calibration object is completely under the camera's field of view, and the proposed calibration method can accurately obtain the positional relationship between the characteristic points of the calibration object and the mobile device, thereby liberating productivity and improving calibration efficiency and calibration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A structural diagram of a mobile calibration device suitable for a large-scale monitoring network provided by an embodiment of the present invention;

[0024] Figure 2 A structural diagram of a fixing bracket provided in an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of a calibration plate provided in an embodiment of the present invention;

[0026] Figure 4 A flow chart of a mobile calibration method applicable to a large-scene monitoring network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0028] The following will clearly and completely describe the concept, specific structure and technical effects of the present invention in combination with the embodiments and drawings, so as to fully understand the purpose, characteristics and effects of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by technicians in this field without creative work are all within the scope of protection of the present invention. In addition, all the connection / connection relationships involved in the patent do not refer to the direct connection of components, but refer to the formation of a better connection structure by adding or reducing connection accessories according to the specific implementation situation. The various technical features in the invention can be combined interchangeably without conflicting with each other.

[0029] As mentioned above, existing camera calibration requires manual holding of the calibration plate and making the feature points of the calibration plate completely within the field of view of the camera. Multiple images are collected for camera calibration by continuously changing the position of the calibration plate, which is time-consuming and labor-intensive in large scenes.

[0030] Therefore, the present invention provides a mobile calibration device suitable for large-scale monitoring networks, such as Figure 1-2 As shown, it includes a monitoring camera end and a mobile platform end, the monitoring camera end includes a monitoring camera and a monitoring server, the monitoring server is used to control and process the monitoring camera image data, the mobile platform end includes a mobile chassis 1, a calibration plate 2, an LED synchronization signal light (not shown in the figure), a sensor module 3 and a fixed bracket 4, the fixed bracket 4 is fixedly arranged on the mobile chassis 1, the calibration plate 2 is arranged around the fixed bracket 4, the LED synchronization signal light is fixedly arranged on the fixed bracket 4, the sensor module 3 is fixedly arranged on the top of the fixed bracket 4, and is in a relatively static state; the mobile platform end drives the calibration plate 2 to move around the entire site, so that the monitoring camera end can collect the image on the calibration plate 2 for internal parameter calibration, and at the same time uses the LED synchronization signal light and the sensor module 3 to solve the external parameters of the monitoring camera under the scene.

[0031] Optionally, a control system is installed in the mobile chassis 1 and is wirelessly connected to the sensor module 3. The control system generates motion instructions based on the scene information obtained by the sensor module 3, thereby driving the mobile calibration device to move around the entire scene, so that the camera in the scene can collect images on the calibration plate for internal and external parameter calibration.

[0032] In a possible implementation, the sensor module 3 is composed of at least one camera or radar sensor and at least one computer device for running a SLAM algorithm.

[0033] SLAM (Simultaneous Localization and Mapping) algorithms obtain environmental perception information through perception sensors (such as cameras, lidar, etc.), align this information with the device's own position estimate, and achieve simultaneous positioning and mapping. Sensors used in SLAM algorithms include cameras, lidar, inertial measurement units (IMUs), and odometers.

[0034] Optionally, the sensor module 3 may include a monocular vision sensor for taking pictures at two preset positions in the calibration scene to obtain two sets of corresponding images; or a binocular vision sensor for taking pictures in the calibration scene to obtain two sets of corresponding images, and calculating the depth information of the structural objects appearing in the image based on the two sets of images, and then reconstructing the sparse point cloud contour of the calibration scene. Optionally, according to actual needs, the dense point cloud contour of the calibration scene can be further calculated based on the sparse point cloud contour.

[0035] In addition, the sensor module 3 may also be a structured light sensor (such as an RGB-D camera) for obtaining depth information of the calibration scene while photographing the calibration scene. Alternatively, it may be a combination of multiple sensors, such as a combination of a camera and a laser radar, or a combination of a camera, a laser radar and an inertial sensor (such as an inertial measurement unit IMU), etc., which is not specifically limited here.

[0036] In one possible way, Figure 2 As shown, the fixed bracket 4 includes a bottom plate 41 , a support column 42 , a bracket 43 and a top plate 44 , and the bottom plate 41 , the bracket 43 and the top plate 44 are combined into a rigid body through the support column 42 .

[0037] Among them, the bottom plate 41 and the top plate 44 are provided with through holes for screws to pass through, and the bottom plate 41 and the mobile chassis 1 are connected by screws; the top plate 44 and the sensor module 3 are connected by screws; the bracket 43 is arranged around the support column 42 for connecting the calibration plate 2.

[0038] In a possible implementation, the calibration plate 2 is in the shape of an isosceles trapezoid, and together with the fixing bracket 4 forms a pyramid-shaped three-dimensional calibration device.

[0039] Through the above embodiments, when the calibration plate provided by the present application is calibrated in a scene, due to the pyramid-shaped three-dimensional calibration device, the calibration plate is always in a three-dimensional state relative to the camera, and does not need to be manually held, thereby liberating productivity.

[0040] In a possible implementation, the positional relationship between the calibration plate and the sensor and computing unit module is determined by the installation method of the fixing bracket and the calibration plate, and the positional relationship between the two remains unchanged after installation.

[0041] In a possible implementation, the present application uses AruCo patterns of different sizes to calibrate the sensor module, and the AruCo pattern is a binary square reference mark for camera pose estimation, such as Figure 3 As shown, the calibration plate 2 is provided with a wide black border and a square mark; the square mark is used to determine the identifier of the black border.

[0042] The wide black border facilitates its rapid detection in the image, the binary code inside the border is used to identify the marker and provide error detection and correction, and optionally, the identifier of the black border consists of a binary matrix. The size of the ArUco marker determines the size of the internal matrix, for example, a marker of size 4x4 consists of a 16-bit binary matrix.

[0043] By performing camera calibration through the above embodiment, it is not necessary for all patterns to be within the camera's field of view, and each pattern has an independent ID mark. At the same time, the detection is simple and fast, and the pattern also has strong robustness, which is conducive to improving the calibration accuracy.

[0044] In a possible implementation, the working state of the LED synchronization signal light is controlled by the sensor module.

[0045] Specifically, the working state of the LED synchronous signal light includes color and flashing frequency.

[0046] In the mobile calibration device, the LED synchronization signal light plays a vital role. Its working state, including color and flashing frequency, is controlled by the sensor module. This design enables the LED synchronization signal light to accurately synchronize with the sensor module and other system components, thereby ensuring the accuracy and reliability of the calibration process.

[0047] Specifically, the LED synchronization signal light can transmit different synchronization information by changing color and flashing frequency. For example, when the mobile platform starts to move, the LED synchronization signal light can flash a specific color or frequency to notify the monitoring camera to collect images. When the mobile platform reaches a preset position or completes an action, the LED synchronization signal light can change color or stop flashing to mark the completion of the position or action.

[0048] In addition, the LED synchronization signal light can also be used to detect the communication status between the mobile platform and the surveillance camera. If there is a problem with the communication between the two, the LED synchronization signal light can send out an alarm signal so that timely measures can be taken to solve the problem.

[0049] like Figure 4 As shown, the present invention provides a mobile calibration method applicable to a large-scale monitoring network, which is applied to the mobile calibration device as described above, and includes the following steps:

[0050] Step 401, the mobile platform uses the sensor module to run the SLAM algorithm before starting in the scene, and outputs the pose relationship sensor_T_origin between the mobile platform (the sensor module is used as the coordinate origin of the mobile platform) and the starting position in real time, which is composed of a rotation matrix and a translation vector.

[0051] The rotation matrix is ​​in the form:

[0052]

[0053] The translation vector is of the form:

[0054]

[0055] Step 402, taking the start position of the mobile terminal device as the scene origin, according to the external reference relationship tag_T_sensor between the calibration plate and the sensor module and the relationship sensor_T_origin between the sensor module and the scene origin, the real-time position and posture relationship tag_T_origin between the calibration plate and the scene origin is obtained.

[0056] As mentioned above, since the positional relationship between the calibration plate and the sensor and computing unit module remains unchanged after installation, the external parameter relationship tag_T_sensor between the calibration plate and the sensor module can be obtained through simple calculation.

[0057] Then, according to formula (3), the real-time position and posture relationship tag_T_origin between the calibration plate and the scene origin can be obtained.

[0058]

[0059] Step 403, the mobile platform moves in the calibration scene, the monitoring camera collects the image data of the monitoring camera in real time, and detects whether there is a calibration plate therein, and if it is detected, saves the corresponding image data for calibration internal parameters.

[0060] The mobile platform moves in the calibration scene according to the preset path or the autonomously planned path. During the movement, the mobile platform maintains a certain speed and stability to ensure that the surveillance camera can capture clear and accurate image data.

[0061] The surveillance camera collects the image data of the surveillance camera in real time and processes and analyzes the image data using image processing algorithms. During this process, the surveillance camera detects whether there is a calibration plate in the image. The detection methods may include template matching, feature point detection, etc.

[0062] When the surveillance camera detects the presence of a calibration plate in the image, it will immediately save the corresponding image data. This image data will be used in the subsequent calibration internal reference process. At the same time, the surveillance camera will also record the timestamp and location information of the calibration plate to synchronize and match with the data on the mobile platform.

[0063] To ensure the accuracy of calibration, the surveillance camera may collect image data of the calibration plate multiple times at different positions, angles and postures. These image data will form a rich data set for subsequent calibration algorithm processing and optimization.

[0064] The mobile platform remains stationary for a certain time interval during the movement, and then the LED synchronization signal light expresses the synchronization identification information by changing the color and flashing frequency. At the same time, the mobile platform saves the current tag_T_origin information and the corresponding synchronization identification information. When the monitoring camera detects the synchronization identification information transmitted by the LED synchronization signal light, the monitoring camera saves the image data and the corresponding synchronization identification information, and then according to formula (4), uses the tag_T_origin information in the same synchronization identification information to obtain the monitor_T_origin position of the monitoring camera in the scene.

[0065]

[0066] Through the autonomous wandering of the mobile platform, combined with the real-time output of the posture relationship between the mobile platform and the starting position by the SLAM algorithm, manual intervention is greatly reduced and the calibration efficiency is improved. At the same time, the use of LED synchronization signal lights enables the monitoring camera to accurately capture the calibration plate image, further shortening the calibration time. The present invention utilizes the fixed position relationship between the sensor module and the calibration plate, as well as the precise posture information provided by the SLAM algorithm, to accurately calculate the real-time position and posture relationship between the calibration plate and the scene origin. This precise calculation method effectively avoids the errors caused by human factors in the traditional calibration method and improves the accuracy of calibration.

[0067] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A mobile calibration device suitable for large-scale monitoring networks, characterized in that: It includes a monitoring camera end and a mobile platform end. The monitoring camera end includes a monitoring camera and a monitoring server. The monitoring server is used to control and process the monitoring camera image data. The mobile platform end includes a mobile chassis, a calibration board, an LED synchronization signal light, a sensor module and a fixed bracket. The fixed bracket is fixedly arranged on the mobile chassis, the calibration board is arranged around the fixed bracket, the LED synchronization signal light is fixedly arranged on the fixed bracket, and the sensor module is fixedly arranged on the top of the fixed bracket and is in a relatively static state. The mobile platform end drives the calibration board to move around the entire site, so that the monitoring camera end can collect the image on the calibration board for internal parameter calibration, and at the same time uses the LED synchronization signal light and the sensor module to solve the external parameters of the monitoring camera under the scene.

2. The mobile calibration device applicable to a large-scale monitoring network according to claim 1, characterized in that: The fixed bracket comprises a bottom plate, a support column, a bracket and a top plate, and the bottom plate, the bracket and the top plate are combined into a rigid body through the support column.

3. The mobile calibration device applicable to a large-scale monitoring network according to claim 2, characterized in that: The bottom plate and the top plate are provided with through holes for screws to pass through, and the fixed bracket is connected to the mobile chassis through the bottom plate by screws; the fixed bracket is connected to the sensor module through the top plate by screws; the bracket is arranged around the support column for connecting the calibration plate.

4. The mobile calibration device applicable to a large-scale monitoring network according to claim 1, characterized in that: The calibration plate is in the shape of an isosceles trapezoid, and together with the fixing bracket forms a pyramid-shaped three-dimensional calibration device.

5. The mobile calibration device applicable to a large-scale monitoring network according to claim 4 is characterized in that: The calibration plate is provided with a wide black frame and a square mark; the square mark is used to determine the identifier of the black frame.

6. The mobile calibration device applicable to a large-scale monitoring network according to claim 5, characterized in that: The identifier of the black frame consists of a binary matrix.

7. The mobile calibration device applicable to a large-scale monitoring network according to claim 1, characterized in that: The working state of the LED synchronous signal light is controlled by the sensor module.

8. The mobile calibration device applicable to a large-scale monitoring network according to claim 1, characterized in that: The sensor module is composed of at least one camera or radar sensor and at least one computer device for running a SLAM algorithm.

9. The mobile calibration device applicable to a large-scale monitoring network according to claim 1, characterized in that: The positional relationship between the calibration plate and the sensor and computing unit module is determined by the fixing bracket and the installation method, and the positional relationship between the two remains unchanged after installation.

10. A mobile calibration method applicable to a large-scale monitoring network, applied to the mobile calibration device according to any one of claims 1 to 9, characterized in that: The method comprises: Before the mobile platform is started in the scene, the sensor module is used to run the SLAM algorithm, and the position relationship sensor_T_origin between the mobile platform (the sensor module is used as the coordinate origin of the mobile platform) and the start position is output in real time, which is composed of a rotation matrix and a translation vector. The rotation matrix is ​​in the form: The translation vector is of the form: Taking the start position of the mobile terminal device as the scene origin, according to the external reference relationship tag_T_sensor between the calibration plate and the sensor module and the relationship sensor_T_origin between the sensor module and the scene origin, the real-time position and posture relationship tag_T_origin between the calibration plate and the scene origin is obtained; The mobile platform moves in a calibration scene, and the monitoring camera collects image data of the monitoring camera in real time and detects whether there is a calibration plate therein. If it is detected that there is a calibration plate, the corresponding image data is saved for calibration of internal parameters.

Images