Computer room camera plane online calibration method, system, computer equipment and medium

Abstract

The present invention provides a method, system, computer equipment and medium for online calibration of a computer room camera plane. A preset number of cabinet binocular time-series images are obtained through a binocular camera, a corresponding cabinet depth map and a cabinet 3D point cloud of a camera coordinate system are extracted, and multiple two-dimensional cabinet key point coordinates of the cabinet binocular image are extracted through a preset object detection model. After obtaining the corresponding three-dimensional cabinet key point coordinates according to the two-dimensional cabinet key point coordinates and the cabinet depth map, the corresponding potential cabinet plane equations obtained by fitting the cabinet 3D point cloud of the camera coordinate system are combined to obtain a multi-plane equation group, and the multi-plane equation group is solved to obtain the camera relative to the ground plane extrinsic parameter matrix. The technical solution can realize online correction of camera parameters based on the prior information of the computer room without a calibration plate, which can not only avoid the modification of the on-site environment, but also ensure the efficiency and accuracy of online calibration, as well as the high robustness to adapt to environmental changes.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration of machine vision, and in particular to a method, system, computer equipment and storage medium for online plane calibration of a computer room camera based on cabinet information. Background Art

[0002] In the application of unattended visual positioning in the computer room, the calibration of camera parameters is a very critical link. The accuracy of the calibration results and the stability of the method directly affect the camera calculation accuracy, and ultimately affect the accuracy of visual positioning. Therefore, providing a convenient, accurate, and easy-to-maintain camera calibration method is an important link in the application of unattended visual positioning in the computer room.

[0003] Traditional camera calibration methods usually adopt an offline calibration method to calibrate the external parameters of the camera and patrol car once when the equipment leaves the factory. However, this method is mainly limited in two aspects in practical applications: 1) The ground at the application site is not absolutely flat. During the positioning and navigation process of the automatic patrol car in the computer room, the flatness of the ground causes the car to have positioning errors, and the positioning errors will gradually accumulate in long-term operation, forming a larger cumulative error, which will eventually lead to inaccurate positioning of the computer room patrol car or even failure; 2) The computer room patrol car operates 24 hours a day, and mechanical fatigue will cause the camera fixing mechanism to loosen. The camera posture is very easy to deviate from the factory calibration parameters, and regular maintenance of fixed equipment is required, which increases the application cost and reduces the flexibility of the solution. Although some scholars have proposed online camera calibration methods to address the defects of traditional offline calibration, the existing online calibration methods still have application defects. For example, the on-site environment needs to be modified during application, and when the ground information is obviously obscured, effective calibration results cannot be output. In addition, the characteristic calibration algorithm used is sensitive to conditions such as lighting, which significantly reduces the robustness of the online calibration.

[0004] Therefore, there is an urgent need to provide a camera plane online calibration method suitable for autonomous patrol of the computer room, which can meet the calibration accuracy requirements and operate robustly in different lighting environments of the computer room scene. Summary of the invention

[0005] The purpose of the present invention is to provide an online calibration method for a computer room camera plane based on cabinet information. Depth information and 3D point cloud information are obtained by performing binocular matching on a preset number of acquired binocular time-series images of cabinets, and after fitting the potential cabinet plane equation based on Hough space voting using the 3D point cloud information, a multi-plane equation group is obtained by jointly establishing the potential cabinet plane equations corresponding to the preset number of cabinet binocular time-series images, and combined with the cabinet key point information of the computer room cabinet image extracted using a preset object detection model, the environmental information is maximized, and the camera relative to the ground plane extrinsic matrix is ​​updated in real time, so as to realize efficient, accurate and highly robust online calibration of the camera plane.

[0006] In order to achieve the above objectives, it is necessary to provide a method, system, computer equipment and storage medium for online calibration of a camera plane in a computer room in response to the above technical problems.

[0007] In a first aspect, an embodiment of the present invention provides a method for online calibration of a plane of a camera in a computer room, the method comprising the following steps:

[0008] Obtain a preset number of cabinet binocular time-sequential images through a binocular camera; the cabinet binocular images include a left cabinet image and a right cabinet image;

[0009] According to the binocular images of each cabinet, the corresponding cabinet depth map and the cabinet 3D point cloud of the camera coordinate system are extracted;

[0010] By presetting the object detection model, according to the multiple two-dimensional cabinet key point coordinates of the binocular image of each cabinet, and according to the two-dimensional cabinet key point coordinates and the cabinet depth map, the corresponding three-dimensional cabinet key point coordinates are obtained;

[0011] According to the cabinet 3D point clouds of each camera coordinate system, the corresponding potential cabinet plane equation is fitted; the potential cabinet plane equation includes a potential left cabinet plane equation and a potential right cabinet plane equation;

[0012] All potential cabinet plane equations are combined to obtain a multi-plane equation group, and the camera relative to the ground plane extrinsic parameter matrix is ​​obtained by solving the multi-plane equation group.

[0013] Furthermore, the step of extracting the corresponding cabinet depth map and the cabinet 3D point cloud of the camera coordinate system according to the binocular images of each cabinet includes:

[0014] Get the intrinsic parameter matrix and binocular epipolar distance of the binocular camera;

[0015] According to the binocular epipolar distance, binocular epipolar matching is performed on the left cabinet image and the right cabinet image, and a corresponding cabinet disparity map is generated by the SGBM algorithm;

[0016] Obtaining a corresponding cabinet depth map according to the cabinet disparity map and the binocular baseline;

[0017] According to the cabinet depth map and the intrinsic parameter matrix, a 3D point cloud of the cabinet in the camera coordinate system is obtained.

[0018] Furthermore, the step of obtaining the cabinet 3D point cloud of the camera coordinate system according to the cabinet depth map and the intrinsic parameter matrix includes:

[0019] According to the cabinet depth map and the internal parameter matrix, a 3D point cloud of the cabinet in the initial camera coordinate system is calculated;

[0020] According to the octree structure, the cabinet 3D point cloud of the initial camera coordinate system is divided into voxels to obtain a sparse 3D data tree with the same time complexity and space complexity;

[0021] Using a preset discrete value filter, the nodes of the sparse 3D data tree are subjected to sparse outlier removal processing to obtain a dense 3D point cloud of the cabinet;

[0022] The dense 3D point cloud of the cabinet is processed by voxel downsampling and Gaussian curvature filtering in sequence to obtain the 3D point cloud of the cabinet in the camera coordinate system.

[0023] Furthermore, the preset object detection model is a bottom-up openpose key point detection model with mobilenet as the backbone network, which is pre-trained according to an offline training data set; the offline training data set includes cabinet image data under various environmental backgrounds, various environmental lightings, and various camera perspectives in the computer room;

[0024] The step of obtaining corresponding three-dimensional cabinet key point coordinates according to a plurality of two-dimensional cabinet key point coordinates of binocular images of each cabinet by using a preset object detection model and according to the two-dimensional cabinet key point coordinates and the cabinet depth map comprises:

[0025] Inputting the left-eye cabinet image or the right-eye cabinet image into the preset object detection model, and obtaining a plurality of two-dimensional cabinet key point coordinates of the cabinet foot area in the cabinet binocular image;

[0026] An image coordinate system is established according to the binocular images of each cabinet, and the cabinet depth map is queried according to the coordinates of each two-dimensional cabinet key point to obtain the corresponding key point depth information;

[0027] The depth information of each key point is converted into the three-dimensional coordinates of the camera coordinate system to obtain the corresponding three-dimensional cabinet key point coordinates.

[0028] Furthermore, the step of fitting the corresponding potential cabinet plane equation according to the cabinet 3D point cloud of each camera coordinate system includes:

[0029] Convert the 3D Euclidean space coordinates of the cabinet 3D point cloud in the camera coordinate system into the corresponding Hough space coordinates in the polar coordinate system;

[0030] Determine the initial parameter range of the plane to be fitted according to a plurality of intersection points of the parameter surfaces corresponding to the Hough space coordinates;

[0031] According to the prior information of the cabinet plane, the initial parameter range of the plane to be fitted is screened to obtain candidate points of the plane to be fitted;

[0032] Performing Hough space voting on the candidate points of the plane to be fitted, and extracting the cabinet plane point cloud area with a voting number greater than a preset threshold;

[0033] According to the cabinet plane point cloud area, a corresponding potential cabinet plane equation is obtained by fitting.

[0034] Furthermore, the step of obtaining the camera relative to the ground plane external parameter matrix by solving the multi-plane equation group includes:

[0035] According to the multi-plane equation group, a corresponding minimization error equation group is constructed, and the solution problem of the minimization error equation group is converted into a least squares problem of the corresponding linear equation group;

[0036] Performing SVD decomposition on the cross covariance matrix corresponding to the linear equation system to obtain a ground plane rotation matrix of the camera coordinate system, and performing angular axis decomposition on the ground plane rotation matrix to obtain a ground plane normal vector;

[0037] Obtaining the height of the camera coordinate system relative to the ground plane according to the ground plane normal vector and the coordinates of the key points of the three-dimensional cabinet;

[0038] According to the Y-axis vector of the camera coordinate system and the ground plane normal vector, a rotation axis and a rotation angle are obtained, and the rotation axis and the rotation angle are converted into a rotation matrix of the camera coordinate system relative to the ground plane;

[0039] According to the height of the camera coordinate system relative to the ground plane and the rotation matrix of the camera coordinate system relative to the ground plane, the camera relative to the ground plane extrinsic parameter matrix is ​​obtained; the camera relative to the ground plane extrinsic parameter matrix is ​​expressed as:

[0040] T=[R|t]

[0041] In the formula,

[0042]

[0043]

[0044]

[0045] Where T represents the camera's external parameter matrix relative to the ground plane; R and t represent the camera coordinate system rotation matrix and the camera coordinate system's height relative to the ground plane, respectively; θ, and They represent the rotation axis, rotation angle, ground plane equation normal vector and the Y-axis vector of the camera coordinate system respectively.

[0046] Furthermore, the step of obtaining the height of the camera coordinate system relative to the ground plane according to the ground plane normal vector and the three-dimensional cabinet key point coordinates includes:

[0047] Substituting the ground plane normal vector and the coordinates of each key point of the three-dimensional cabinet into the plane equation formula respectively, to obtain corresponding candidate ground plane equations respectively;

[0048] The ground plane equation is obtained by averaging the candidate ground plane equations;

[0049] According to the ground plane equation, the height of the camera coordinate system relative to the ground plane is obtained; the height of the camera coordinate system relative to the ground plane is expressed as:

[0050] t=(0,0,height)

[0051] In the formula,

[0052]

[0053] Where t represents the height of the camera coordinate system relative to the ground plane; D represents the intercept of the equation from the origin to the ground plane; Represents the normal vector of the ground plane equation The norm of .

[0054] In a second aspect, an embodiment of the present invention provides a system for online calibration of a plane of a camera in a computer room, the system comprising:

[0055] An image acquisition module is used to obtain a preset number of cabinet binocular time-series images through a binocular camera; the cabinet binocular images include left cabinet images and right cabinet images;

[0056] The image processing module is used to extract the corresponding cabinet depth map and the cabinet 3D point cloud of the camera coordinate system according to the binocular image of each cabinet;

[0057] A key point extraction module, used to obtain corresponding three-dimensional cabinet key point coordinates according to a plurality of two-dimensional cabinet key point coordinates of the binocular image of each cabinet through a preset object detection model, and according to the two-dimensional cabinet key point coordinates and the cabinet depth map;

[0058] A plane fitting module is used to fit the corresponding potential cabinet plane equation according to the cabinet 3D point cloud of each camera coordinate system; the potential cabinet plane equation includes a potential left cabinet plane equation and a potential right cabinet plane equation;

[0059] The external parameter solving module is used to solve all potential cabinet plane equations to obtain a multi-plane equation group, and obtain the camera relative to the ground plane external parameter matrix by solving the multi-plane equation group.

[0060] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0061] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0062] The present application provides a method, system, computer equipment and storage medium for online calibration of the plane of a computer room camera. Through the method, a preset number of binocular time-series images of a cabinet are obtained through a binocular camera, and the corresponding cabinet depth map and the cabinet 3D point cloud of the camera coordinate system are extracted. The coordinates of multiple two-dimensional cabinet key points of the cabinet binocular image are extracted through a preset object detection model, and the corresponding three-dimensional cabinet key point coordinates are obtained according to the two-dimensional cabinet key point coordinates and the cabinet depth map. Then, the corresponding potential cabinet plane equation obtained by fitting the cabinet 3D point cloud of the camera coordinate system is combined to obtain a multi-plane equation group, and the multi-plane equation group is solved to obtain the camera relative to the ground plane external parameter matrix. Compared with the prior art, the online calibration method of the camera plane of the computer room camera can reasonably and effectively utilize the on-site information of the computer room, and can realize the online correction of camera parameters based on the prior information of the computer room without the need for a calibration board. It can not only avoid the modification of the on-site environment, but also ensure the efficiency and accuracy of online calibration, as well as the high robustness to adapt to environmental changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 Schematic diagram of an application scenario of the online calibration method for a camera plane in a computer room according to an embodiment of the present invention;

[0064] Figure 2 It is a schematic diagram of the process of the online calibration method of the camera plane in the computer room according to an embodiment of the present invention;

[0065] Figure 3 Schematic diagram of key points of the machine feet of a machine room cabinet in an embodiment of the present invention;

[0066] Figure 4 is a structural schematic diagram of a plane online calibration system for a computer room camera according to an embodiment of the present invention;

[0067] Figure 5 It is a diagram of the internal structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical scheme and beneficial effects of the present application clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Obviously, the embodiments described below are part of the embodiments of the present invention and are only used to illustrate the present invention, but are not used to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] The online calibration method for a computer room camera plane provided by the present invention can be applied to Figure 1 The terminal or server shown. Among them, the terminal can be but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server can be implemented by an independent server or a server cluster composed of multiple servers. For example, the server can be trained based on the cabinet image data under various environmental backgrounds, various environmental lighting, and various camera perspectives in the computer room as a training data set to obtain a bottom-up openpose key point detection model with mobilenet as the backbone network as a preset object detection model, based on the cabinet binocular image obtained by taking a binocular color camera in real time, the computer room camera plane online calibration method provided by the present invention is used to complete the online calibration of the computer room camera plane, and directly control the camera extrinsic parameter matrix adjustment, or transmit it to the terminal for the terminal user to receive and evaluate and adjust the camera extrinsic parameter matrix for use.

[0070] The method of the present invention is mainly used to calibrate the external parameter projection matrix of the patrol camera in the computer room. In the environment of computer room inspection, the cabinet is an original objective object, and the cabinet plane is perpendicular to the ground plane and arranged regularly. When the patrol car walks in the cabinet environment, the cabinets on both sides of the aisle can be seen, that is, strong position prior information is hidden behind them. Calibration of the camera plane based on the cabinet plane can not only avoid the modification of the on-site environment and ensure the accuracy and efficiency of real-time calibration, but also ensure high robustness to adapt to environmental changes; in the camera calibration process based on the cabinet plane information, four coordinate systems, namely, the image coordinate system, the image world coordinate system, the binocular camera coordinate system and the vehicle body coordinate system on which the binocular camera is installed, are required, all of which are established with reference to the right-hand coordinate system. Specifically, the origin of the image coordinate system is the center point of the left eye image in the binocular camera, the positive direction of the x-axis points to the right of the image, the positive direction of the y-axis points to the bottom of the image, and the z-axis is established according to the right-hand coordinate system; the origin of the binocular camera coordinate system is the geometric center of the left eye lens, the positive direction of the x-axis points to the right eye lens, the positive direction of the y-axis points to the bottom of the camera, and the z-axis is established according to the right-hand coordinate system; the image world coordinate system is a relative quantity. In the description of the key points of the binocular image of the cabinet, a key point with the smallest coordinate value is taken as the coordinate origin, the positive direction of the x-axis is the same as that of the binocular camera coordinate system, and the positive direction of the y-axis is the same as that of the binocular camera coordinate system; the positive direction of the x-axis of the vehicle body coordinate system points to the front of the vehicle, and the positive direction of the y-axis points to the left side of the vehicle body. The z-axis is established according to the right-hand coordinate system, and the origin coincides with the origin of the binocular camera coordinate system on the z-axis; the following embodiments will be based on the above four coordinate systems to describe the online calibration method of the computer room camera plane of the present invention in detail.

[0071] In one embodiment, Figure 2 As shown, a method for online calibration of a camera plane in a computer room is provided, comprising the following steps:

[0072] S11. Obtain a preset number of binocular time-series images of the cabinet through a binocular camera; the binocular images of the cabinet include a left-eye cabinet image and a right-eye cabinet image; wherein the binocular camera may be a binocular color camera installed on a differential wheel patrol AGV (Automated Guided Vehicle), which is rigidly connected to the AGV body and cannot be moved; correspondingly, the left-eye cabinet image and the right-eye cabinet image are images of the actual cabinets in the computer room collected in real time by the binocular color camera; the preset number of binocular time-series images of the cabinet can be determined according to actual application requirements, and no specific restrictions are made here. The preset number directly affects the number of plane equations for solving the camera extrinsic parameters by the subsequent simultaneous multi-plane equations. In principle, the larger the preset number, the more accurate the subsequent online calibration results can be guaranteed;

[0073] S12. According to the binocular images of each cabinet, extract the corresponding cabinet depth map and the cabinet 3D point cloud of the camera coordinate system; wherein, the cabinet depth map and the cabinet 3D point cloud of the camera coordinate system can be realized according to the prior art. In this example, the SGBM algorithm in OpenCV is preferably used to perform stereo matching on the cabinet binocular images to obtain the cabinet depth map, and then the camera coordinate system cabinet 3D point cloud is calculated in combination with the camera intrinsic parameters; specifically, the step of extracting the corresponding cabinet depth map and the cabinet 3D point cloud of the camera coordinate system according to the binocular images of each cabinet includes:

[0074] Get the intrinsic parameter matrix and binocular epipolar distance of the binocular camera; the intrinsic parameter matrix can be obtained through the driver API of the binocular camera;

[0075] According to the binocular epipolar distance, binocular epipolar matching is performed on the left cabinet image and the right cabinet image, and a corresponding cabinet disparity map is generated by the SGBM algorithm;

[0076] Obtaining a corresponding cabinet depth map according to the cabinet disparity map and the binocular baseline;

[0077] According to the cabinet depth map and the intrinsic parameter matrix, a 3D point cloud of the cabinet in the camera coordinate system is obtained.

[0078] In principle, the two-dimensional cabinet depth map can be directly converted into a 3D point cloud in three-dimensional space according to the cabinet depth map obtained and the corresponding internal parameter matrix. The 3D point cloud of the cabinet in the camera coordinate system obtained by this method can be directly used for subsequent online calibration. However, considering that there are sparse outliers in the 3D point cloud and the parallax obtained by the SGBM method of OpenCV is Figure 1Generally, there is more speckle noise. In order to obtain better depth point cloud data in the computer room scene, this embodiment preferably performs pre-processing such as filtering and downsampling on the 3D point cloud data directly obtained by the above method to remove noise and redundant point data information, and then uses it for camera plane calibration, thereby effectively ensuring the efficiency and accuracy of the camera online calibration processing; Specifically, the step of obtaining the cabinet 3D point cloud of the camera coordinate system according to the cabinet depth map and the intrinsic parameter matrix includes:

[0079] According to the cabinet depth map and the internal parameter matrix, the cabinet 3D point cloud of the initial camera coordinate system is calculated; wherein the three-dimensional coordinate points in the cabinet 3D point cloud of the initial camera coordinate system satisfy the following formula:

[0080]

[0081]

[0082] Among them, u and v represent the x-axis and y-axis coordinate information of the two-dimensional point in the image coordinate system; u0 and v0 represent the center coordinates of the image respectively; f / dx and f / dy represent the length of the focal length in the x-axis and y-axis directions described by pixels respectively; x w ,y w 、z w Represents the x-axis, y-axis, and z-axis coordinate information of a three-dimensional point in the camera coordinate system; c Represents the z-axis value of the camera coordinate system, that is, the distance from the target to the camera; R and T represent the 3x3 rotation matrix and 3x1 translation matrix of the extrinsic parameter matrix respectively. The 3D point here is relative to the camera coordinate system, and [RT] is taken as the unit transformation matrix.

[0083] According to the octree structure, the cabinet 3D point cloud of the initial camera coordinate system is divided into voxels to obtain a sparse 3D data tree with the same time complexity and space complexity; wherein, the method for obtaining the sparse 3D data tree can be obtained by using the existing octree voxel division, which will not be repeated here;

[0084] The nodes of the sparse 3D data tree are subjected to sparse outlier removal processing by a preset discrete value filter to obtain a dense 3D point cloud of the cabinet; wherein the discrete value filter is a statistical analysis filter created according to the following formula; since the object field of view and the environment field of view in the computer room scene are not too far away, when the statistical analysis filter is used for speckle filtering in this embodiment, the filter window size is selected to be 20 pixels and the speckle threshold is 10, so as to ensure that better depth point cloud data is obtained in the computer room scene;

[0085]

[0086] In the formula,

[0087]

[0088]

[0089] Where C represents the local covariance matrix; Represents the geometric center coordinates within the domain; p i Represents the coordinates of the point cloud in the current camera coordinate system; k represents the number of points; ξ i Indicates the current point p i The weights of μ and d i Represent the current query point p i mean and distance from the domain;

[0090] All nodes of the sparse 3D data tree obtained above are input into the discrete value filter, and the neighborhood of each node is subjected to outlier statistical analysis similar to the local covariance, and those points that do not meet certain standards are pruned, and the valid 3D point cloud after discrete value filtering is output; specifically, each node is traversed, and the average distance from the point to all its neighboring points is calculated. Assuming that the result is a Gaussian distribution, whose shape is determined by the mean and standard deviation, then the points whose average distance is outside the standard range (defined by the global distance mean and variance) can be defined as outliers and can be removed from the data set. For example, the distance radius of each field analysis is set to 5 cm, and the multiple of the standard deviation is set to 1, that is, if the distance of a point exceeds the average distance by more than one standard deviation, the point is marked as an outlier and removed.

[0091] Since the above-mentioned method of stereo matching of binocular cabinet images to obtain a cabinet depth map uses binocular polar line search matching, there are some inherent measurement accuracy errors. If you take an image of a flat wall and then check the depth value, you will see some non-planar blurry phenomena in the corners of the image, and when checking the range (or standard deviation) of the depth value of each (x, y) pixel on multiple images, there is a noise difference between the center pixel and the edge pixel. In order to solve the above problems, the dense 3D point cloud data of the cabinet obtained in the above steps in this embodiment is further processed by voxel downsampling and Gaussian curvature filtering using the following method, retaining more samples in places with large point cloud curvature, increasing the sampling rate in quasi-plane areas, and removing redundant point cloud data, thereby effectively improving subsequent calculation efficiency;

[0092] The dense 3D point cloud of the cabinet is subjected to voxel downsampling and Gaussian curvature filtering in sequence to obtain the 3D point cloud of the cabinet in the camera coordinate system. Specifically, voxel downsampling and Gaussian curvature filtering, i.e., curvature resampling, is also called geometric sampling. The greater the curvature of the point cloud, i.e., the more obvious the geometric features, the more sampling points are distributed. The calculation of the point cloud curvature is relatively time-consuming. This embodiment preferably adopts a simple method to approximate the curvature effect. The specific method is as follows:

[0093] a) Preset the target sampling number S to 20 and the sampling uniformity U to 0.32; traverse each point cloud data, select the K neighboring points of each point, construct the local covariance, decompose and take the eigenvector corresponding to the maximum eigenvalue, i.e. the normal vector; then determine the angle between the normal lines of two adjacent points, i.e. the curvature. The larger the angle, the greater the curvature.

[0094] b) Set an angle threshold, such as 5 degrees; points whose neighborhood angle values ​​are greater than this threshold are placed in the geometric feature area G, and the rest are placed in the inconspicuous area, dividing the point cloud into two parts: the geometric feature area G and other areas;

[0095] c) Uniformly sample the geometric feature area G and other areas respectively, and the corresponding sampling numbers are S*(1-U) and S*U respectively.

[0096] Through the above-mentioned geometric division of the dense 3D point cloud of the cabinet and the normal vector curvature filtering, the redundant information of the point cloud after sampling can be effectively reduced, making it more noise-resistant, providing a reliable guarantee for the effectiveness of the subsequent potential cabinet plane fitting, and also ensuring the accuracy of the subsequent online calibration.

[0097] S13, obtaining corresponding three-dimensional cabinet key point coordinates according to a plurality of two-dimensional cabinet key point coordinates of binocular images of each cabinet through a preset object detection model, and according to the two-dimensional cabinet key point coordinates and the cabinet depth map; wherein, the cabinet binocular image is as described above, which is the left and right RGB images collected by the binocular camera, and the cabinet physical image is imported into a preset specific object detection model, and a specific number (generally greater than 4) of key point two-dimensional image coordinate data are predicted and generated;

[0098] In principle, the preset object detection model can adopt other existing network models that can realize the required feature extraction. However, in order to take into account both the efficiency and accuracy of the cabinet key point information extraction, in this embodiment, preferably, the preset object detection model collects cabinet images including various environmental backgrounds, various environmental lighting, and various camera perspectives in the computer room in advance, and uses the cabinet foot information in the cabinet image as the key point to make an offline training data set, and then performs offline training to generate an openpose key point detection model with mobilenet as the backbone; wherein openpose is a bottom-up key point extraction model, and mobilenet is a lightweight feature extraction network that can run in real time on a consumer-grade CPU. With mobilenet as the backbone network, the number of parameters after replacing the second-order feature extraction module in openpose is only 15% of the original openpose model parameters, while the performance is almost the same (accuracy is reduced by 1%). When the improved preset object detection model runs on an Intel i7-7700 PC with 8G memory, it can achieve a processing performance of 10 frames per second. Specifically, the step of obtaining corresponding three-dimensional cabinet key point coordinates according to a plurality of two-dimensional cabinet key point coordinates of binocular images of each cabinet by using a preset object detection model and according to the two-dimensional cabinet key point coordinates and the cabinet depth map includes:

[0099] Inputting the left-eye cabinet image or the right-eye cabinet image into the preset object detection model, and obtaining a plurality of two-dimensional cabinet key point coordinates of the cabinet foot area in the cabinet binocular image;

[0100] An image coordinate system is established according to the binocular images of each cabinet, and the cabinet depth map is queried according to the coordinates of each two-dimensional cabinet key point to obtain the corresponding key point depth information;

[0101] The depth information of each key point is converted into the three-dimensional coordinates of the camera coordinate system to obtain the corresponding three-dimensional cabinet key point coordinates.

[0102] In the actual computer room environment, Figure 3As shown, the plane represented by the key points of the cabinet feet is equivalent to the ground plane; in principle, the coordinates of the cabinet key points extracted in this embodiment can be directly used to calculate the ground plane equation represented, but because the accuracy of extracting key points by the above-mentioned preset object detection model is highly dependent on the accuracy of the image annotation data in the offline training data set, therefore, on the basis of the difficulty in ensuring the accuracy of the image annotation in the offline training data set, in order to ensure the accuracy of solving the ground plane, this embodiment preferably adopts the following method, based on the cabinet 3D point cloud fitting of the camera coordinate system corresponding to the binocular time-series images of each cabinet to obtain the corresponding potential cabinet plane equation, and then the camera extrinsic parameter matrix is ​​obtained by solving the multi-plane joint optimization method of the multi-plane equation generated based on the joint of all potential cabinet plane equations.

[0103] S14, according to the cabinet 3D point clouds of each camera coordinate system, the corresponding potential cabinet plane equation is fitted; the potential cabinet plane equation includes a potential left cabinet plane equation and a potential right cabinet plane equation; wherein, the fitting of the potential cabinet plane equation preferably adopts plane fitting based on Hough space voting, by converting the Euclidean space into the Hough space of spatial linear parameters, performing Hough space voting on each candidate point, extracting the left and right cabinet plane point cloud areas and the corresponding plane area equations, and laying a foundation for the next step of solving the external parameter rotation matrix by simultaneously solving the multi-plane equations; specifically, the step of fitting the corresponding potential cabinet plane equation according to the cabinet 3D point clouds of each camera coordinate system includes:

[0104] The 3D Euclidean space coordinates of the cabinet 3D point cloud in the camera coordinate system are converted into the corresponding Hough space coordinates in the polar coordinate system; wherein the 3D Euclidean space of the cabinet 3D point cloud in the camera coordinate system is represented in a three-dimensional plane as follows:

[0105] a0X+a1Y+a3Z+D=0(D≠0)(1)

[0106] Among them, X, Y, Z are the X-axis, Y-axis and Z-axis coordinates of the point cloud belonging to the same plane respectively; (a0, a1, a2) represents the normal vector of the plane, and D is the intercept from the origin (0, 0, 0) to the plane; substituting the plane normal vector into the polar coordinate system, the expression of the plane normal vector in the polar coordinate system can be obtained:

[0107]

[0108] Among them, θ∈[0,2π], φ∈[0,2π]; the Hough parameter space can be discretized by discretizing θ, φ and the distance ρ from the origin to the plane. Correspondingly, the plane equation can be expressed as:

[0109] (cosφsinθ)x+(sinφsinθ)y+(cosθ)z+ρ=0 (2)

[0110] For any three-dimensional point, convert from Euclidean space to Hough space, and substitute (θ, φ) into the equation to obtain the corresponding ρ. Therefore, each point can form a corresponding parametric surface in Hough parameter space.

[0111] According to the multiple intersection points of the parametric surface corresponding to the Hough space coordinates, the initial parameter range of the plane to be fitted is determined; wherein the parametric surface corresponding to the multiple point Hough space coordinates will form multiple intersection points, wherein any three non-collinear intersection points can determine a plane, that is, only the initial parameter range of the plane to be fitted can be preliminarily obtained according to all the intersection points, and then the intersection points are screened by the following method steps to further determine the corresponding cabinet plane point cloud area, that is, when the number of intersection points corresponding to the parameters (θ, φ, ρ) is greater than the preset threshold, it corresponds to the desired plane equation;

[0112] According to the prior information of the cabinet plane, the initial parameter range of the plane to be fitted is screened to obtain candidate points of the plane to be fitted; wherein, the prior information of the cabinet plane can be understood as the normal vectors of the cabinet plane are parallel to each other, (x, y, z) is equivalent to (-x, -y, -z), and its normal vector is parallel to the ground plane; accordingly, combined with the specific meaning of the polar coordinate Hough space (θ, φ), the value range of the above parameters can be further narrowed to θ∈[0,π], φ∈[0,π], and the candidate points of the plane to be fitted are obtained, which can not only exclude the influence of irrelevant planes, but also greatly reduce the amount of calculation, thereby effectively improving the efficiency of the algorithm; however, the planes obtained in the candidate points of the plane to be fitted also contain a certain amount of noise, and due to the influence of the noise, the same plane may be divided into multiple (θ, φ, ρ) parameter spaces in the Hough space, which will cause the Hough peak value in the subsequent statistical accumulator to become blurred. To address this phenomenon, this embodiment preferably adopts the following voting method to solve it, that is, when performing voting cumulative value peak statistics in the Hough space, the number of points in each basic unit is not counted to obtain the maximum value, but a neighborhood of each unit is merged and counted, thereby solving the peak dispersion problem caused by noise.

[0113] The candidate points of the plane to be fitted are voted in Hough space, and the cabinet plane point cloud area with a number of votes greater than a preset threshold is extracted; wherein the idea of ​​Hough space voting can be summarized as follows: each candidate point of the plane to be fitted votes for the Hough space parameter that meets the requirement, and the Hough space parameter with the most votes wins and becomes the optimal plane estimation;

[0114] Specifically, a buffer is initialized to construct a three-dimensional array corresponding to the parameter space, and all its data are set to 0. The size of the three-dimensional array is the step size of the parameter discretization. In this embodiment, stepθ=π / 180 and stepφ=π / 180 are selected respectively; the third dimension stores the number of points that conform to the plane, which is called an accumulator or a voter; at the same time, a preset threshold is determined according to the actual situation, and the Hough space is discretized according to the following process based on this threshold: the Hough space coordinates corresponding to each three-dimensional point are traversed, and the corresponding (θ, φ, ρ) array is accumulated. Add 1 to the vote, that is, in the cumulative statistical stage, add the values ​​that fall into the same discretization interval as the cumulative voting score of this interval. At the same time, due to the symmetry of the normal vector, when counting the number of accumulators, it is considered that θ=0 and θ=π are adjacent and belong to the same plane. Similarly, φ=0 and φ=π are adjacent and belong to the same plane. After the traversal voting is completed, find the area in the Hough space parameter whose cumulative value is greater than the preset threshold, and calculate the average value of this area to determine the final cabinet plane point cloud area, that is, the left cabinet plane point cloud area and the right cabinet plane point cloud area are obtained.

[0115] According to the cabinet plane point cloud area, the corresponding potential cabinet plane equation is fitted; then the selected cabinet plane point cloud area satisfies the following equation:

[0116] ρ=xcos(θ)sin(φ)+ysin(θ)sin(φ)+zcos(φ), ρ∈[0,R] (3)

[0117] Among them, R is the maximum value in the three-dimensional data;

[0118] From formula (3), we can determine:

[0119] A=cos(θ)sin(φ)

[0120] B=sin(θ)sin(φ)

[0121] C=cos(φ)

[0122] D=-ρ

[0123] That is, the corresponding potential cabinet plane equation obtained by fitting is: Ax+By+Cz+D=0, and the corresponding Hough space parameter area is the point cloud cluster belonging to the same plane.

[0124] The Hough space plane fitting method used in this embodiment can fit the equations of multiple different planes at the same time through one calculation. Compared with the plane fitting method based on random sampling consensus (RANSAC) and its variants, the amount of calculation is not significantly increased due to the increase in the number of fitting planes.

[0125] S15. Combine all potential cabinet plane equations to obtain a multi-plane equation group, and obtain the camera relative to the ground plane extrinsic parameter matrix by solving the multi-plane equation group; wherein the number of potential cabinet plane equations is twice the preset number of cabinet binocular time-series images; the patrol car does not destroy the assumption that the normal vector of the cabinet plane is parallel to the ground plane during the movement of the machine room. Based on this, this embodiment preferably uses a sliding window method to combine multiple potential cabinet plane equations corresponding to a preset number of cabinet binocular time-series images, and obtains the minimum value with the least squares meaning through SVD decomposition to determine the optimal camera extrinsic parameter rotation matrix;

[0126] Specifically, if the preset number of cabinet binocular time-series images is set to 20, each cabinet binocular image in the window obtains 2 cabinet plane point cloud clusters and corresponding plane equations according to the above method, that is, 40 equations can be obtained by combining the potential cabinet plane equations of 20 cabinet binocular time-series images; specifically, the step of obtaining the camera relative to the ground plane external parameter matrix by solving the multi-plane equation group includes:

[0127] According to the multi-plane equation group, a corresponding minimization error equation group is constructed, and the solution problem of the minimization error equation group is converted into a least squares problem of the corresponding linear equation group; wherein, the multi-plane equation group as described above can be understood as a potential cabinet plane equation that includes all cabinet binocular time-series images at the same time, assuming that PG is the ground plane to be determined, that is, the unknown quantity to be optimized, PL and PR are the potential left cabinet plane and the potential right cabinet plane obtained by fitting, respectively, then the error equation corresponding to the two potential cabinet plane equations of each cabinet binocular time-series image can be expressed as:

[0128]

[0129]

[0130] In the formula,

[0131]

[0132]

[0133]

[0134] Among them, i and N represent the sequence number and total number of binocular time-series images of the cabinet, respectively; Represents the normal vector of the key point plane PG, which is the variable to be optimized, and and Respectively represent the normal vector of plane PL and the normal vector of plane PR; R x90 , R -x90 Respectively represent the positive and negative rotation of π / 2 around the X-axis of the vehicle coordinate system;

[0135] For the on-site patrol scene of the computer room, and Logically represents the vertical plane of the field cabinet surface shell, so and After the rotation matrix of is rotated 90° in both the forward and reverse directions, it should be parallel to the ground plane PG. Therefore, according to the above two error equations, the corresponding minimum error equation group can be constructed, and then the minimum error problem can be converted into a least squares problem for solving the linear equation group AX=b, that is,

[0136] x=(A T A) -1 A T b

[0137]

[0138] Among them, (x i ,y i , z i ) represents the input ground plane normal vector; the corresponding cross covariance matrix can be expressed as:

[0139]

[0140]

[0141] The cross covariance matrix corresponding to the linear equation system is decomposed by SVD to obtain the ground plane rotation matrix of the camera coordinate system, and the ground plane rotation matrix is ​​decomposed by angular axis to obtain the ground plane normal vector; wherein the cross covariance matrix is ​​as shown above, through UQV T The SVD matrix decomposition method is used to obtain the ground plane rotation matrix of the camera coordinate system: R = VU T , and according to Rodrigues's Formula, the angular axis form is decomposed into That is, the optimized ground plane normal vector is obtained;

[0142] According to the ground plane normal vector and the three-dimensional cabinet key point coordinates, the height of the camera coordinate system relative to the ground plane is obtained; wherein, after the ground plane normal vector is determined by the above method, it can be combined with the three-dimensional cabinet key point coordinates obtained above to determine the required ground plane equation, and then the height of the camera coordinate system relative to the ground plane is calculated according to the ground plane equation; specifically, the step of obtaining the height of the camera coordinate system relative to the ground plane according to the ground plane normal vector and the three-dimensional cabinet key point coordinates includes:

[0143] Substitute the ground plane normal vector and the coordinates of each key point of the three-dimensional cabinet into the plane equation formula to obtain the corresponding candidate ground plane equations; wherein the candidate ground plane equations are expressed as:

[0144] a0X+a1Y+a3Z+D i =0(D i ≠0, i=1, 2, 3, ...)

[0145] The ground plane equation is obtained by averaging the candidate ground plane equations; the ground plane equation is expressed as:

[0146] a0X+a1Y+a3Z+D=0

[0147] Where D is D i The average value of

[0148] According to the ground plane equation, the height of the camera coordinate system relative to the ground plane is obtained; the height of the camera coordinate system relative to the ground plane is expressed as:

[0149] t=(0,0,height)

[0150] In the formula,

[0151]

[0152] Where t represents the height of the camera coordinate system relative to the ground plane; D represents the intercept of the equation from the origin to the ground plane; Represents the normal vector of the ground plane equation The norm of

[0153] In this embodiment, the ground plane normal vector is determined based on the prior information of the computer room environment, and a large number of cabinet key points extracted from the binocular time-series images of a preset number of cabinets are combined to determine the final ground plane equation, providing a reliable guarantee for the accuracy of the ground plane equation;

[0154] According to the Y-axis vector of the camera coordinate system and the ground plane normal vector, the rotation axis and the rotation angle are obtained, and the rotation axis and the rotation angle are converted into the rotation matrix of the camera coordinate system relative to the ground plane; wherein, the plane formed by the Y-axis vector of the camera coordinate system and the ground plane normal vector must be the plane where the rotation angle is located, and the rotation axis must be perpendicular to the plane, that is, the rotation axis and the rotation angle can be expressed as:

[0155]

[0156]

[0157] In the formula, θ, and Respectively represent the rotation axis, rotation angle, ground plane equation normal vector and the Y-axis vector of the camera coordinate system; after normalizing the obtained rotation axis to obtain the unit rotation axis, the Rodrigues' rotation formula is used to obtain the rotation matrix R of the camera coordinate system relative to the ground plane;

[0158] According to the height of the camera coordinate system relative to the ground plane and the rotation matrix of the camera coordinate system relative to the ground plane, the camera relative to the ground plane extrinsic parameter matrix is ​​obtained; the camera relative to the ground plane extrinsic parameter matrix is ​​expressed as:

[0159] T=[R|t]

[0160] In the formula,

[0161]

[0162] Where T represents the external parameter matrix of the camera relative to the ground plane; R and t represent the rotation matrix of the camera coordinate system and the height of the camera coordinate system relative to the ground plane, respectively.

[0163] The embodiment of the present application obtains depth information and 3D point cloud information by performing binocular matching on a preset number of acquired binocular time-series images of cabinets, and after the 3D point cloud information is processed by discrete value filtering, voxel downsampling and Gaussian curvature filtering in sequence, the potential cabinet plane equation is fitted based on Hough space voting, and then a multi-plane equation group is obtained by jointly establishing the potential cabinet plane equations corresponding to the preset number of cabinet binocular time-series images, and combined with the cabinet key point information of the computer room cabinet image extracted by a specific lightweight key point extraction network, the environmental information is maximized, and the camera relative ground plane extrinsic parameter matrix is ​​updated in real time without a calibration board, that is, the online correction of camera parameters is realized based on the prior information of the computer room, which not only avoids the modification of the on-site environment, but also ensures the high efficiency and accuracy of online calibration, as well as the high robustness to adapt to environmental changes.

[0164] It should be noted that although the steps in the above flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders.

[0165] In one embodiment, Figure 4 As shown, a plane online calibration system for a computer room camera is provided, and the system comprises:

[0166] The image acquisition module 1 is used to obtain a preset number of cabinet binocular time-series images through a binocular camera; the cabinet binocular images include left cabinet images and right cabinet images;

[0167] Image processing module 2, used to extract the corresponding cabinet depth map and camera coordinate system cabinet 3D point cloud according to the binocular image of each cabinet;

[0168] The key point extraction module 3 is used to obtain the corresponding three-dimensional cabinet key point coordinates according to the multiple two-dimensional cabinet key point coordinates of the binocular image of each cabinet through a preset object detection model, and according to the two-dimensional cabinet key point coordinates and the cabinet depth map;

[0169] A plane fitting module 4 is used to fit the corresponding potential cabinet plane equation according to the cabinet 3D point cloud of each camera coordinate system; the potential cabinet plane equation includes a potential left cabinet plane equation and a potential right cabinet plane equation;

[0170] The external parameter solving module 5 is used to solve all potential cabinet plane equations to obtain a multi-plane equation group, and obtain the camera relative to the ground plane external parameter matrix by solving the multi-plane equation group.

[0171] For the specific definition of a computer room camera plane online calibration system, please refer to the definition of a computer room camera plane online calibration method mentioned above, which will not be repeated here. Each module in the above-mentioned computer room camera plane online calibration system can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0172] Figure 5 FIG. 1 shows an internal structure diagram of a computer device in an embodiment, and the computer device may specifically be a terminal or a server. Figure 5 As shown, the computer device includes a processor, a memory, a network interface, a display and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for online calibration of a camera plane in a computer room is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0173] It can be understood by those skilled in the art that Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the present application scheme, and does not constitute a limitation on the computer device to which the present application scheme is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have the same component arrangement.

[0174] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the computer program.

[0175] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0176] In summary, the embodiments of the present invention provide a computer room camera plane online calibration method, system, computer equipment and storage medium, wherein the computer room camera plane online calibration method realizes obtaining a preset number of cabinet binocular time-series images through a binocular camera, extracting the corresponding cabinet depth map, camera coordinate system cabinet 3D point cloud, and further screening the 3D point cloud information through discrete value filtering, voxel downsampling and Gaussian curvature filtering in sequence, and extracting multiple two-dimensional cabinet key point coordinates of the cabinet binocular image through a specific lightweight key point extraction network, and extracting the two-dimensional cabinet key point coordinates according to the two-dimensional cabinet key point extraction network. After obtaining the corresponding three-dimensional cabinet key point coordinates from the key point coordinates and the cabinet depth map, the corresponding potential cabinet plane equations obtained by fitting the cabinet 3D point cloud according to the camera coordinate system are combined to obtain a multi-plane equation group, and the multi-plane equation group is solved to obtain the camera relative to the ground plane extrinsic matrix. By reasonably and effectively utilizing the on-site information of the computer room, the camera parameters can be corrected online based on the prior information of the computer room without a calibration board. This can not only avoid the modification of the on-site environment, but also ensure the efficiency and accuracy of online calibration, as well as the high robustness to adapt to environmental changes.

[0177] Each embodiment in this specification is described in a progressive manner, and the same or similar parts of each embodiment can be directly referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. It should be noted that the technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0178] The above-mentioned embodiments only express several preferred implementation modes of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in the technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be regarded as the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be based on the protection scope of the claims.

Claims

1. A method for online calibration of a camera plane in a computer room, characterized in that: The method comprises the following steps: Obtain a preset number of cabinet binocular images through a binocular camera; the cabinet binocular images include a left cabinet image and a right cabinet image; According to the binocular images of each cabinet, the corresponding cabinet depth map and the cabinet 3D point cloud of the camera coordinate system are extracted; By presetting the object detection model, according to the multiple two-dimensional cabinet key point coordinates of the binocular image of each cabinet, and according to the two-dimensional cabinet key point coordinates and the cabinet depth map, the corresponding three-dimensional cabinet key point coordinates are obtained; According to the cabinet 3D point clouds of each camera coordinate system, the corresponding potential cabinet plane equation is fitted; the potential cabinet plane equation includes a potential left cabinet plane equation and a potential right cabinet plane equation; All potential cabinet plane equations are combined to obtain a multi-plane equation group, and the camera relative to the ground plane extrinsic matrix is ​​obtained by solving the multi-plane equation group; The step of fitting the corresponding potential cabinet plane equation according to the cabinet 3D point cloud of each camera coordinate system includes: Convert the 3D Euclidean space coordinates of the cabinet 3D point cloud in the camera coordinate system into the corresponding Hough space coordinates in the polar coordinate system; Determine the initial parameter range of the plane to be fitted according to a plurality of intersection points of the parameter surfaces corresponding to the Hough space coordinates; According to the prior information of the cabinet plane, the initial parameter range of the plane to be fitted is screened to obtain candidate points of the plane to be fitted; Performing Hough space voting on the candidate points of the plane to be fitted, and extracting the cabinet plane point cloud area with a voting number greater than a preset threshold; According to the cabinet plane point cloud area, a corresponding potential cabinet plane equation is obtained by fitting; The step of obtaining the camera relative to the ground plane external parameter matrix by solving the multi-plane equation group comprises: According to the multi-plane equation group, a corresponding minimization error equation group is constructed, and the solution problem of the minimization error equation group is converted into a least squares problem of the corresponding linear equation group; Performing SVD decomposition on the cross covariance matrix corresponding to the linear equation system to obtain a ground plane rotation matrix of the camera coordinate system, and performing angular axis decomposition on the ground plane rotation matrix to obtain a ground plane normal vector; Obtaining the height of the camera coordinate system relative to the ground plane according to the ground plane normal vector and the coordinates of the key points of the three-dimensional cabinet; According to the Y-axis vector of the camera coordinate system and the ground plane normal vector, a rotation axis and a rotation angle are obtained, and the rotation axis and the rotation angle are converted into a rotation matrix of the camera coordinate system relative to the ground plane; According to the height of the camera coordinate system relative to the ground plane and the rotation matrix of the camera coordinate system relative to the ground plane, the camera relative to the ground plane extrinsic parameter matrix is ​​obtained; the camera relative to the ground plane extrinsic parameter matrix is ​​expressed as: In the formula, = =arccos( ) Where T represents the camera's external parameter matrix relative to the ground plane; R and t represent the camera coordinate system rotation matrix and the camera coordinate system's height relative to the ground plane, respectively; , , and They represent the rotation axis, rotation angle, ground plane equation normal vector and the Y-axis vector of the camera coordinate system respectively.

2. The online calibration method for a computer room camera plane as claimed in claim 1, characterized in that: The step of extracting the corresponding cabinet depth map and the cabinet 3D point cloud of the camera coordinate system according to the binocular images of each cabinet includes: Get the intrinsic parameter matrix and binocular epipolar distance of the binocular camera; According to the binocular epipolar distance, binocular epipolar matching is performed on the left cabinet image and the right cabinet image, and a corresponding cabinet disparity map is generated by the SGBM algorithm; Obtaining a corresponding cabinet depth map according to the cabinet disparity map and the binocular baseline; According to the cabinet depth map and the intrinsic parameter matrix, a 3D point cloud of the cabinet in the camera coordinate system is obtained.

3. The online calibration method for a computer room camera plane as claimed in claim 2, characterized in that: The step of obtaining the cabinet 3D point cloud in the camera coordinate system according to the cabinet depth map and the intrinsic parameter matrix comprises: According to the cabinet depth map and the internal parameter matrix, a 3D point cloud of the cabinet in the initial camera coordinate system is calculated; According to the octree structure, the cabinet 3D point cloud of the initial camera coordinate system is divided into voxels to obtain a sparse 3D data tree with the same time complexity and space complexity; Using a preset discrete value filter, the nodes of the sparse 3D data tree are subjected to sparse outlier removal processing to obtain a dense 3D point cloud of the cabinet; The dense 3D point cloud of the cabinet is processed by voxel downsampling and Gaussian curvature filtering in sequence to obtain the 3D point cloud of the cabinet in the camera coordinate system.

4. The method for online calibration of a camera plane in a computer room as claimed in claim 1, characterized in that: The preset object detection model is a bottom-up openpose key point detection model based on mobilenet as the backbone network, which is pre-trained according to an offline training data set; the offline training data set includes cabinet image data under various environmental backgrounds, various ambient lighting, and various camera perspectives in the computer room; The step of obtaining corresponding three-dimensional cabinet key point coordinates according to a plurality of two-dimensional cabinet key point coordinates of binocular images of each cabinet by using a preset object detection model and according to the two-dimensional cabinet key point coordinates and the cabinet depth map comprises: Inputting the left-eye cabinet image or the right-eye cabinet image into the preset object detection model, and obtaining a plurality of two-dimensional cabinet key point coordinates of the cabinet foot area in the cabinet binocular image; An image coordinate system is established according to the binocular images of each cabinet, and the cabinet depth map is queried according to the coordinates of each two-dimensional cabinet key point to obtain the corresponding key point depth information; The depth information of each key point is converted into the three-dimensional coordinates of the camera coordinate system to obtain the corresponding three-dimensional cabinet key point coordinates.

5. The online calibration method for a camera plane in a computer room as claimed in claim 1, characterized in that: The step of obtaining the height of the camera coordinate system relative to the ground plane according to the ground plane normal vector and the three-dimensional cabinet key point coordinates comprises: Substituting the ground plane normal vector and the coordinates of each key point of the three-dimensional cabinet into the plane equation formula respectively, to obtain the corresponding candidate ground plane equations respectively; The ground plane equation is obtained by averaging the candidate ground plane equations; According to the ground plane equation, the height of the camera coordinate system relative to the ground plane is obtained; the height of the camera coordinate system relative to the ground plane is expressed as: In the formula, Where t represents the height of the camera coordinate system relative to the ground plane; D represents the intercept of the equation from the origin to the ground plane; Represents the normal vector of the ground plane equation The norm of ; Indicates the Z-axis coordinate value of the camera coordinate system relative to the ground plane.

6. A computer room camera plane online calibration method system, characterized in that: The system comprises: An image acquisition module is used to obtain a preset number of cabinet binocular images through a binocular camera; the cabinet binocular images include left cabinet images and right cabinet images; The image processing module is used to extract the corresponding cabinet depth map and the cabinet 3D point cloud of the camera coordinate system according to the binocular image of each cabinet; A key point extraction module, used to obtain corresponding three-dimensional cabinet key point coordinates according to a plurality of two-dimensional cabinet key point coordinates of the binocular image of each cabinet through a preset object detection model, and according to the two-dimensional cabinet key point coordinates and the cabinet depth map; A plane fitting module is used to fit the corresponding potential cabinet plane equation according to the cabinet 3D point cloud of each camera coordinate system; the potential cabinet plane equation includes a potential left cabinet plane equation and a potential right cabinet plane equation; An extrinsic parameter solving module is used to solve all potential cabinet plane equations to obtain a multi-plane equation group, and obtain a camera relative to the ground plane extrinsic parameter matrix by solving the multi-plane equation group; The plane fitting module is specifically used for: Convert the 3D Euclidean space coordinates of the cabinet 3D point cloud in the camera coordinate system into the corresponding Hough space coordinates in the polar coordinate system; Determine the initial parameter range of the plane to be fitted according to a plurality of intersection points of the parameter surfaces corresponding to the Hough space coordinates; According to the prior information of the cabinet plane, the initial parameter range of the plane to be fitted is screened to obtain candidate points of the plane to be fitted; Performing Hough space voting on the candidate points of the plane to be fitted, and extracting the cabinet plane point cloud area with a voting number greater than a preset threshold; According to the cabinet plane point cloud area, a corresponding potential cabinet plane equation is obtained by fitting; External parameter solving module, specifically used for: According to the multi-plane equation group, a corresponding minimization error equation group is constructed, and the solution problem of the minimization error equation group is converted into a least squares problem of the corresponding linear equation group; Performing SVD decomposition on the cross covariance matrix corresponding to the linear equation system to obtain a ground plane rotation matrix of the camera coordinate system, and performing angular axis decomposition on the ground plane rotation matrix to obtain a ground plane normal vector; Obtaining the height of the camera coordinate system relative to the ground plane according to the ground plane normal vector and the coordinates of the key points of the three-dimensional cabinet; According to the Y-axis vector of the camera coordinate system and the ground plane normal vector, a rotation axis and a rotation angle are obtained, and the rotation axis and the rotation angle are converted into a rotation matrix of the camera coordinate system relative to the ground plane; According to the height of the camera coordinate system relative to the ground plane and the rotation matrix of the camera coordinate system relative to the ground plane, the camera relative to the ground plane extrinsic parameter matrix is ​​obtained; the camera relative to the ground plane extrinsic parameter matrix is ​​expressed as: In the formula, = =arccos( ) Where T represents the camera's external parameter matrix relative to the ground plane; R and t represent the camera coordinate system rotation matrix and the camera coordinate system's height relative to the ground plane, respectively; , , and They represent the rotation axis, rotation angle, ground plane equation normal vector and the Y-axis vector of the camera coordinate system respectively.

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images