Object monitoring method based on multi-camera joint calibration technology and monitoring camera thereof

Abstract

The invention relates to the technical field of vision, in particular to an object monitoring method based on a multi-camera joint calibration technology and a monitoring camera thereof. The method comprises the following steps: driving all cameras to synchronously shoot a calibration reference object with known geometric characteristics, and solving internal and external parameters of each camera and an accurate space pose relationship between the internal and external parameters by utilizing shot images to form a joint space relationship chain; controlling the camera array to synchronously shoot a target object, and correspondingly converting the two-dimensional image points acquired by the cameras into spatial point coordinates in a unified three-dimensional coordinate system by using the joint spatial relation chain to generate three-dimensional point cloud data of the target object; and processing the generated three-dimensional point cloud data, extracting key geometric features of the target object in real time, performing comparative analysis on the extracted features and a preset standard or a historical state, and outputting a state monitoring result of the target object. According to the invention, the influence of temperature drift and mechanical vibration on the measurement precision is effectively suppressed, and the stability of long-term monitoring of an industrial field is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of visual technology, in particular to an object monitoring method based on multi-camera joint calibration technology and a monitoring camera thereof. BACKGROUND

[0002] In the fields of industrial detection and intelligent manufacturing, high-precision three-dimensional object monitoring is crucial for product quality control. Traditional single-camera systems are limited by field of view and occlusion problems, making it difficult to cover large-size or complex-structure target objects, resulting in measurement blind spots and reduced precision. Although multi-camera systems can compensate for this deficiency through multi-view observation, joint calibration and data fusion between cameras face major challenges: on the one hand, factors such as mechanical vibration and temperature drift can undermine the stability of calibration parameters; on the other hand, high-precision pose solving algorithms are needed for spatial consistency alignment of multi-source images. In existing technologies, single calibration methods based on calibration boards are difficult to cope with dynamic environmental disturbances, while inertial measurement unit assisted pose tracking has the problem of cumulative error. In addition, traditional monitoring cameras lack hardware-level synchronization and temperature compensation mechanisms, resulting in timing deviations and thermal deformation errors when multiple cameras are used for collaborative acquisition. Therefore, there is an urgent need for a monitoring method that integrates rigid constraints, dynamic calibration compensation, and multi-modal sensing, as well as a dedicated camera hardware, to achieve stable and high-precision three-dimensional object reconstruction and state analysis in complex industrial environments. SUMMARY

[0003] The present application overcomes the shortcomings of the prior art and provides an object monitoring method based on multi-camera joint calibration technology and a monitoring camera thereof.

[0004] To achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows: The first aspect of the present application discloses an object monitoring method based on multi-camera joint calibration technology, comprising the following steps: S102: Fix multiple cameras at a preset relative orientation to form a camera array with a basic spatial constraint relationship, and establish an initial spatial relationship chain; S104: Drive all cameras to synchronously capture a calibration reference object with known geometric features, and use the captured images to solve the internal and external parameters of each camera and their accurate spatial pose relationship with each other, forming a joint spatial relationship chain; S106: In the monitoring state, control the camera array to synchronously capture the target object, and use the joint spatial relationship chain to correspondingly convert the two-dimensional image points captured by each camera into spatial point coordinates in a unified three-dimensional coordinate system, generating three-dimensional point cloud data of the target object; S108: Process the generated three-dimensional point cloud data, real-time extract the key geometric features of the target object, and compare and analyze the extracted features with the preset standard or historical state, output the state monitoring result of the target object; S110: During the system operation, real-time sensing of physical disturbances that may cause changes in the relative pose of the cameras, and triggering the automatic verification and update process of the joint spatial relationship chain according to the sensing information, to maintain the three-dimensional measurement accuracy.

[0005] Preferably, the S102, in particular: Based on the size of the target object and the coverage requirement, the preset camera relative pose parameters are determined, including the three-dimensional position coordinates and the orientation angles of each camera, to define the geometric layout of the basic spatial constraint; Using rigid connection structure, the multiple cameras are physically fixed according to the determined pose parameters to form a camera array, and the rigid frame of the array is ensured to provide the basic spatial constraint relationship, thereby suppressing the random changes of the relative pose; Through the micro inertial measurement unit integrated on the structure, the actual relative pose data between the cameras, including the distance deviation and the angle deviation, are collected in real time as spatial constraint data; According to the spatial constraint data, the relative positions and orientations of each camera are determined through geometric relationship in combination with the preset pose parameters, and an initial spatial relationship chain is established.

[0006] Preferably, according to the spatial constraint data, the relative positions and orientations of each camera are determined through geometric relationship in combination with the preset pose parameters, and an initial spatial relationship chain is established, in particular: The relative pose data collected by the micro inertial measurement unit is filtered and denoised to eliminate high-frequency vibration noise, and stable relative displacement and angle change data between the cameras are extracted; Taking the optical center of the specified master camera in the preset pose parameters as the origin and its optical axis direction as the reference axis, a global coordinate system is established, and the preprocessed relative displacement and angle change data are converted to the coordinate system to obtain the initial position and orientation data of each slave camera; Based on the initial position and orientation data in the global coordinate system, a camera pose optimization model is constructed using the rigid connection constraint between the cameras, and the relative positions and orientation angles of each camera are determined by minimizing the distance deviation between the measured distance and the theoretical distance of the preset pose parameters between adjacent cameras; According to the optimized camera pose data, the residual error of the actual distance between each camera and the preset theoretical distance is calculated, and if the residual error exceeds the deformation threshold of the rigid connection structure, the pose data is adjusted until the residual error does not exceed the deformation threshold of the rigid connection structure; The optimized camera pose data is arranged in spatial topological order with the master camera as the reference, and the three-dimensional offset and orientation angle of each slave camera relative to the master camera are recorded to form an initial spatial relationship chain, wherein the confidence of each node is determined by the optimization residual error.

[0007] Preferably, the S104, in particular: driving all cameras to shoot the calibration reference with a preset geometric pattern at the same time to obtain a raw image set of each camera; performing sub-pixel level corner detection on each image to extract feature point coordinates of the calibration reference, and establishing a mapping relationship of corresponding point pairs across cameras based on spatial distribution consistency of the feature points; inputting preset optical center positions and optical axis orientations of each camera as prior constraints of the initial spatial relationship chain; taking the mapping relationship of corresponding point pairs as observation values, fusing pose constraints of the initial spatial relationship chain to construct a multi-camera joint re-projection error model, and obtaining intrinsic parameters and extrinsic parameters of all cameras to construct an intrinsic parameter matrix, an extrinsic rotation matrix and an extrinsic translation vector; connecting the extrinsic rotation matrix and the extrinsic translation vector according to a spatial topology of the cameras, taking a master camera coordinate system as a reference, and calculating a relative pose transformation matrix between adjacent cameras step by step to form a tree-shaped pose conversion chain; integrating all relative pose transformation matrices and the intrinsic parameter matrix to construct a joint spatial relationship chain containing spatial pose dependency relationships between the cameras.

[0008] Preferably, the S106 specifically comprises: controlling all cameras to shoot a target object at the same time to obtain a raw image set, performing denoising and edge enhancement processing on each image, and extracting two-dimensional image feature points with sub-pixel level precision; correcting the extracted two-dimensional image feature points for lens distortion by using the intrinsic parameter matrix stored in the joint spatial relationship chain to generate a set of de-distorted image points, and mapping the set of de-distorted image points of each camera to a virtual imaging plane under the master camera coordinate system based on the tree-shaped pose conversion chain of the joint spatial relationship chain; on the virtual imaging plane, performing bidirectional matching on image feature points in overlapping regions according to epipolar constraint relationships between adjacent cameras to establish corresponding point pairs across cameras, and obtaining initial spatial coordinates of the corresponding point pairs in three-dimensional space; performing weighted optimization on the initial spatial coordinates according to pose confidence of each node in the joint spatial relationship chain: projecting multiple sets of initial spatial coordinates corresponding to the corresponding point pairs to the master camera coordinate system, assigning weights according to confidence of the camera nodes participating in the calculation, fusing and generating optimized spatial point coordinates by weighted least squares method, and eliminating outlying point coordinates caused by confidence below a threshold; mapping the optimized spatial point coordinates of all successfully matched corresponding point pairs to a unified world coordinate system in a spatial topological order, integrating all spatial point coordinates and adding identification information of the source cameras to form a dense three-dimensional point cloud data set with consistent spatial topology.

[0009] Preferably, the S108 specifically comprises: A spatial topology index is applied to input three-dimensional point cloud data, and outlier noise points are removed through voxel grid filtering; meanwhile, the spatial topology index is used to reconstruct the neighborhood relationship of the point cloud and generate a surface mesh structure with connectivity; On the reconstructed surface mesh, the spatial principal direction axis of the target object is analyzed, and the spatial principal direction axis is set as a geometric feature calculation reference; and equidistant cross sections are divided along the spatial principal direction axis to generate a series of orthogonal projection planes; The cross section contour boundary points are fitted in the projection plane to obtain the envelope rectangular size, center coordinate and contour curvature entropy of each cross section, forming a size-position- appearance feature vector; Based on the surface mesh normal vector field, the multi-scale surface gradient features of each vertex are calculated to generate a surface topography change map; The size-position-appearance feature vector of the current frame is matched with the feature vector of the preset standard model in the Euclidean distance, and the size deviation and position offset are output; The multi-scale surface gradient features are input into the pre-trained defect classification model to identify the local abnormal area through convolution kernel matching; The size deviation, position offset and surface defect recognition result are integrated, and a weighted confidence fusion is adopted to obtain a comprehensive state evaluation index of the target object; when the comprehensive state evaluation index exceeds the preset index threshold, a state abnormality alarm is triggered.

[0010] Preferably, the S110, in particular: The six-degree-of-freedom vibration acceleration and angular velocity data of the camera array are obtained in real time, and the local temperature rise gradient of each camera node is monitored to form a spatiotemporally synchronized physical disturbance feature vector; The physical disturbance feature vector is input into the pre-calibrated camera structure deformation transfer model to determine the relative pose offset prediction value of each camera node caused by the temperature gradient and mechanical vibration, and generate a pose drift feature matrix; Based on the offset prediction value of each node in the pose drift feature matrix, and combined with the original pose confidence in the joint spatial relationship chain, the dynamic reliability index of the current spatial relationship chain is determined; When the reliability index is lower than the preset index threshold, the camera array is controlled to synchronously capture a preset fixed orientation calibration reference object in the scene, and the known three-dimensional geometric constraint of the calibration reference object is used to inversely calculate the actual pose change amount through the multi-view imaging difference, and generate a measured pose compensation vector; The measured pose compensation vector and the prediction value of the pose drift feature matrix are fused by Kalman gain weighting to obtain an optimized pose correction amount, and the external parameter rotation matrix and the external parameter translation vector in the joint spatial relationship chain are updated node by node in the topological order of the tree-like pose conversion chain, and the node pose confidence weight is adjusted; The updated joint space relation chain is applied to the real-time acquisition of the target object point cloud reconstruction, the root mean square variation rate of the point cloud registration error between adjacent frames is counted, if the root mean square variation rate does not converge to a preset variation rate threshold, iteration is continued until the root mean square variation rate converges to the preset variation rate threshold.

[0011] Preferably, the method for constructing the pre-calibrated camera structure deformation transfer model comprises the following steps: Based on the material properties and geometric layout of the rigid connection structure of the camera array, a theoretical deformation response parameter matrix of the structure is established to represent the linear and nonlinear deformation effects of temperature change and mechanical vibration on the structure; During the calibration stage, a controllable temperature gradient change and mechanical vibration excitation are applied to the camera array, and six-degree-of-freedom pose offset data output by the micro inertial measurement unit are recorded synchronously to form an excitation-response sample set, wherein the excitation data includes temperature change rate data and vibration frequency spectrum data, and the response data includes measured displacement data and angle deviation data; Using the excitation-response sample set, a mapping relationship between the temperature gradient and the vibration frequency spectrum and the camera pose offset is established by least squares fitting to generate an initial deformation transfer coefficient matrix, and a recursive optimization method is used to correct the nonlinear error items in the theoretical deformation response parameter matrix to obtain the deformation transfer model; Based on the vibration frequency spectrum data, the dynamic response characteristics of the structure under different vibration frequencies are analyzed, the steady-state offset and the transient vibration component are separated, and corresponding compensation sub-models are established respectively; The compensation sub-models and the deformation transfer model are fused to obtain a camera structure deformation transfer model for real-time prediction of the pose offset caused by physical disturbance.

[0012] The second aspect of the application discloses a monitoring camera applied to any of the object monitoring methods based on the multi-camera joint calibration technology, comprising: A rigid connection base unit is made of a low thermal expansion coefficient alloy material, and is provided with a standardized mechanical interface and a locking mechanism for forming a rigid topological connection with adjacent cameras or support structures; A multi-axis attitude adjustment module is integrated in the base and includes a three-degree-of-freedom micro-displacement platform and a two-axis pitch-yaw adjustment mechanism; An embedded inertial measurement unit is fixed at the optical center axis of the camera and includes a three-axis MEMS accelerometer and a fiber optic gyroscope with a sampling frequency of ≥1kHz, which outputs six-degree-of-freedom pose data in real time; A temperature control compensation module includes a distributed temperature sensor array attached to the camera housing and a semiconductor refrigeration plate; A synchronous trigger interface supports hardware-level external trigger signal input and cascaded output; Optical imaging module, including global shutter CMOS sensor, telecentric lens and tunable polarization filter, working waveband covers 400-1000nm.

[0013] The present application solves the technical defects in the background art, and has the following beneficial effects: the present application first establishes a stable multi-view measurement reference through rigid array layout and joint calibration, solving the problem of insufficient single-camera field of view; secondly, it ensures the consistency of multi-source image data and improves the integrity of complex surface reconstruction through synchronous acquisition and unified spatial coordinate conversion; thirdly, it realizes the automated detection of key parameters such as size and appearance through real-time feature extraction and standard comparison; finally, it effectively suppresses the influence of temperature drift and mechanical vibration on measurement accuracy through environmental disturbance perception and adaptive calibration mechanism, ensuring the stability of long-term monitoring in industrial field. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings of embodiments according to these drawings without creative labor.

[0015] Figure 1 The present object monitoring method is a whole method flowchart; Figure 2 The present object monitoring method is a part of the method flowchart; Figure 3 The present monitoring camera is a module schematic diagram. DETAILED DESCRIPTION

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings of embodiments according to these drawings without creative labor.

[0017] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, however, the present application can also be implemented in other ways different from those described herein, therefore, the scope of protection of the present application is not limited by the specific embodiments disclosed below.

[0018] As Figure 1 shown, the first aspect of the present application discloses an object monitoring method based on multi-camera joint calibration technology, comprising the following steps: S102: fixing a plurality of cameras at a preset relative orientation to form a camera array with basic spatial constraint relationship, and establishing an initial spatial relationship chain; S104: Drive all cameras to synchronously capture the calibration reference of known geometric features, and solve the internal and external parameters of each camera and the accurate spatial pose relationship between them by using the captured images to form a joint spatial relationship chain; S106: In the monitoring state, control the camera array to synchronously capture the target object, and convert the two-dimensional image points captured by each camera into spatial point coordinates in a unified three-dimensional coordinate system by using the joint spatial relationship chain to generate three-dimensional point cloud data of the target object; S108: Process the generated three-dimensional point cloud data, real-time extract the key geometric features (such as size, shape, position, and surface topography) of the target object, and compare and analyze the extracted features with the preset standard or historical state to output the state monitoring result (such as qualification judgment, defect identification, and position deviation) of the target object; S110: In the process of system operation, real-time perceive the physical disturbance (such as temperature drift and mechanical vibration) that may cause the relative pose change of the camera, and trigger the automatic verification and update process of the joint spatial relationship chain according to the perceived information to maintain the three-dimensional measurement accuracy.

[0019] It should be noted that the present application realizes high-precision three-dimensional reconstruction and online detection of large-size industrial parts through multi-camera joint calibration and dynamic calibration technology. Specifically, firstly, a stable multi-view measurement reference is established through rigid array layout and joint calibration, solving the problem of insufficient field of view of a single camera. Secondly, the consistency of multi-source image data is ensured by synchronous acquisition and spatial coordinate unified conversion, improving the integrity of complex surface reconstruction. Thirdly, real-time feature extraction and standard comparison are realized to realize automatic detection of key parameters such as size and topography. Finally, with the help of environmental disturbance perception and adaptive calibration mechanism, the influence of temperature drift and mechanical vibration on measurement accuracy is effectively suppressed, and the stability of long-term monitoring in industrial field is ensured.

[0020] Preferably, the S102 is as shown in the specific embodiments. Figure 2 The specific embodiments are as follows: S202: Based on the size and coverage requirement of the target object, determine the preset relative pose parameters of the cameras, including the three-dimensional position coordinates and orientation angles of each camera, to define the geometric layout of the basic spatial constraint; S204: Use a rigid connection structure to physically fix multiple cameras according to the determined relative pose parameters to form a camera array, and ensure that the rigid frame of the array provides the basic spatial constraint relationship to suppress the random change of the relative pose; S206: Real-time collect the actual relative pose data between the cameras, including distance deviation and angle offset, by using the micro inertial measurement unit integrated on the structure, as spatial constraint data; S208: According to the spatial constraint data, combined with the preset orientation parameters, the relative positions and orientations of each camera are determined through geometric relationship to establish an initial spatial relationship chain.

[0021] According to the spatial constraint data, combined with the preset orientation parameters, the relative positions and orientations of each camera are determined through geometric relationship to establish an initial spatial relationship chain, specifically: The relative pose data collected by the miniature inertial measurement unit is filtered and denoised to eliminate high-frequency vibration noise and extract stable relative displacement and angle change data between cameras. The optical center of the specified master camera in the preset orientation parameters is taken as the origin, and the optical axis direction is taken as the reference axis to establish a global coordinate system. The preprocessed relative displacement and angle change data are converted to the coordinate system to obtain the initial position and orientation data of each slave camera. Based on the initial position and orientation data in the global coordinate system, a camera pose optimization model is constructed using the rigid connection constraint between cameras. The relative positions and orientation angles of each camera are determined by minimizing the deviation between the measured distance between adjacent cameras and the theoretical distance of the preset orientation parameters. According to the optimized camera pose data, the residual error between the actual distance and the preset theoretical distance between each camera is calculated. If the residual error exceeds the deformation threshold of the rigid connection structure, the pose data is adjusted until the residual error does not exceed the deformation threshold of the rigid connection structure. The optimized camera pose data is arranged in spatial topological order, taking the master camera as the reference, and recording the three-dimensional offset and orientation angle of each slave camera relative to the master camera to form an initial spatial relationship chain, where the confidence level of each node is determined by the optimization residual error.

[0022] It should be noted that the camera relative displacement and angle data collected by the inertial measurement unit are filtered and denoised to eliminate high-frequency vibration interference, and then the optical center of the master camera (such as camera 1) is taken as the origin of the coordinate system, and the optical axis direction is taken as the Z axis to establish a global coordinate system. Then the measured pose data of each slave camera is converted to the global coordinate system, combined with the preset camera distance (such as 1.2m) and angle parameters (such as 45°) to determine the actual three-dimensional position and orientation angle of each camera. Finally, taking the master camera as the reference node, the three-dimensional offset and Euler angle of each camera relative to the previous camera are recorded in the topological order of "master camera-2 camera-3 camera-4 camera" to form an initial spatial relationship chain with confidence rating. When the measured distance between adjacent cameras exceeds the residual error of the theoretical value (such as 0.2mm), the pose data iterative optimization is automatically triggered until the accuracy requirement is met.

[0023] For example, in the automobile body-in-white welding quality detection scene, for the three-dimensional topography measurement requirement of a door frame with a length of 5 m, the embodiment adopts four monitoring cameras to build a camera array: based on the size of the door frame, the preset camera layout is linear arrangement, the distance between adjacent cameras is 1.2 m, the optical axis forms a 45° angle with the normal direction of the door frame, and the pitch angle of each camera is uniformly set to 10°; the truss structure made of steel is rigidly connected to the cameras, and the relative pose between the cameras is monitored in real time through the MEMS inertial measurement unit integrated at the connection node; when the distance deviation between adjacent cameras exceeds 0.1 mm or the angle deviation exceeds 0.3°, the online correction of the initial spatial relationship chain is triggered to ensure that the basic spatial constraint error of the camera array is controlled within ±0.15 mm under the condition that the workshop environment temperature fluctuates ±5℃.

[0024] Preferably, the S104, in particular: Drive all cameras to take pictures of the calibration reference with a preset geometric pattern (such as a combination of a checkerboard array and an ArUco marker) at the same time, and obtain a set of original images of each camera; Perform sub-pixel level corner detection on each image, extract feature point coordinates of the calibration reference, and establish a mapping relationship between corresponding points across cameras based on the consistency of the spatial distribution of feature points; Input the preset optical center position and optical axis orientation of each camera as prior constraints based on the initial spatial relationship chain; fuse the pose constraints of the initial spatial relationship chain with the mapping relationship between corresponding points as observation values, and construct a multi-camera joint re-projection error model; and obtain the intrinsic and extrinsic parameters of all cameras, and construct an intrinsic parameter matrix, an extrinsic rotation matrix, and an extrinsic translation vector; It should be noted that the initial spatial relationship chain data generated in the S102 step is read, including the preset optical center coordinates and optical axis orientation angles of each camera, and then these initial pose parameters are input as initial values. At the same time, based on the corresponding relationship between the pixel coordinates of the same feature points extracted from the calibration board images of all cameras, a joint optimization model is constructed: the pose parameters provided by the initial spatial relationship chain are used as constraint conditions, and the pixel coordinate matching relationship of the same points is used as observation data to establish a re-projection error function containing all camera intrinsic and extrinsic parameters. Finally, the nonlinear least squares method is used for synchronous optimization and solution to minimize the total re-projection error of all cameras under the constraint of the initial spatial relationship chain, so as to obtain accurate camera intrinsic and extrinsic parameters.

[0025] It should be noted that by acquiring the intrinsic parameters (including focal length, principal point coordinates and lens distortion coefficients) and extrinsic parameters (including rotation matrix and translation vector between cameras) of all cameras, an intrinsic matrix, an extrinsic rotation matrix and an extrinsic translation vector are constructed. Specifically, by extracting the intrinsic data of each camera, including focal length, principal point coordinates and distortion coefficients, a 3x3 intrinsic matrix is constructed, and then according to the pose relationship between cameras, the extrinsic rotation matrix R (3x3 matrix, representing the camera orientation) and the extrinsic translation vector (3x1 vector, representing the camera position) of each camera relative to the master camera coordinate system are determined. Finally, these parameters are stored in order of camera number, wherein the rotation matrix of the master camera is set as a 3x3 unit matrix, and the translation vector is set as a 3x1 zero vector, and the extrinsic parameters of the slave cameras are expressed as the transformation relationship relative to the master camera, completing the systematic construction of all camera parameters.

[0026] The extrinsic rotation matrix and the extrinsic translation vector are connected according to the camera space topology, taking the master camera coordinate system as the reference, and the relative pose transformation matrix between adjacent cameras is calculated step by step to form a tree-like pose conversion chain. It should be noted that the master camera coordinate system is selected as the global reference coordinate system, the extrinsic rotation matrix of the master camera is set as a 3x3 unit matrix, and the translation vector is set as a 3x1 zero vector, and then the relative pose transformation between adjacent cameras is determined according to the physical connection order of the camera array (such as master camera→2nd camera→3rd camera→4th camera). For each slave camera, the extrinsic rotation matrix and translation vector are used to perform matrix multiplication operation with the extrinsic parameters of the previous camera to obtain the cumulative transformation matrix of the camera relative to the master camera. Finally, all camera transformation matrices are stored in a tree structure to form a pose conversion chain starting from the master camera node and branching out step by step, wherein each node contains the complete pose information (rotation + translation) of the camera relative to the master camera and its conversion relationship with adjacent cameras.

[0027] All relative pose transformation matrices and intrinsic matrices are integrated to construct a joint spatial relationship chain containing the spatial pose dependency between cameras, wherein the pose confidence of each node is represented by the inverse of the covariance matrix of the re-projection error.

[0028] It should be noted that the intrinsic matrix, the extrinsic rotation matrix and the translation vector of each camera are associated and integrated according to the topological structure of the tree-like pose conversion chain, and the re-projection error of each camera node in the calibration process is obtained, and the corresponding covariance matrix is generated by statistically distributing the pixel-level error of all feature points. Then, the inverse matrix of the covariance matrix is taken as the pose confidence weight of the camera node, and the confidence of the main camera node is set to the highest level. Finally, a complete joint spatial relationship chain data structure is constructed, which not only contains the spatial transformation relationship between the cameras, but also records the confidence weight of each node, which is used for error weighted optimization in subsequent three-dimensional reconstruction. When the re-projection error of a certain camera node is large, the inverse value of its covariance matrix will automatically decrease, and the weight of the node in the spatial relationship chain will also decrease.

[0029] For the aforementioned 4 rigidly connected camera array, the embodiment performs joint calibration: drive all cameras to synchronously shoot a composite calibration board (such as containing a 20x15 checkerboard array and an ArUco marker, with a square size of 10mm±0.005mm) placed at the center of the door frame. Extract feature points from each image by sub-pixel corner detection, and establish a mapping relationship between 128 same-named point pairs across cameras. Take the initial spatial relationship chain generated in S102 as a prior constraint (the confidence of the main camera position is ±0.1mm), input the preset optical center coordinates and optical axis orientation (pitch angle 10°±0.2°), and construct a re-projection error model that integrates pose constraints. Synchronously solve the intrinsic matrix (such as focal length f=8mm±0.02mm, distortion coefficient k1=-0.15±0.01) and the extrinsic matrix (translation vector t=[1200.2, -3.5, 85.6] mm), and determine the relative pose transformation matrix (such as the rotation matrix Ry=44.8° of the No.2 camera relative to the main camera) according to the topology of “main camera→No.2→No.3→No.4”, forming a tree-like conversion chain. Finally, a joint spatial relationship chain is generated, and the node pose confidence is represented by the inverse of the re-projection error covariance matrix (the main node error is 0.15 pixels), which provides precision guarantee for subsequent three-dimensional reconstruction.

[0030] Preferably, the S106, in particular: Control all cameras to simultaneously shoot target objects and obtain a set of original images; perform denoising and edge enhancement processing on each image to extract two-dimensional image feature points (such as corner points and edge intersection points) with sub-pixel level precision; Use the intrinsic matrix (including distortion coefficients) stored in the joint spatial relationship chain to correct the extracted two-dimensional image feature points for lens distortion, generating a set of de-distorted image points; map the de-distorted image point sets of each camera to a virtual imaging plane in the main camera coordinate system based on the tree-like pose conversion chain of the joint spatial relationship chain; It should be noted that the intrinsic matrix and lens distortion coefficients of each camera stored in the joint spatial relationship chain are called to perform reverse distortion calculation on the extracted two-dimensional image feature point coordinates, eliminate the image deformation caused by the optical characteristics of the lens, and obtain accurate de-distortion feature point coordinates. Then, according to the spatial relationship between the cameras recorded in the tree-like pose conversion chain, the corrected feature point coordinates of each subordinate camera are converted to the coordinate system of the main camera through coordinate transformation, specifically, the feature point coordinates of the subordinate camera are first converted to the world coordinate system through its extrinsic matrix, and then converted to the main camera coordinate system through the inverse matrix of the extrinsic matrix of the main camera, and finally a unified virtual imaging plane is formed in the main camera coordinate system, so that the feature points of all cameras can be matched and calculated in the same spatial reference system.

[0031] On the virtual imaging plane, according to the epipolar constraint relationship between adjacent cameras, the image feature points in the overlapping area are bidirectionally matched, the homonymous point pairs across the cameras are established, and the initial spatial coordinates of the homonymous point pairs in the three-dimensional space are obtained. It should be noted that the spatial geometric constraint conditions between any two adjacent cameras are determined according to the relative position and orientation relationship between the cameras recorded in the tree-like pose conversion chain. For the two cameras that need to be matched, the rotation and translation parameters of each camera relative to the main camera are obtained by querying the conversion chain, and then the relative rotation angle and translation distance between the two cameras are derived. Based on these relative pose parameters, the geometric correspondence between the imaging planes of the two cameras is established. This relationship is that when the position of a feature point in one camera image is determined, the corresponding matching point in the other camera image must fall on a specific straight line. This straight line determined by the relative pose parameters is the epipolar constraint relationship, which limits the search range during feature matching, and can effectively improve the accuracy and efficiency of cross-camera feature matching. In actual application, the system will search and match the feature points in the target camera image along the predicted epipolar direction according to the calculated epipolar constraint relationship.

[0032] According to the pose confidence of each node in the joint spatial relationship chain, the initial spatial coordinates are weighted and optimized: project the multiple sets of initial spatial coordinates corresponding to the homonymous point pairs to the main camera coordinate system; according to the confidence of the camera nodes participating in the calculation, assign weights and generate optimized spatial point coordinates by weighted least squares method; eliminate outlying point coordinates caused by confidence below the threshold; Map the optimized spatial point coordinates of all matched homonymous point pairs to the unified world coordinate system in the spatial topological order; integrate all spatial point coordinates and add the identification information of the source camera to form a dense three-dimensional point cloud dataset with consistent spatial topology.

[0033] For the aforementioned calibrated 4-camera array (joint spatial relationship chain reprojection error 0.15 pixels), this embodiment performs three-dimensional reconstruction: control all cameras to take pictures of the door frame synchronously, perform Gaussian filtering and SUSAN edge enhancement processing on the obtained original images, and extract sub-pixel level corner points. The internal parameter matrix in the joint spatial relationship chain is used to correct the lens distortion to generate a deformed point set; based on the tree-like pose conversion chain (main camera → No. 2 → No. 3 → No. 4), the point sets of each camera are mapped to the virtual imaging plane of the main camera; in the overlapping area, bidirectional matching is performed according to the epipolar constraint (search radius 5 pixels) to establish 320 pairs of homonymous points and calculate the initial spatial coordinates (such as the initial coordinates of point P [1200.5, 45.2, -30.8] mm); the initial coordinates are weighted and optimized (weighted least squares fusion, outlier threshold 0.3 mm) according to the node confidence (main camera weight 1.0, No. 4 camera weight 0.92), and finally a dense three-dimensional point cloud is generated, which is mapped to the vehicle body coordinate system and output as a spatial topology data set with camera ID.

[0034] Preferably, the S108, in particular: The input three-dimensional point cloud data is applied to a spatial topology index, and the voxel grid filtering is used to remove outlier noise points; meanwhile, the spatial topology index is used to reconstruct the point cloud neighborhood relationship to generate a surface mesh structure with connectivity; The spatial topology index is a spatial data structure used to organize three-dimensional point cloud data, which realizes fast query and analysis by establishing the geometric neighborhood relationship between point clouds. In the present application, the determination method of the spatial topology index is as follows: first, the input three-dimensional point cloud is spatially grid divided (such as using octree or KD-Tree structure), and the initial grid resolution (for example, 2 mm cubic voxel) is set. Then, the density distribution characteristics of the point cloud in each voxel unit are calculated, and the high-density area with a density value greater than a preset value (for example, greater than 150 points / mm³) is adaptively subdivided until the preset accuracy requirement (for example, the number of points in a single voxel is not more than 50) is met; meanwhile, the topological connection relationship between points is established, and the k-nearest neighbor information (usually k=10-30) of each point is recorded to form a spatial index structure with multi-level resolution characteristics. The index not only contains the spatial coordinate information of the points, but also stores the connection relationship between the points.

[0035] It should be noted that the voxel grid filtering (voxel size 0.5 mm³) is used to remove outlier noise points, and the spatial topology index (KD-Tree radius search 2 mm) is used to reconstruct the surface mesh to generate a connected structure containing 120,000 triangular facets.

[0036] On the reconstructed surface mesh, analyze the spatial principal direction axis of the target object, and set the spatial principal direction axis as the geometric feature calculation reference; and divide equidistant cross sections along the spatial principal direction axis to generate a series of orthogonal projection planes; The spatial principal direction axis refers to the main direction axis line representing the overall geometric distribution trend of the target object in the three-dimensional point cloud data of the object, corresponding to the length, width or height direction of the object.

[0037] It should be noted that, taking the grid principal direction axis (length direction) as the reference, 50 orthogonal cross sections are generated at an interval of 2 mm.

[0038] Fitting the cross section contour boundary points in the projection plane, obtaining the envelope rectangular size, center coordinates and contour curvature entropy of each cross section, and forming the size-position-landscape feature vector; Based on the surface mesh normal vector field, the multi-scale surface gradient features (combining Gaussian curvature and average curvature gradient) of each vertex are calculated to generate a surface landscape change map; It should be noted that, based on the reconstructed surface mesh structure, the unit normal vector of each triangular facet is determined, and the smooth normal vector field of each vertex is obtained by weighted average of the normal vectors of the adjacent facets of the vertex. Then the Gaussian curvature and the average curvature are calculated in the neighborhood of the vertex, and the curvature gradient value is obtained by difference operation. The curvature gradient features at different scales are normalized and fused to generate the composite deformation feature value of the vertex. Finally, these feature values are mapped to the grid surface, and a visual surface landscape change map is generated using the heat map coding method (such as blue to red gradient), wherein the high gradient area (red) indicates abnormal deformation such as surface depression or protrusion, and the low gradient area (blue) represents flat surface.

[0039] The size-position-landscape feature vector of the current frame is matched with the feature vector of the preset standard model in Euclidean distance, and the size deviation and position offset are output; wherein the preset standard model refers to a digital reference template established by collecting three-dimensional point cloud data of historical qualified samples, containing specification parameters such as size tolerance and landscape feature (such as door frame length 500.0 mm ± 0.1 mm, curvature entropy threshold 0.15), which is used for deviation comparison in real-time detection.

[0040] The multi-scale surface gradient features are input into a pre-trained defect classification model (model parameters derived from historical defect-free samples), and the local abnormal area is identified by convolution kernel matching, including surface scratches and pits; The size deviation, position offset and surface defect recognition result are integrated, and weighted confidence fusion (weight derived from camera pose confidence) is used to obtain the comprehensive state evaluation index of the target object; when the comprehensive state evaluation index exceeds the preset index threshold, a state abnormal alarm is triggered.

[0041] It should be noted that the size deviation (such as the door frame length error 0.2mm), the position offset (such as the welding point center offset 0.3mm) and the surface defect identification result (such as the pit confidence 92%) are normalized to convert into the standardized numerical value in the range of 0-1. Then, according to the pose confidence of each camera in the joint space relationship chain (such as the main camera weight 1.0 and the No. 4 camera weight 0.92), the weight coefficient of each detection item is distributed, wherein the size deviation weight is set to 0.6 (because it depends on the accuracy of the global coordinate system), the position offset weight is 0.3, and the defect result weight is 0.1. Then, the comprehensive state evaluation index (such as 0.6 x size deviation + 0.3 x position offset + 0.1 x defect score) is calculated by weighted summation, when the value exceeds the preset threshold (such as 0.85), the alarm is triggered, and the invalid item (such as "size out-of-tolerance ratio 70%") is marked, and finally the detection report with weighted analysis is output.

[0042] Preferably, the S110, in particular: The six-degree-of-freedom vibration acceleration and angular velocity data of the camera array are acquired in real time, and the local temperature rise gradient of each camera node is monitored to form a time-space synchronous physical disturbance feature vector; The physical disturbance feature vector is input into a pre-calibrated camera structure deformation transfer model to determine the relative pose offset prediction value of each camera node caused by temperature gradient and mechanical vibration, and a pose drift feature matrix is generated; Based on the offset prediction value of each node in the pose drift feature matrix, combined with the original pose confidence (represented by the inverse of the re-projection error covariance matrix) in the joint space relationship chain, the dynamic reliability index of the current space relationship chain is determined; It should be noted that the predicted offset of each camera node (such as 0.12mm for No. 2 camera X-axis) is extracted from the pose drift feature matrix, and the pose confidence weight of the corresponding node (such as the main camera confidence 1.0 and the No. 2 camera 0.92) is obtained from the joint space relationship chain, then the reliability component of each node is determined, that is, the predicted offset is weighted and normalized by the confidence weight (such as the No. 2 camera component = 0.12mm x 0.92); then the reliability components of all nodes are weighted and averaged, wherein the weight proportion of the main camera node is the highest (such as 50%), and finally the calculation result is mapped to the index range of 0-1, when the weighted average offset exceeds the preset safety threshold (such as 0.15mm), the dynamic reliability index is reduced in proportion (such as from 0.9 to 0.85), which reflects the measurement confidence of the current space relationship chain in real time.

[0043] When the reliability index is lower than the preset index threshold, the control camera array synchronously captures the preset fixed orientation calibration reference object (such as a passive optical coding mark installed on the monitoring tool) in the scene, uses the known three-dimensional geometric constraint of the calibration reference object, inverses the actual pose change amount through multi-view imaging difference, and generates a measured pose compensation vector; It should be noted that when the system detects that the dynamic reliability index is lower than the preset threshold (such as 0.9), a hardware synchronization trigger signal is immediately sent to all cameras to control the array to simultaneously capture the optical coding mark (mark point spacing 10 mm ± 0.002 mm) fixed on the detection tool. Through the mark images collected by each camera, the sub-pixel level precision mark center coordinates are extracted, and the known three-dimensional space position of the mark is combined to determine the actual space position change amount of each camera relative to the mark; the pose change data of each camera is compared with the calibration initial pose to generate a measured pose compensation vector (such as the compensation vector of camera No. 2 is [0.15 mm, 0.02 mm, -0.01 mm, 0.3°, 0.1°, 0.05°]) containing translation and rotation components, which is used for subsequent space relationship chain correction.

[0044] The measured pose compensation vector and the predicted value of the pose drift feature matrix are Kalman gain weighted and fused to obtain an optimized pose correction amount, and the external parameter rotation matrix and the external parameter translation vector in the joint space relationship chain are updated node by node in the topological order of the tree-shaped pose conversion chain, and the node pose confidence weight is adjusted; It should be noted that the Kalman gain coefficient is dynamically set according to the historical measurement accuracy of each camera node (such as the gain of the main camera is 0.8, and the gain of the slave camera is 0.6-0.7), the measured pose compensation vector (such as the measured offset of camera No. 2 is [0.15 mm, 0.02 mm, -0.01 mm]) and the predicted value of the drift feature matrix (such as [0.12 mm, 0.03 mm, -0.02 mm]) are weighted and fused, and then the external parameter matrix of each node is updated in the topological order of “main camera→No. 2→No. 3→No. 4”: for the main camera node, the fused compensation amount is directly used to update the translation vector and the rotation matrix. For the slave camera node, it is necessary to recursively combine the relative pose relationship with the upper node, and finally the pose confidence weight is proportionally improved (such as from 0.92 to 0.94) according to the compensation adjustment amplitude (such as the X-axis compensation of camera No. 2 is 0.13 mm), and the global confidence index of the entire space relationship chain is re-determined to complete the closed-loop correction.

[0045] The updated joint space relationship chain is applied to the real-time acquisition of the target object point cloud reconstruction, the root mean square change rate of the point cloud registration error between adjacent frames is calculated, and if the root mean square change rate does not converge to the preset change rate threshold, iteration is continued until the root mean square change rate converges to the preset change rate threshold.

[0046] For example, for a rigid array of 4 cameras (steel truss connection), by monitoring the vibration and temperature rise in real time, when the predicted pose offset of camera No. 2 exceeds the threshold (X axis 0.12mm), the synchronous calibration process is triggered: control the array to shoot the optical encoding mark (spacing 10mm±0.002mm) on the tooling, and after Kalman fusion (gain 0.7) of the measured pose offset (X axis 0.15mm) and the model predicted value, update the joint space relationship chain (No. 2 camera external parameter translation correction [1200.17, -3.48, 85.63]^T mm), so that the point cloud registration error is reduced from 0.25mm to 0.08mm, to ensure that the detection accuracy requirement (threshold 0.1mm) is met.

[0047] Preferably, the construction method step of the pre-calibrated camera structure deformation transfer model is specifically: Based on the material properties (such as thermal expansion coefficient, elastic modulus) and geometric layout (such as camera spacing, connecting rod length) of the rigid connection structure of the camera array, a theoretical deformation response parameter matrix of the structure is established to represent the linear and nonlinear deformation effects of temperature change and mechanical vibration on the structure; During the calibration phase, apply controllable temperature gradient changes and mechanical vibration excitations to the camera array, and simultaneously record the six-degree-of-freedom pose offset data output by the micro inertial measurement unit to form an excitation-response sample set, wherein the excitation data includes temperature change rate data and vibration frequency spectrum data, and the response data includes measured displacement data and angle deviation data; Using the excitation-response sample set, the mapping relationship between temperature gradient and vibration frequency spectrum to camera pose offset is established by least squares fitting to generate an initial deformation transfer coefficient matrix, and the nonlinear error term in the theoretical deformation response parameter matrix is corrected by using a recursive optimization method to obtain a deformation transfer model; Based on the vibration frequency spectrum data, analyze the dynamic response characteristics of the structure under different vibration frequencies, separate the steady-state offset and transient vibration components, and establish corresponding compensation sub-models respectively; Feature fusion is performed on the compensation sub-model and the deformation transfer model to obtain a camera structure deformation transfer model for real-time prediction of the pose offset caused by physical disturbance.

[0048] It should be noted that firstly the characteristic parameters of the steady-state deformation compensation sub-model (established based on the temperature gradient and the material thermal expansion coefficient) and the transient vibration compensation sub-model (established based on the vibration frequency spectrum and the structure inherent frequency) are normalized. Then the output results of the two sub-models are integrated, wherein the steady-state compensation weight is set to 0.7 (mainly reflecting the long-term temperature drift effect), and the transient compensation weight is set to 0.3 (mainly inhibiting high-frequency vibration interference), and then the fused parameter matrix is input into the deformation transmission main model to establish a complete prediction model containing temperature-vibration coupling effect. Finally, by inputting the current monitored temperature change rate and vibration frequency spectrum data in real time, the six-degree-of-freedom pose offset prediction values of each camera node (such as X-axis displacement ±0.15mm and Yaw angle ±0.1°) are output.

[0049] In the embodiment, the object monitoring method further comprises the following steps: The original acceleration and angular velocity data of the inertial measurement unit are synchronously collected at the camera exposure moment, the instantaneous pose is solved through quaternion complementary filtering, and the surface curvature feature point set of the moving object is extracted from the three-dimensional point cloud; The filtered pose data is decomposed into linear velocity vector and angular velocity vector according to rigid body kinematics, and the local surface motion deformation gradient tensor is determined based on the k-nearest neighbor topological relationship of the point cloud feature point set; The linear velocity vector is projected onto the principal direction axis of the deformation gradient tensor to generate a velocity-deformation coupling vector field; and the outer product operation of the angular velocity vector and the surface normal vector field is used to construct a rotation-curvature constraint matrix; The original point cloud is compensated for rigid-flexible mixed motion with the coupling vector field as the displacement constraint and the rotation-curvature matrix as the direction constraint, to generate a motion normalized feature point cloud; The iterative closest point (ICP) matching residual error of the motion normalized point cloud and the actual point cloud is calculated, the acceleration observation value of the inertial measurement unit is fused through an extended Kalman filter, and the motion compensation parameter is dynamically adjusted; The optimized motion compensation parameter is used as the time sequence weight, and the spatial distribution histogram of the motion normalized point cloud is used as the geometric feature, to construct a composite feature descriptor with spatio-temporal joint coding.

[0050] It should be noted that, in order to solve the point cloud distortion problem caused by the coupling of rigid body displacement and surface deformation of the moving object in the dynamic monitoring process. The embodiment realizes real-time decoupling and compensation of the surface deformation of the moving object by fusing inertial measurement data and point cloud geometric features, effectively eliminates the influence of motion artifacts on the three-dimensional reconstruction accuracy, and finally generates a spatio-temporal joint coding feature descriptor which can accurately represent the object topographic features in the motion state, providing a reliable motion invariance feature expression for dynamic industrial detection scenes.

[0051] In the present embodiment, the object monitoring method further comprises the following steps: By using the spatial topology index of the historical point cloud frame sequence, a motion trajectory model of the target object is established through point cloud registration between adjacent frames to extract a motion direction vector and an instantaneous speed amplitude as spatiotemporal correlation features; wherein the spatial topology index of the historical point cloud frame sequence refers to a structured data organization method for recording the position change trajectory and neighborhood connection relationship of point cloud feature points between adjacent frames by establishing the spatial correspondence (such as KD-Tree or octree structure) between multiple frames of continuous three-dimensional point cloud data; According to the dispersion of the motion direction vector and the fluctuation rate of the instantaneous speed amplitude, the uncertainty of the motion state is quantified to generate a spatiotemporal dynamic motion entropy weight value; It should be noted that by obtaining the standard deviation (such as dispersion 0.15 rad) of the motion direction vector between continuous frames, and simultaneously counting the coefficient of variation (such as fluctuation rate 8%) of the instantaneous speed amplitude, the two parameters are then normalized to the range of 0-1, and the initial entropy weight value is obtained by weighted summation according to the preset weight proportion (such as direction dispersion accounting for 60% and speed fluctuation rate accounting for 40%). Finally, according to the motion characteristics of the target object, a dynamic threshold (such as 0.7) is set, the regions with entropy weight values higher than the threshold are marked as high dynamic regions (such as the hinge of the car door with weight value 0.85), and the regions with entropy weight values lower than the threshold are marked as low dynamic regions (such as the outer panel of the car door with weight value 0.2), and finally the spatiotemporal dynamic motion entropy weight value is outputted after layered quantization; According to the spatiotemporal dynamic motion entropy weight value, the current frame point cloud is divided into a non-uniform voxel grid: regions with motion entropy weight values higher than a preset entropy weight value (such as entropy weight value 0.9) are retained with fine-grained voxels to preserve detailed features, and regions with motion entropy weight values not higher than the preset entropy weight value are merged with coarse-grained voxels to combine static background points; The surface curvature gradient and the normal vector angle of the point cloud clusters in the fine-grained voxels are calculated, and the feature points above the curvature gradient threshold are retained; the coarse-grained voxels are represented by dimension reduction using the centroid coordinates and the bounding box size to form compressed data blocks containing layered geometric feature identifiers; The compressed data blocks are reconstructed into simplified point clouds, and the difference degree of the surface normal vector field between the simplified point clouds and the original point clouds in the regions with motion entropy weight values higher than the preset entropy weight value is calculated. If the difference degree exceeds the reconstruction tolerance, the motion entropy weight value of the corresponding region is increased by a preset proportion and local re-compression is triggered until the feature fidelity constraint condition is met.

[0052] It should be noted that in order to solve the balance problem between point cloud data processing efficiency and feature fidelity when the moving object and the static background are mixed in the dynamic monitoring scene. The present embodiment realizes intelligent layered processing of point cloud data through motion trajectory analysis and dynamic entropy weight evaluation, while ensuring the detailed features of the motion region and reducing the data redundancy of the static background, thereby providing an efficient and accurate point cloud data processing method for motion object monitoring.

[0053] As Figure 3 shown, the second aspect of the present application discloses a monitoring camera applied to any of the object monitoring methods based on multi-camera joint calibration technology, comprising: A rigid connection base unit 1 made of low thermal expansion coefficient alloy material, provided with standardized mechanical interface and locking mechanism, used to form rigid topological connection with adjacent cameras or supporting structure; A multi-axis attitude adjustment module 2 integrated in the base, including a three-degree-of-freedom micro-displacement platform and a two-axis pitch-yaw adjustment mechanism; An embedded inertial measurement unit 3 fixed at the camera optical center axis, including a three-axis MEMS accelerometer and a fiber-optic gyroscope, with a sampling frequency ≥1kHz, real-time output of six-degree-of-freedom pose data; A temperature control compensation module 4, including a distributed temperature sensor array attached to the camera shell and a semiconductor cooling plate; A synchronous trigger interface 5 supporting hardware-level external trigger signal input and cascading output; An optical imaging module 6, including a global shutter CMOS sensor, a telecentric lens, and a tunable polarization filter, with a working waveband covering 400-1000nm.

[0054] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any skilled person in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. An object monitoring method based on multi-camera joint calibration technology, characterized in that, Includes the following steps: S102: Fix multiple cameras in preset relative positions to form a camera array with basic spatial constraints and establish an initial spatial relationship chain; S104: Drives all cameras to simultaneously capture images of a calibration reference with known geometric features, and uses the captured images to solve for the intrinsic and extrinsic parameters of each camera and their precise spatial pose relationships, forming a joint spatial relationship chain; S106: In the monitoring state, control the camera array to synchronously capture the target object, and use the joint spatial relationship chain to convert the two-dimensional image points collected by each camera into spatial point coordinates in a unified three-dimensional coordinate system to generate three-dimensional point cloud data of the target object. S108: Process the generated 3D point cloud data, extract key geometric features of the target object in real time, compare and analyze the extracted features with preset standards or historical states, and output the status monitoring results of the target object. S110: During system operation, physical disturbances that may cause changes in the relative pose of the camera are sensed in real time, and the automatic verification and update process of the joint spatial relationship chain is triggered based on the sensed information to maintain the accuracy of three-dimensional measurement.

2. The object monitoring method based on multi-camera joint calibration technology according to claim 1, characterized in that, Specifically, S102 is as follows: Based on the size of the target object and the coverage requirements, the preset camera relative orientation parameters are determined, including the three-dimensional position coordinates and orientation angles of each camera, in order to define the geometric layout of the basic spatial constraints. Using rigid connection structural components, multiple cameras are physically fixed according to the determined orientation parameters to form a camera array, and the rigid frame of the array provides basic spatial constraints to suppress random changes in relative pose. By integrating a miniature inertial measurement unit onto the structural component, the actual relative pose data between cameras is collected in real time, including distance deviation and angular offset, as spatial constraint data; Based on the spatial constraint data and combined with preset orientation parameters, the relative positions and orientations of each camera are determined through geometric relationships, and an initial spatial relationship chain is established.

3. The object monitoring method based on multi-camera joint calibration technology according to claim 2, characterized in that, Based on the spatial constraint data and combined with preset orientation parameters, the relative positions and orientations of each camera are determined through geometric relationships, and an initial spatial relationship chain is established, specifically as follows: The relative pose data acquired by the miniature inertial measurement unit is filtered and denoised to eliminate high-frequency vibration noise and extract stable relative displacement and angle change data between cameras. Using the optical center of the main camera specified in the preset orientation parameters as the origin and its optical axis as the reference axis, a global coordinate system is established, and the preprocessed relative displacement and angle change data are transformed into this coordinate system to obtain the initial position and orientation data of each slave camera. Based on the initial position and orientation data in the global coordinate system, a camera pose optimization model is constructed using rigid connection constraints between cameras. By minimizing the deviation between the measured distance between adjacent cameras and the theoretical distance of the preset orientation parameters, the relative position and orientation angle of each camera are determined. Based on the optimized camera pose data, the residual between the actual distance between each camera and the preset theoretical distance is calculated. If the residual exceeds the deformation threshold of the rigid connection structure, the pose data is adjusted until the residual does not exceed the deformation threshold of the rigid connection structure. The optimized camera pose data are arranged in spatial topological order. Taking the main camera as the reference, the three-dimensional offset and orientation angle of each slave camera relative to the main camera are recorded to form an initial spatial relationship chain, where the confidence of each node is determined by the optimized residual.

4. The object monitoring method based on multi-camera joint calibration technology according to claim 1, characterized in that, Specifically, S104 is as follows: Drive all cameras to simultaneously photograph a calibration reference object with a preset geometric pattern, and acquire the original image set of each camera; Subpixel-level corner detection is performed on each image to extract the feature point coordinates of the calibration reference object, and a cross-camera mapping relationship of corresponding point pairs is established based on the consistency of the spatial distribution of feature points. The initial spatial relationship chain is used as a priori constraint, and the preset optical center position and optical axis orientation of each camera are input. Using the mapping relationship of corresponding point pairs as observations, and integrating the pose constraints of the initial spatial relationship chain, a multi-camera joint reprojection error model is constructed. And obtain the intrinsic and extrinsic parameters of all cameras, and construct the intrinsic parameter matrix, extrinsic parameter rotation matrix and extrinsic parameter translation vector; The extrinsic rotation matrix and extrinsic translation vector are connected according to the camera space topology. With the main camera coordinate system as the reference, the relative pose transformation matrix between adjacent cameras is calculated step by step to form a tree-like pose transformation chain. By integrating all relative pose transformation matrices and intrinsic parameter matrices, a joint spatial relationship chain containing the spatial pose dependencies between cameras is constructed.

5. The object monitoring method based on multi-camera joint calibration technology according to claim 1, characterized in that, Specifically, S106 is as follows: Control all cameras to simultaneously capture images of the target object and acquire the original image set; perform noise reduction and edge enhancement processing on each image and extract two-dimensional image feature points with sub-pixel precision; Using the intrinsic parameter matrix stored in the joint spatial relation chain, lens distortion correction is performed on the extracted two-dimensional image feature points to generate a distortion-free image point set; based on the tree-like pose transformation chain of the joint spatial relation chain, the distortion-free image point sets of each camera are mapped to the virtual imaging plane in the main camera coordinate system. On the virtual imaging plane, based on the epipolar constraint relationship between adjacent cameras, bidirectional matching is performed on the image feature points in the overlapping area to establish cross-camera corresponding point pairs, and the initial spatial coordinates of the corresponding point pairs in three-dimensional space are obtained. Based on the pose confidence of each node in the joint spatial relationship chain, the initial spatial coordinates are weighted and optimized: multiple sets of initial spatial coordinates corresponding to the same point pair are projected to the main camera coordinate system; weights are assigned according to the confidence of the camera nodes involved in the calculation, and optimized spatial point coordinates are generated by fusion using the weighted least squares method. Remove the coordinates of outliers caused by confidence levels below a threshold; Traverse all successfully matched pairs of points with the same name, optimize their spatial point coordinates and map them to a unified world coordinate system in spatial topological order; integrate all spatial point coordinates and attach the identification information of the source camera to form a dense 3D point cloud dataset with spatial topological consistency.

6. The object monitoring method based on multi-camera joint calibration technology according to claim 1, characterized in that, Specifically, S108 is as follows: Spatial topology indexing is applied to the input 3D point cloud data, and outlier noise points are removed by voxel raster filtering; at the same time, the spatial topology indexing is used to reconstruct the neighborhood relationship of the point cloud and generate a surface mesh structure with connectivity. On the reconstructed surface mesh, the spatial principal orientation axis of the target object is analyzed and set as the geometric feature calculation reference; and equal-spaced sections are divided along the spatial principal orientation axis to generate a series of orthogonal projection planes. Fit the boundary points of the cross-section profile in the projection plane to obtain the size of the envelope rectangle, the coordinates of the center, and the profile curvature entropy of each cross-section, forming a size-position-shape feature vector; Based on the surface mesh normal vector field, the multi-scale surface gradient characteristics of each vertex are calculated to generate a surface morphology change map. The size-position-shape feature vector of the current frame is matched with the feature vector of the preset standard model using Euclidean distance, and the size deviation and position offset are output. Multi-scale surface gradient features are input into a pre-trained defect classification model, and local abnormal regions are identified by convolution kernel matching. By combining the results of dimensional deviation, positional offset, and surface defect identification, and using weighted confidence fusion, a comprehensive state evaluation index for the target object is obtained; when the comprehensive state evaluation index exceeds a preset index threshold, a state abnormality alarm is triggered.

7. The object monitoring method based on multi-camera joint calibration technology according to claim 1, characterized in that, Specifically, S110 is as follows: Real-time acquisition of six-degree-of-freedom vibration acceleration and angular velocity data of the camera array, while monitoring the local temperature rise gradient of each camera node, forming a spatiotemporally synchronized physical disturbance feature vector; The physical disturbance feature vector is input into the pre-calibrated camera structure deformation transfer model to determine the predicted value of the relative pose offset of each camera node caused by temperature gradient and mechanical vibration, and generate the pose drift feature matrix. Based on the predicted offset values ​​of each node in the pose drift feature matrix, and combined with the original pose confidence in the joint spatial relationship chain, the dynamic reliability index of the current spatial relationship chain is determined. When the reliability index is lower than the preset index threshold, the camera array is controlled to synchronously shoot the preset fixed orientation calibration reference in the scene. Using the known three-dimensional geometric constraints of the calibration reference, the actual pose change is inverted through multi-view imaging differences to generate the measured pose compensation vector. The measured pose compensation vector and the predicted value of the pose drift feature matrix are fused by Kalman gain to obtain the optimized pose correction amount. The extrinsic rotation matrix and extrinsic translation vector in the joint spatial relation chain are updated node by node according to the topological order of the tree pose transformation chain, while the pose confidence weight of each node is adjusted. The updated joint spatial relationship chain is applied to the reconstruction of the target object point cloud acquired in real time. The root mean square rate of change of the point cloud registration error between adjacent frames is calculated. If the root mean square rate of change does not converge to the preset rate of change threshold, the iteration continues until the root mean square rate of change converges to the preset rate of change threshold.

8. The object monitoring method based on multi-camera joint calibration technology according to claim 7, characterized in that, The specific steps for constructing the pre-calibrated camera structure deformation transfer model are as follows: Based on the material properties and geometric layout of the rigid connection structure of the camera array, a theoretical deformation response parameter matrix of the structure is established to characterize the linear and nonlinear deformation effects of temperature change and mechanical vibration on the structure. During the calibration phase, a controllable temperature gradient change and mechanical vibration excitation are applied to the camera array, and the six-degree-of-freedom pose offset data output by the micro inertial measurement unit are recorded simultaneously to form an excitation-response sample set. The excitation data includes temperature change rate data and vibration spectrum data, and the response data includes measured displacement data and angle deviation data. Using the excitation-response sample set, the mapping relationship between temperature gradient and vibration spectrum to camera pose offset is established by least squares fitting, generating an initial deformation transfer coefficient matrix. Then, the nonlinear error term in the theoretical deformation response parameter matrix is ​​corrected by recursive optimization method to obtain the deformation transfer model. Based on the vibration spectrum data, the dynamic response characteristics of the structural components at different vibration frequencies are analyzed, the steady-state offset and transient vibration components are separated, and corresponding compensation sub-models are established respectively. The compensation sub-model and the deformation transfer model are fused to obtain the camera structure deformation transfer model, which is used to predict the pose offset caused by physical disturbances in real time.

9. A monitoring camera, applied to the object monitoring method based on multi-camera joint calibration technology as described in any one of claims 1 to 8, characterized in that, include: The rigid connection base unit is made of a low thermal expansion coefficient alloy material and is equipped with a standardized mechanical interface and locking mechanism to form a rigid topological connection with adjacent cameras or support structures. The multi-axis attitude adjustment module is integrated inside the base and includes a three-degree-of-freedom micro-displacement platform and a two-axis pitch-yaw adjustment mechanism; An embedded inertial measurement unit, fixed at the optical center axis of the camera, includes a three-axis MEMS accelerometer and a fiber optic gyroscope, with a sampling frequency ≥1kHz, and outputs six-degree-of-freedom pose data in real time; The temperature control compensation module includes a distributed temperature sensor array and a semiconductor cooling chip attached to the camera housing; Synchronous trigger interface, supporting hardware-level external trigger signal input and cascaded output; The optical imaging module includes a global shutter CMOS sensor, a telecentric lens, and a tunable polarizing filter, with a working wavelength range of 400-1000nm.

Images