Non-standard site adaptive visual calibration and trajectory correction system and method

Abstract

This invention relates to the field of computer vision technology, and more particularly to an adaptive visual calibration and trajectory correction system and method for non-standard sites. It achieves adaptive illumination compensation and viewpoint interference elimination by fusing multimodal data, and utilizes precise mapping of inherent geometric references of the site to quickly solve camera parameters under complex environmental conditions. Employing a residual monitoring algorithm based on a sliding time window, it can evaluate pixel deviations in the corrected image online, automatically triggering recalibration when accuracy decreases without interrupting system operation. This overcomes the shortcomings of existing technologies that require frequent interruptions and struggle to detect accuracy drift in real time. Simultaneously, by combining sensor compensation data with spatiotemporal alignment, it suppresses cumulative errors caused by environmental factors and slow changes in lens distortion, ensuring calibration consistency over long-term operation. It is particularly suitable for real-time visual measurement and trajectory guidance tasks on non-standard sites such as running tracks and sandpits.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a non-standard site adaptive visual calibration and trajectory correction system and method. Background Technology

[0002] In fields such as sports training, motion analysis, and field safety monitoring, it is often necessary to use cameras installed around non-standard fields such as running tracks and sandpits for long jumps to track the trajectory, measure distance, or analyze the posture of moving targets (such as athletes and equipment). To achieve high-precision visual measurements, the cameras must be calibrated to obtain their intrinsic parameters (such as focal length and principal point coordinates) and lens distortion (such as radial distortion and tangential distortion) must be corrected.

[0003] However, existing technologies cannot assess calibration accuracy online when there are changes in lighting, viewing angle, and slow drift due to lens distortion. They often require interruption and recalibration, making it difficult to achieve efficient and continuous reliable visual correction. Summary of the Invention

[0004] The purpose of this invention is to provide a non-standard site adaptive visual calibration and trajectory correction system and method, which solves the technical problem that the calibration accuracy cannot be evaluated online when there are changes in lighting, viewing angle and slow drift due to lens distortion. It often requires interruption and recalibration, making it difficult to achieve efficient and continuous reliable visual correction.

[0005] To achieve the above objectives, this invention provides a method for adaptive visual calibration and trajectory correction in non-standard sites, comprising: By setting up at least one camera around the non-standard site, an image sequence containing the site's inherent geometric references is acquired, and during the acquisition process, sensor compensation data, spatiotemporal location data, and privacy protection parameters protected by homomorphic encryption transmission are acquired simultaneously for each frame of the image. A multimodal data fusion method is used to perform adaptive illumination compensation on the acquired images to eliminate illumination and viewing angle interference. Then, pixel-level representations of geometric reference objects are automatically extracted. Geometric reference objects include inherent field features with standard real dimensions, such as runway line width and sandpit boundary length. Based on the geometric mapping relationship between the true size of the geometric reference object and the pixel representation, the camera intrinsic parameters and lens distortion parameters are solved. During the solution process, the parameters are corrected for environmental factors using sensor compensation data, and the spatiotemporal position data is used to perform alignment constraints on multi-view or multi-time data. An inverse distortion mapping table is generated based on distortion parameters to correct lens distortion in the video stream. A residual monitoring algorithm based on a sliding time window is used to evaluate the pixel deviation of geometric references in the corrected image. When the deviation exceeds a preset threshold, it is automatically marked that recalibration is needed without interrupting the calibration system to trigger recalibration. The process data generated during the entire calibration and correction process is stored in a traceable manner, and the algorithm parameters are optimized using a stepwise trial and error update mechanism. In the data transmission and processing stages, an encrypted transmission method that supports the execution of necessary operations without decryption is used to protect critical parameters.

[0006] The encrypted transmission method is as follows: at the camera end, the camera intra-parameters, distortion parameters obtained in the calibration step, and the inverse distortion mapping table generated in the correction step are converted into ciphertext data using a homomorphic encryption scheme; this ciphertext data is transmitted to the analysis server; the server performs parameter storage, quality evaluation, and calibration consistency verification directly based on homomorphic arithmetic operations on the ciphertext without completely decrypting it; the terminal only performs complete decryption when it needs to output visualization results or plaintext trajectories to an authorized terminal.

[0007] The sensor compensation data includes: ambient light intensity, camera attitude angle, lens temperature, and motion blur factor. Adaptive illumination compensation includes: dynamically adjusting the contrast stretching coefficient of the image based on the ambient light intensity, performing adaptive deblurring filtering based on the motion blur factor, and correcting focal length drift based on the lens temperature.

[0008] The spatiotemporal location data includes: assigning a globally unified timestamp to each frame of the image, and the spatial coordinates and orientation angle of the camera in the world coordinate system; The spatiotemporal alignment constraints in the calibration process include: aligning image features from different frames or different cameras to the same time reference through linear interpolation or spline interpolation, and unifying the multi-view image coordinate system to the same world coordinate system through rigid body transformation.

[0009] Specifically, the traceable storage method is as follows: Each complete record generated from the acquisition step to the calibration step is stored in the process database. The complete record includes the original image feature values, sensor compensation parameters, intermediate calibration results, and final trajectory error feedback. The gradual testing and update mechanism is as follows: the background optimization engine periodically scans the process database, mines statistical patterns of calibration errors under different environments, generates multiple sets of candidate algorithm parameter groups, and compares the correction accuracy of each parameter group through online low-traffic tests, and gradually updates the parameter group with the best performance to the front-end algorithm module.

[0010] The pixel-level representation extraction of geometric references, specifically for runway linewidth, employs the following method: Identify the two parallel edge lines and the middle lane line of the straight area of ​​the runway, measure the average pixel width of the lines in the image, and recover the camera pitch angle and optical center height by combining the vanishing point constraint.

[0011] Among them, the pixel-level representation extraction of geometric reference objects, for the sandpit long jump field, uses both the edge length of the take-off board and the distance between the boundary lines on both sides of the sandpit as geometric reference objects; in the calibration step, the known lengths of these two orthogonal directions are used to jointly calculate the radial distortion coefficient and tangential distortion coefficient of the camera in order to correct the coordinate deviation of the long jump landing point.

[0012] This invention also provides a non-standard site adaptive visual calibration and trajectory correction system for performing the non-standard site adaptive visual calibration and trajectory correction method described above. It includes an image acquisition module, a modal environment perception module, an image preprocessing module, an automatic geometric feature extraction module, a multi-source fusion calibration module, a dynamic correction and recalibration module, and an encrypted transmission module; The image acquisition module is used to acquire image sequences that include inherent geometric references of the site. The multimodal environment perception module is used to collect sensor compensation data and spatiotemporal location data in real time; The image preprocessing module receives the output of the image acquisition module and performs adaptive illumination compensation and deblurring filtering. The automatic geometric feature extraction module is used to automatically identify and extract pixel-level representations of inherent geometric references of the site, such as runway width and sandpit boundaries. The multi-source fusion calibration module utilizes the geometric mapping relationship between the true size of the geometric reference object and the pixel representation to fuse sensor compensation data and jointly solve the camera intrinsic parameters and distortion parameters. At the same time, it uses spatiotemporal position data to complete the spatiotemporal alignment constraints of multi-view images. The dynamic correction and recalibration module generates an inverse distortion mapping table based on distortion parameters to correct the video stream, and uses a residual monitoring algorithm with a sliding time window to automatically trigger recalibration. The encrypted transmission module is used to transmit key calibration parameters to the analysis server after homomorphic encryption, and to perform parameter quality evaluation and consistency verification without decrypting the ciphertext.

[0013] This invention discloses an adaptive visual calibration and trajectory correction system and method for non-standard sites. It achieves adaptive illumination compensation and viewpoint interference elimination by fusing multimodal data and utilizes precise mapping of inherent geometric references to rapidly solve camera parameters under complex environmental conditions. Employing a residual monitoring algorithm based on a sliding time window, it can evaluate pixel deviations in the corrected image online and automatically trigger recalibration when accuracy decreases without interrupting system operation, overcoming the shortcomings of existing technologies that require frequent interruptions and struggle to detect accuracy drift in real time. Simultaneously, by combining sensor compensation data with spatiotemporal alignment, it suppresses cumulative errors caused by environmental factors and slow changes in lens distortion, ensuring calibration consistency over long-term operation. Encrypted transmission and traceable storage mechanisms protect the privacy of key parameters while supporting gradual parameter optimization, making the entire calibration process highly accurate, highly secure, and with low maintenance costs. It is particularly suitable for real-time visual measurement and trajectory guidance tasks on non-standard sites such as running tracks and sandpits. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0015] Figure 1 This is a flowchart of the non-standard site adaptive visual calibration and trajectory correction method of the present invention. Detailed Implementation

[0016] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0017] Please refer to Figure 1 , Figure 1 This is a flowchart of the non-standard site adaptive visual calibration and trajectory correction method of the present invention. An embodiment of the present invention provides a non-standard site adaptive visual calibration and trajectory correction method, including: S1. By setting up at least one camera around the non-standard site, an image sequence containing the inherent geometric reference of the site is acquired, and during the acquisition process, sensor compensation data, spatiotemporal location data and privacy protection parameters protected by homomorphic encryption transmission are acquired simultaneously for each frame of the image. In this specific implementation, the sensing compensation data includes: ambient light intensity, camera attitude angle, lens temperature, and motion blur factor; Adaptive illumination compensation includes: dynamically adjusting the image contrast stretching coefficient based on ambient light intensity, performing adaptive deblurring filtering based on motion blur factor, and correcting focus drift based on lens temperature. Spatiotemporal location data includes: assigning a globally unified timestamp to each frame, and the camera's spatial coordinates and orientation angle in the world coordinate system; The spatiotemporal alignment constraints in the calibration process include: aligning image features from different frames or different cameras to the same time reference through linear interpolation or spline interpolation, and unifying the multi-view image coordinate system to the same world coordinate system through rigid body transformation.

[0018] S2. Adaptive illumination compensation is performed on the acquired images using a multimodal data fusion method to eliminate illumination and viewing angle interference. Then, pixel-level representations of geometric reference objects are automatically extracted. Geometric reference objects include inherent field features with standard real dimensions, such as runway width and sandpit boundary length. S3. Based on the geometric mapping relationship between the true size of the geometric reference and the pixel representation, solve the camera intrinsic parameters and lens distortion parameters. In the process of solving, use sensor compensation data to correct the parameters for environmental factors, and use spatiotemporal position data to perform alignment constraints on multi-view or multi-time data. S4. Generate an inverse distortion mapping table based on distortion parameters to correct lens distortion in the video stream. Use a residual monitoring algorithm based on a sliding time window to evaluate the pixel deviation of geometric references in the corrected image. When the deviation exceeds a preset threshold, it is automatically marked that recalibration is required without interrupting the calibration system to trigger recalibration. S5. Store the process data generated during the entire calibration and correction process in a traceable manner, and optimize the algorithm parameters using a stepwise trial and update mechanism; In this specific implementation method, the traceable storage method is as follows: Each complete record generated from the acquisition step to the calibration step is stored in the process database. The complete record includes the original image feature values, sensor compensation parameters, intermediate calibration results, and final trajectory error feedback. The gradual testing and update mechanism is as follows: the background optimization engine periodically scans the process database, mines statistical patterns of calibration errors under different environments, generates multiple sets of candidate algorithm parameter groups, and compares the correction accuracy of each parameter group through online low-traffic tests, and gradually updates the parameter group with the best performance to the front-end algorithm module.

[0019] S6. In the data transmission and processing stage, a cryptographic transmission method that supports the execution of necessary operations without decryption is adopted to protect key parameters.

[0020] In this specific implementation, the encrypted transmission method is as follows: at the camera end, the camera intrinsic parameters, distortion parameters obtained in the calibration step, and the inverse distortion mapping table generated in the correction step are converted into ciphertext data using a homomorphic encryption scheme; the ciphertext data is transmitted to the analysis server; the server performs parameter storage, quality evaluation, and calibration consistency verification directly based on homomorphic arithmetic operations on the ciphertext without completely decrypting it; the terminal only performs complete decryption when it needs to output visualization results or plaintext trajectories to an authorized terminal.

[0021] The pixel-level representation extraction of geometric references, specifically for runway linewidth, employs the following method: Identify the two parallel edge lines and the middle lane line of the straight area of ​​the runway, measure the average pixel width of the lines in the image, and recover the camera pitch angle and optical center height by combining the vanishing point constraint.

[0022] Among them, the pixel-level representation extraction of geometric reference objects, for the sandpit long jump field, uses both the edge length of the take-off board and the distance between the boundary lines on both sides of the sandpit as geometric reference objects; in the calibration step, the known lengths of these two orthogonal directions are used to jointly calculate the radial distortion coefficient and tangential distortion coefficient of the camera in order to correct the coordinate deviation of the long jump landing point.

[0023] Secondly, adaptive illumination compensation also includes image enhancement steps based on histogram equalization, adaptive gamma correction, or Retinex theory to eliminate contrast distortion in non-uniform illumination and shadow areas.

[0024] Simultaneously, the quality of the corrected image is evaluated, with evaluation metrics including image sharpness, signal-to-noise ratio, structural similarity (SSIM), or peak signal-to-noise ratio (PSNR). When the quality metrics fall below a set threshold, the illumination compensation parameters are automatically adjusted and the calibration is re-performed.

[0025] Furthermore, anomaly detection is performed on the corrected trajectory data. Outlier identification methods based on Mahalanobis distance, isolated forest, or local outlier factor (LOF) are used to automatically mark and remove trajectory flying points caused by instantaneous occlusion or motion blur.

[0026] This invention also provides a non-standard site adaptive visual calibration and trajectory correction system for performing the non-standard site adaptive visual calibration and trajectory correction method described above. It includes an image acquisition module, a modal environment perception module, an image preprocessing module, an automatic geometric feature extraction module, a multi-source fusion calibration module, a dynamic correction and recalibration module, and an encrypted transmission module; The image acquisition module is used to acquire image sequences that include inherent geometric references of the site. The multimodal environment perception module is used to collect sensor compensation data and spatiotemporal location data in real time; The image preprocessing module receives the output of the image acquisition module and performs adaptive illumination compensation and deblurring filtering. The automatic geometric feature extraction module is used to automatically identify and extract pixel-level representations of inherent geometric references of the site, such as runway width and sandpit boundaries. The multi-source fusion calibration module utilizes the geometric mapping relationship between the true size of the geometric reference object and the pixel representation to fuse sensor compensation data and jointly solve the camera intrinsic parameters and distortion parameters. At the same time, it uses spatiotemporal position data to complete the spatiotemporal alignment constraints of multi-view images. The dynamic correction and recalibration module generates an inverse distortion mapping table based on distortion parameters to correct the video stream, and uses a residual monitoring algorithm with a sliding time window to automatically trigger recalibration. The encrypted transmission module is used to transmit key calibration parameters to the analysis server after homomorphic encryption, and to perform parameter quality evaluation and consistency verification without decrypting the ciphertext.

[0027] This invention provides an adaptive visual calibration and trajectory correction system and method for non-standard sites. By fusing multimodal data, it achieves adaptive illumination compensation and viewpoint interference elimination. Utilizing precise mapping of inherent geometric references within the site, it can rapidly solve for camera parameters under complex environmental conditions. Employing a residual monitoring algorithm based on a sliding time window, it can evaluate pixel deviations in the corrected image online. When accuracy decreases, it automatically triggers recalibration without interrupting system operation, overcoming the shortcomings of existing technologies that require frequent interruptions and struggle to detect accuracy drift in real time. Simultaneously, by combining sensor compensation data with spatiotemporal alignment, it suppresses cumulative errors caused by environmental factors and slow changes in lens distortion, ensuring calibration consistency over long-term operation. Encrypted transmission and traceable storage mechanisms protect the privacy of key parameters while supporting gradual parameter optimization, making the entire calibration process highly accurate, secure, and cost-effective. It is particularly suitable for real-time visual measurement and trajectory guidance tasks in non-standard sites such as runways and sandpits.

[0028] The above-disclosed embodiments are merely one or more preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art can understand that all or part of the processes for implementing the above embodiments and equivalent changes made in accordance with the claims of this application still fall within the scope of this application.

Claims

1. A method for adaptive visual calibration and trajectory correction in non-standard sites, characterized in that, include: By setting up at least one camera around the non-standard site, an image sequence containing the site's inherent geometric references is acquired, and during the acquisition process, sensor compensation data, spatiotemporal location data, and privacy protection parameters protected by homomorphic encryption transmission are acquired simultaneously for each frame of the image. A multimodal data fusion method is used to perform adaptive illumination compensation on the acquired images to eliminate illumination and viewing angle interference. Then, pixel-level representations of geometric reference objects are automatically extracted. Geometric reference objects include inherent field features with standard real dimensions, such as runway line width and sandpit boundary length. Based on the geometric mapping relationship between the true size of the geometric reference object and the pixel representation, the camera intrinsic parameters and lens distortion parameters are solved. During the solution process, the parameters are corrected for environmental factors using sensor compensation data, and the spatiotemporal position data is used to perform alignment constraints on multi-view or multi-time data. An inverse distortion mapping table is generated based on distortion parameters to correct lens distortion in the video stream. A residual monitoring algorithm based on a sliding time window is used to evaluate the pixel deviation of geometric references in the corrected image. When the deviation exceeds a preset threshold, it is automatically marked that recalibration is needed without interrupting the calibration system to trigger recalibration. The process data generated during the entire calibration and correction process is stored in a traceable manner, and the algorithm parameters are optimized using a stepwise trial and error update mechanism. In the data transmission and processing stages, an encrypted transmission method that supports the execution of necessary operations without decryption is used to protect critical parameters.

2. The non-standard site adaptive visual calibration and trajectory correction method as described in claim 1, characterized in that, The encrypted transmission method is as follows: at the camera end, the camera intra-parameters, distortion parameters obtained in the calibration step, and the inverse distortion mapping table generated in the correction step are converted into ciphertext data using a homomorphic encryption scheme; this ciphertext data is transmitted to the analysis server; the server performs parameter storage, quality evaluation, and calibration consistency verification directly based on homomorphic arithmetic operations on the ciphertext without completely decrypting it; only when it is necessary to output visualization results or plaintext trajectories to authorized terminals will the terminals perform complete decryption.

3. The non-standard site adaptive visual calibration and trajectory correction method as described in claim 2, characterized in that, Sensor compensation data includes: ambient light intensity, camera attitude angle, lens temperature, and motion blur factor; Adaptive illumination compensation includes: dynamically adjusting the contrast stretching coefficient of the image based on the ambient light intensity, performing adaptive deblurring filtering based on the motion blur factor, and correcting focal length drift based on the lens temperature.

4. The non-standard site adaptive visual calibration and trajectory correction method as described in claim 3, characterized in that, The spatiotemporal location data includes: assigning a globally unified timestamp to each frame of the image, as well as the spatial coordinates and orientation angle of the camera in the world coordinate system; The spatiotemporal alignment constraints in the calibration process include: aligning image features from different frames or different cameras to the same time reference through linear interpolation or spline interpolation, and unifying the multi-view image coordinate system to the same world coordinate system through rigid body transformation.

5. The non-standard site adaptive visual calibration and trajectory correction method as described in claim 4, characterized in that, The traceable storage method specifically refers to: Each complete record generated from the acquisition step to the calibration step is stored in the process database. The complete record includes the original image feature values, sensor compensation parameters, intermediate calibration results, and final trajectory error feedback. The gradual testing and update mechanism is as follows: the background optimization engine periodically scans the process database, mines statistical patterns of calibration errors under different environments, generates multiple sets of candidate algorithm parameter groups, and compares the correction accuracy of each parameter group through online low-traffic tests, and gradually updates the parameter group with the best performance to the front-end algorithm module.

6. The non-standard site adaptive visual calibration and trajectory correction method as described in claim 1, characterized in that, Pixel-level representation extraction of geometric references, specifically for runway linewidth, employs the following method: Identify the two parallel edge lines and the middle lane line of the straight area of ​​the runway, measure the average pixel width of the lines in the image, and recover the camera pitch angle and optical center height by combining the vanishing point constraint.

7. The non-standard site adaptive visual calibration and trajectory correction method as described in claim 1, characterized in that, Pixel-level representation extraction of geometric reference objects: For the sandpit long jump field, the geometric reference objects simultaneously adopt the edge length of the take-off board and the distance between the boundary lines on both sides of the sandpit; In the calibration step, the known lengths of these two orthogonal directions are used to jointly calculate the radial distortion coefficient and tangential distortion coefficient of the camera to correct the coordinate deviation of the long jump landing point.

8. A non-standard site adaptive visual calibration and trajectory correction system, used to execute the non-standard site adaptive visual calibration and trajectory correction method as described in claim 7, characterized in that, It includes an image acquisition module, a modal environment perception module, an image preprocessing module, an automatic geometric feature extraction module, a multi-source fusion calibration module, a dynamic correction and recalibration module, and an encrypted transmission module; The image acquisition module is used to acquire image sequences that include inherent geometric references of the site. The multimodal environment perception module is used to collect sensor compensation data and spatiotemporal location data in real time; The image preprocessing module receives the output of the image acquisition module and performs adaptive illumination compensation and deblurring filtering. The automatic geometric feature extraction module is used to automatically identify and extract pixel-level representations of inherent geometric references of the site, such as runway width and sandpit boundaries. The multi-source fusion calibration module utilizes the geometric mapping relationship between the true size of the geometric reference object and the pixel representation to fuse sensor compensation data and jointly solve the camera intrinsic parameters and distortion parameters. At the same time, it uses spatiotemporal position data to complete the spatiotemporal alignment constraints of multi-view images. The dynamic correction and recalibration module generates an inverse distortion mapping table based on distortion parameters to correct the video stream, and uses a residual monitoring algorithm with a sliding time window to automatically trigger recalibration. The encrypted transmission module is used to transmit key calibration parameters to the analysis server after homomorphic encryption, and to perform parameter quality evaluation and consistency verification without decrypting the ciphertext.

Images