A method for correcting large depth of field and multiple viewpoints
By constructing a multi-layer concentric circle camera array and dynamic depth-of-field calculation, the problem of image quality degradation in dynamic scenes when a single depth-of-field camera array is used is solved, and efficient 3D video reconstruction and real-time depth-of-field adjustment are achieved.
Patent Information
- Application Number
- CN202611249857.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-18
- Publication Date
- 2026-11-17
AI Technical Summary
In existing technologies, a single fixed depth-of-field camera array cannot capture both fine local movements and large-scale motions, resulting in a decrease in the quality of 3D video reconstruction of dynamic scenes. Furthermore, global depth calculation is time-consuming and cannot adapt to changes in the viewer's viewpoint position in real time.
A multi-layer concentric circle camera array is constructed. The shallow depth of field and deep depth of field reference parameters are calibrated for the inner and outer layers of the camera group, respectively. A proportional mathematical mapping model is established in combination with the observation distance to dynamically calculate the average depth of field value of the target. The inner and outer layer video images are then processed by interpolation to generate a 3D reconstructed video.
It improves the quality of 3D reconstruction videos captured in dynamic scenes, enhances computing speed and response efficiency, and can adjust the depth of field in real time to adapt to changes in the viewer's viewpoint.
Smart Images

Figure CN123123549A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of video image processing technology, specifically to a method for correcting large depth of field and multiple viewpoints. Background Technology
[0002] In 3D video display and arbitrary viewpoint video reconstruction applications, it is typically necessary to capture multiple video sources using a camera array to synthesize the 3D reconstructed image. Taking the shooting and reconstruction of first aid training videos as an example, the actions of the subjects are diverse, ranging from small, delicate movements like hemostasis and bandaging to larger spatial movements such as cardiopulmonary resuscitation and immobilization. Before shooting, geometric and color corrections are usually performed on the array equipment to adjust the placement and focus of the multi-view cameras to the basic shooting state.
[0003] Existing multi-viewpoint correction and video reconstruction methods primarily focus on static or small-scale spatial scenes, with their hardware arrays often relying on a single, fixed depth-of-field parameter for image acquisition. When dealing with dynamic shooting scenes involving both fine local movements and large-scale motion trajectories, a single, fixed optical depth-of-field range is insufficient to simultaneously meet the imaging requirements of different depth spans. When the target's movement is small, a single depth of field can maintain basic image features; however, when the target undergoes large-scale displacement, the sharpness of the moving areas in the image decreases significantly, leading to a reduction in the quality of the reconstructed 3D video. To improve the blurring problem caused by dynamic scene shooting, existing technologies typically rely on complex global depth matching algorithms for pixel correction during video reconstruction. This calculation method involves massive data processing and is time-consuming, resulting in low overall efficiency in video reconstruction. Furthermore, traditional correction calculations do not establish a computational correlation with the spatial position of the external observer. The system cannot quickly calculate and adjust the depth-of-field parameters based on the real-time distance changes of the viewer's perspective, making it difficult for the calculated video to adapt to the dynamic viewing angle requirements of the observer, thus limiting the display effect and response speed of multi-viewpoint dynamic scene video reconstruction. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention provides a large depth-of-field multi-viewpoint correction method, aiming to solve the problems in existing technologies where a single fixed depth-of-field array cannot simultaneously capture both local fine movements and large-scale motions, global depth calculation is time-consuming and lacks computational correlation with real-time viewpoint position, and changes in motion amplitude during dynamic scene shooting lead to a decrease in the clarity of the reconstructed video image.
[0005] To achieve the above objectives, the present invention provides the following technical solution: This invention provides a method for correcting large depth of field and multiple viewpoints, including: A multi-layer concentric circle camera array is constructed, which is divided into an inner layer camera group and an outer layer camera group; Multi-viewpoint parameter correction and depth-of-field reference calibration are performed on the multi-layer concentric circle camera array. The inner average depth-of-field value corresponding to the inner camera group and the outer average depth-of-field value corresponding to the outer camera group are extracted. The inner average depth-of-field value is set as the shallow depth-of-field lower limit reference parameter, and the outer average depth-of-field value is set as the large depth-of-field upper limit reference parameter. A proportional mathematical mapping model is established using the observation distance value obtained at the current acquisition time, the shallow depth of field lower limit benchmark parameter, and the large depth of field upper limit benchmark parameter to calculate the target average depth of field value matched by the observation distance value. The target average depth of field value obtained by dynamic calculation is set as a control variable, and interpolation calculation is performed on the bottom video images acquired by the inner camera group and the outer camera group to output the three-dimensional reconstructed video.
[0006] In one specific embodiment, the system executing the large depth-of-field multi-viewpoint correction method includes a front-end acquisition node and a logic processing hub communicatively connected to the front-end acquisition node. The front-end acquisition node is used to perform multi-viewpoint video image acquisition tasks for dynamic scenes. The logic processing hub includes a reference calibration module, a logic calculation module, and a rendering execution module.
[0007] This invention establishes a multi-layer concentric circle hardware topology and dynamically adjusts the depth-of-field calculation parameters based on the viewpoint distance. It logically binds the fixed optical characteristics of the physical camera with the dynamic viewpoint distance, thus solving the problem that a single fixed depth-of-field camera array cannot simultaneously capture local motion details and large-scale motion trajectories.
[0008] The construction of the multi-layer concentric circle camera array includes: Establish a coordinate space with the center point as the unified origin; arrange the first group of cameras at each vertex of the regular polygon to construct the outer camera group; arrange the second group of cameras at the intersection points of the inscribed circle and each edge of the regular polygon to construct the inner camera group; the positions of the inner camera group completely coincide with the midpoints of each edge of the regular polygon in spatial coordinates.
[0009] In one specific embodiment, the number of cameras in the outer camera group is equal to the number of cameras in the inner camera group, and each group has three or more camera nodes. The basic shape for constructing the outer and inner camera groups is defined as a regular quadrilateral.
[0010] By employing an alternating arrangement of vertices and tangent points of regular polygons, the inner camera is positioned at the midpoint of the line connecting the outer cameras. A regular quadrilateral was chosen as the basic topology, balancing optical occlusion limitations, calibration board size parameters, and hardware costs.
[0011] The multi-layer concentric circle camera array is extended outward to construct a multi-level physical acquisition array; The extended multi-level concentric circle array relies on the rule of reusing the positions of vertices and midpoints in adjacent layers; the radius parameters of all concentric circles in the multi-level physical acquisition array form a geometric sequence.
[0012] Preferably, the total number of layers in the constructed multi-level physical acquisition array is greater than or equal to two. The common ratio of the geometric sequence formed by the radius ratios between two adjacent extended layers is the sine of forty-five degrees.
[0013] Extending outwards based on the nested reuse rules of vertex and midpoint positions, a wide-angle video acquisition matrix is constructed using trigonometric function progressive ratios, providing a spatial data benchmark for depth-of-field fusion between different levels.
[0014] The process of performing multi-viewpoint parameter correction and depth-of-field reference calibration includes: The system triggers all nodes of the inner and outer camera groups to synchronously acquire checkerboard image sequences from different angles; calculates the internal and external parameter matrices of a single camera, and performs spatial geometric distortion correction accordingly; synchronously performs color consistency correction on the multi-viewpoint output images; extracts the depth-of-field data output by the inner camera group after calibration and correction, calculates the arithmetic mean value, and defines the calculation result as the inner average depth-of-field value; extracts the depth-of-field data output by the outer camera group after calibration and correction, calculates the arithmetic mean value, and defines the calculation result as the outer average depth-of-field value.
[0015] In one specific embodiment, before triggering synchronous acquisition, a standard checkerboard correction board is placed at the center of the camera array's shooting area for the front-end node to capture a reference image.
[0016] By extracting the average value of the corrected depth data of the inner and outer arrays as the benchmark parameter, the upper and lower limits of depth calculation are established, and the hardware optical characteristics are transformed into subsequent calculation conditions.
[0017] Before using the observation distance value obtained at the current acquisition time, the following is also included: Set a time statistics window of a predetermined duration; continuously extract the vertical distance data between the human eye and the plane where the display terminal is located within the time statistics window; record the minimum observation distance value that appears within the time statistics window; and synchronously record the maximum observation distance value that appears within the statistics period.
[0018] In one specific embodiment, a viewpoint capture device is configured on the video display terminal to continuously collect spatial coordinate data of the observer, and the numerical values of the distance calculation process are uniformly converted into millimeter-level standard units.
[0019] The minimum and maximum observation distance values are periodically updated to determine the spatial location constraint range of the observers, providing a numerical basis for eliminating the interference of spatial distance differences on the mapping operation.
[0020] The calculation of the target average depth value matching the observed distance numerical value includes: A normalized position scaling factor is set relative to the minimum and maximum observation distance values. The calculation model is set as the ratio of the deviation of the observation distance value from the minimum observation distance value to the deviation of the maximum observation distance value from the observation distance value. The formula for the normalized position scaling factor is substituted into the algebraic mapping equation. After algebraic simplification to eliminate the dynamic distance variable in the denominator, the analytical formula for the target average depth of field is derived. Based on the analytical formula for the target average depth of field, the dynamic depth of field parameter solution process is completed, and the target average depth of field value is calculated.
[0021] Preferably, the algebraic mapping equation is established based on the linear correlation features of similar triangles formed by spatial distance values and image depth parameters.
[0022] By converting absolute spatial distance into relative proportional parameters and eliminating denominator terms containing dynamic variables in algebraic equations in advance through algebraic transformation operations, the calculation error of zero denominator triggered when the observer is at the farthest position is avoided, thus ensuring the stability of the calculation process.
[0023] Before deriving the analytical formula for the target average depth of field, the following is also included: The difference between the maximum observation distance value and the minimum observation distance value within the time statistics window is detected; if the difference is less than the preset distance resolution threshold, which is an empirical constant, the algebraic mapping calculation is skipped, and the target average depth of field value is equal to the large depth of field upper limit reference parameter, or the calculation result of the previous sampling time is maintained.
[0024] In one specific embodiment, when the detected difference is less than a preset distance resolution threshold, the computing node determines that the physical displacement of the observer has not reached the position change recognition condition set by the system.
[0025] Threshold-based filtering of viewpoint jitter interference reduces unnecessary computations in the system.
[0026] Before performing interpolation calculations on the underlying video images acquired by the inner and outer camera groups, the process further includes: The shallow depth-of-field video image frame matrix of the inner camera group after parameter calibration and geometric distortion correction is extracted as the inner image matrix parameter; the large depth-of-field video image frame matrix of the outer camera group after the same correction is extracted simultaneously as the outer image matrix parameter; the inner image matrix parameter and the outer image matrix parameter are timestamped; and spatial resolution consistency resampling is performed on the inner image matrix parameter and the outer image matrix parameter.
[0027] In one specific embodiment, the system computing node receives multiple media streams from the front end in parallel to extract the inner and outer image matrix parameters for comparison and verification.
[0028] Timestamp verification and resampling are performed on the inner and outer layer data matrices to align the inner and outer layer video data in the temporal and spatial dimensions, providing a data foundation for subsequent weighted fusion calculations.
[0029] The interpolation calculation process includes: Based on the relative position of the target average depth of field value within the overall depth of field range, the inner layer weight factor corresponding to the inner layer image matrix parameters and the outer layer weight factor corresponding to the outer layer image matrix parameters are calculated respectively; the inner layer weight factor and the outer layer weight factor are obtained through the set interpolation fusion weight mathematical allocation equation.
[0030] Preferably, the denominator of the interpolation fusion weight mathematical allocation equation is uniformly set as the difference between the upper limit reference parameter of the large depth of field and the lower limit reference parameter of the shallow depth of field.
[0031] By taking into account the spatial layout constraints of the array, the physical shooting distance of the outer camera is always greater than that of the inner camera, which ensures that the difference in the reference parameters is greater than zero, thereby avoiding calculation anomalies, and converting the depth of field values into interpolation allocation ratios.
[0032] The output 3D reconstructed video includes: Pixel-level weighted interpolation is performed on the inner and outer image matrix parameters that are at the same timestamp and dimensionally aligned; a weighted fusion analytical equation is defined based on the inner and outer weight factors; a three-dimensional video image matrix is generated according to the weighted fusion analytical equation, and the three-dimensional reconstructed video is output.
[0033] In one specific embodiment, when the observation distance is reduced to the target's average depth of field value approaching the lower limit of shallow depth of field, the inner layer shallow depth of field feature image is output with a higher proportion according to the weighted fusion analysis equation; when the target's average depth of field value approaches the upper limit of large depth of field, the outer layer large depth of field panoramic feature image is output with an increased proportion according to the weighted fusion analysis equation.
[0034] By fusing the two-layer media streams using weighting factors, local motion details are highlighted when the observation distance is close, and large-scale motion trajectories are presented when the observation distance is far, generating a 3D video that balances local clarity and overall depth of field changes.
[0035] This invention provides a method for correcting large depth of field and multiple viewpoints. It has the following beneficial effects: First, by constructing a multi-layer concentric circle camera array divided into inner and outer camera groups, and separately calibrating and extracting shallow depth-of-field lower limit reference parameters and large depth-of-field upper limit reference parameters, this invention solves the problem that a single fixed depth-of-field camera device cannot simultaneously capture both local fine movements and large-span motion trajectories, providing a physical hardware structure foundation and a pre-set depth-of-field data benchmark for the synchronous capture of multi-range movements.
[0036] Secondly, this invention utilizes the observation distance value obtained at the current acquisition time and the established upper and lower limits of the depth of field to establish a proportional mathematical mapping model, which can dynamically calculate the matching target average depth of field value according to the change of the observer's spatial position. This avoids the huge amount of computation required for global depth measurement in traditional 3D reconstruction, and improves the calculation speed of the target depth of field and the system's response efficiency for viewpoint tracking.
[0037] Third, this invention uses the target average depth of field value obtained by dynamic calculation as a control variable to perform interpolation calculation on the bottom video images collected by the inner and outer camera groups. By fusing the image features of the inner shallow depth of field and the outer large depth of field, it improves the problem of reduced clarity in local motion areas caused by changes in the amplitude of movement during dynamic scene shooting, and improves the display quality of the final output three-dimensional reconstructed video. Attached Figure Description
[0038] Figure 1 This is a diagram illustrating the operational architecture of a large depth-of-field multi-viewpoint system according to an embodiment of the present invention.
[0039] Figure 2 This is a flowchart illustrating a large depth-of-field multi-viewpoint correction method according to an embodiment of the present invention.
[0040] Figure 3 This is a schematic diagram of the structural topology of three camera deployment methods provided in the embodiments of the present invention.
[0041] Figure 4 This is a schematic diagram of the spatial layout for two-layer multi-viewpoint correction provided in an embodiment of the present invention.
[0042] Figure 5 This is a schematic diagram illustrating the principle of proportional calculation of viewpoint distance and depth of field parameters provided in an embodiment of the present invention.
[0043] Figure 6 This is a schematic diagram of the spatial layout for three-layer multi-viewpoint correction provided in an embodiment of the present invention. Detailed Implementation
[0044] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0045] See attached document Figure 1 This invention provides a large depth-of-field multi-viewpoint correction system, which includes a front-end acquisition node and a logic processing hub that is communicatively connected to the front-end acquisition node.
[0046] The front-end acquisition node comprises a multi-layer concentric circle camera array. This array is divided into inner and outer camera groups. These groups are positioned on concentric circles with different radii, forming a spatial topology with a specific geometric relationship. The front-end acquisition node is used to acquire multi-view video images for dynamic scenes such as emergency medical training, obtaining raw multi-view video streams that capture both small, fine movements and large-scale motions.
[0047] The logic processing hub is the system's data calculation and instruction distribution node, specifically including a benchmark calibration module, a logic calculation module, and a rendering execution module.
[0048] The benchmark calibration module is used to perform spatial geometric correction and color correction on the multi-layer concentric circle camera array. The benchmark calibration module extracts the average depth-of-field values corresponding to the inner layer camera group and the outer layer camera group, and uses the extracted data as the lower and upper limits of the depth-of-field benchmark for subsequent logical operations.
[0049] The logic calculation module receives real-time viewpoint distance data from external sources. Based on the extreme values of viewpoint distance over a period of time and the current viewpoint distance, the logic calculation module, combined with the aforementioned depth-of-field reference parameters, calculates the target average depth-of-field value corresponding to the current viewpoint using a proportional mapping algorithm.
[0050] The rendering execution module receives the target average depth of field value output by the logic calculation module. Using this target average depth of field value as a control parameter, the rendering execution module performs interpolation calculations on the underlying video images captured by the inner and outer camera groups, outputting a 3D display video with large depth-of-field correction.
[0051] During the operation cycle of the entire system, after the execution unit deploys the multi-layer concentric-circle camera array to complete hardware networking, it triggers the reference calibration module to perform system initialization calibration, and establishes the depth-of-field reference parameters of the multi-view array. In the subsequent video reconstruction phase, the logic calculation module continuously captures position data and outputs the dynamically solved target depth-of-field value in real time. The rendering execution module performs dynamic fusion processing on the video stream according to the solution result, and drives the hardware to output a three-dimensional reconstructed video.
[0052] With reference to the accompanying Figure 2 , the present invention provides a large-depth-of-field multi-view calibration method, which is executed by the aforementioned system and includes the following steps: S100: Construct the physical topology of the multi-layer concentric-circle multi-view camera array, and establish an inner and outer multi-level physical acquisition array with a specific geometric constraint relationship by arranging cameras alternately at the vertices of a regular polygon and the tangent points of the corresponding inscribed circle; S200: Perform multi-view parameter correction and depth-of-field reference calibration, perform error correction on the camera array based on the checkerboard method, respectively extract the average value of the relatively shallow depth of field corresponding to the inner array and the average value of the relatively deep depth of field corresponding to the outer array, and establish the upper and lower limit reference parameters for system depth-of-field calculation; S300: Perform real-time logical mapping of dynamic depth of field based on viewpoint tracking, collect current viewpoint distance data in real time, establish a proportional mathematical mapping model using the observed distance value acquired at the current acquisition moment, the shallow depth-of-field lower limit reference parameter and the large depth-of-field upper limit reference parameter, and solve the target average depth-of-field value matched with the observed distance value; S400: Perform viewpoint cooperative interpolation and 3D video rendering based on the dynamic depth of field, convert the solved target average depth-of-field value into a control parameter, perform interpolation fusion on the image streams output by the multi-layer camera array, and generate and output the final 3D video that takes into account both picture definition and dynamic depth of field.
[0053] Hereinafter, each key step in the large-depth-of-field multi-view calibration method of the present invention will be described in detail with reference to specific implementation environments and calculation models.
[0054] With reference to the accompanying Figure 3 three deployment methods disclosed and the accompanying Figure 4 two-layer multi-view calibration structure disclosed, the step S100 called for system initialization in the previous sequence is described in detail in this embodiment. The spatial topology of the multi-layer concentric-circle multi-view camera array is constructed, and the specific implementation process includes the following control logic: The total number of deployed camera nodes is 2 and they are equally divided into two groups, each group has camera nodes, and the set constraint condition is , which is used to meet the minimum requirement for the number of basic vertices forming a regular polygon. Establish the accompanying Figure 4The coordinate space with the central point O as the unified origin. (Combined with the appendix...) Figure 3 The optical layout features shown depict the outer camera group, where the first group of cameras is positioned at each vertex of a regular polygon. Figure 3 The red node on the circumcircle is set to have a circumcircle radius parameter of 1. The second group of cameras is positioned at the tangent points where the inscribed circle of the regular polygon intersects with each edge, forming the inner camera group. (Corresponding to the attached...) Figure 3 The blue node on the inscribed circle has its inscribed circle radius parameter set to... According to the geometric tangent theorem, the positions of the inner camera group coincide perfectly with the midpoints of each side of the regular polygon in spatial coordinates.
[0055] Based on the aforementioned nested relationship between concentric circles and regular polygons, the equation constraining the side length of the interval between adjacent cameras is established as follows: ; In the formula, This represents the length of the line connecting two adjacent vertices of a regular polygon. Represents the radius of the circumcircle; This represents the radius of the inscribed circle. The circumscribed circle always encloses the inscribed circle in spatial relationships, and its topological structure always satisfies... This ensures that the result of the square root calculation is always positive, maintaining the rigor of the operational logic. The side length constraint equation clarifies the spatial span relationship between the inner and outer camera arrays.
[0056] Derive the array structure by substituting different numbers of camera nodes. (Combined with the attached...) Figure 3 The three geometric shapes shown, when When corresponding to the topology of a regular triangle, the ratio of the inner and outer circle radii satisfies .when When corresponding to the topology of a regular quadrilateral, the ratio of the inner and outer circle radii satisfies .when When corresponding to the topology of a regular pentagon, the ratio of the inner and outer circle radii satisfies .
[0057] The spatial layout of the array is constrained by multiple optical and geometric conditions. (Radius ratio values) A smaller camera size results in a shallower depth of field for the inner cameras, increasing the probability of target occlusion at close range and simultaneously increasing the hardware requirements for the calibration board size in the subsequent checkerboard calibration stage. The advantage is a smaller total number of camera nodes deployed in the system. Conversely, the radius ratio value... Increasing the size leads to a greater inner depth of field, a decrease in occlusion, and a reduction in the required calibration plate size, at the cost of an increase in the total number of hardware components. Taking into account optical occlusion limitations, calibration plate size parameters, and hardware costs, a regular quadrilateral topology is established as the optimal boundary for implementation, and a specific configuration is defined. Combined with the appendix Figure 4 The two-layer quadrilateral multi-viewpoint correction topology scheme shown has an outer layer containing four viewpoints A, B, C, and D, distributed at the four vertices of the square and all at a distance from the origin O. The inner layer contains four viewpoints, E, F, G, and H, located at the midpoints of the four sides of the square and at a distance from the origin O. In terms of spatial location, the inner viewpoint E is exactly the midpoint of the line connecting the outer viewpoints A and B, achieving a balance between the field of view and hardware configuration.
[0058] See attached document Figure 6 The revealed three-layer multi-viewpoint correction implementation environment, in order to adapt to the shooting needs of large-span dynamic scenes, involves extending a multi-layer concentric circle camera array outwards. Hierarchical physical acquisition array, with the number of layers constrained as follows: Extending a multi-level concentric circle array relies on the rule of reusing vertex and midpoint positions in adjacent layers. A three-layer structure example is introduced, with the first layer having four inner viewpoints E, F, G, and H, corresponding to the adjacent layers... Figure 6 The innermost blue node is located at a radius of On the concentric circles.
[0059] The second level is distributed with four mid-level viewpoints: A, B, C, and D, corresponding to the attached... Figure 6 The red node in the middle is located at the radius of the circumcircle. On the concentric circles.
[0060] The third layer has four outer viewpoints: I, J, K, and L, corresponding to the attached... Figure 6 The outermost green node is located at a radius of On the concentric circles.
[0061] The coordinates of the four viewpoints in the second level coincide exactly with the midpoints of the four sides of the square formed by the lines connecting the four viewpoints in the third level.
[0062] Based on the nesting rules, the four viewpoints A, B, C, and D of the second level coincide exactly with the midpoints of the four sides of the square formed by the lines connecting the outer viewpoints I, J, K, and L of the third level. According to the geometric nesting rules, the ratio of the radii of two adjacent extended levels satisfies... Following this recursive logic, the radius parameters of all concentric circles within the multi-level physical acquisition array... , , ,…, The common ratio is A geometric sequence. A wide-angle video acquisition matrix is constructed using trigonometric functions in a progressive scale, providing a spatial data benchmark for subsequent depth-of-field fusion between different layers.
[0063] After completing the spatial topology construction of the multi-layer concentric circle camera array established in the previous steps, this embodiment provides a detailed description of step S200. Multi-viewpoint parameter correction and depth-of-field reference calibration are performed on the networked camera array. A standard checkerboard correction board is placed at the center of the shooting area, triggering all nodes of the inner and outer camera groups to synchronously acquire checkerboard image sequences from different angles. The acquired image sequences are processed using a conventional checkerboard calibration algorithm to calculate the internal and external parameter matrices of a single camera, and spatial geometric distortion correction is performed accordingly. Simultaneously, color consistency correction is performed on the multi-viewpoint output images to unify chromaticity and brightness distribution characteristics, establishing the initial image reference for the front-end acquisition equipment.
[0064] After completing the equipment parameter calibration, the depth-of-field boundary parameters required for subsequent calculations are extracted. Considering the quadrilateral topology, the inner viewpoints are relatively close to the center of the shooting area. Close-range imaging allows the inner camera array to acquire a shallow depth-of-field range, suitable for capturing target details within a specific depth and minimizing background interference. The depth-of-field data output from the four viewpoints of the inner camera group after calibration are extracted, and the arithmetic mean is calculated. The result is defined as the inner average depth-of-field value. . Set as the lower limit reference parameter for shallow depth of field in subsequent mapping operations.
[0065] Analyzing the imaging state of the outer array, the four viewpoints of the outer layer are relatively far from the center of the shooting area. Long-distance imaging generates a large depth of field, suitable for capturing overall spatial scenes and large-scale motion trajectories. The depth of field data corresponding to the four viewpoints of the outer camera group after calibration and correction are extracted, and the arithmetic mean is calculated. This result is defined as the outer layer average depth of field value. . Set as the upper limit reference parameter for large depth of field in mapping operations. Establish the lower limit reference parameter. With upper limit benchmark parameters This constructs a complete boundary interval for viewpoint tracking mapping operations, transforming hardware optical characteristics into algebraic-level pre-calculation conditions.
[0066] The shallow depth of field lower limit benchmark calculated in step S200 With the upper limit of the depth of field benchmark This embodiment will describe step S300 in detail. A viewpoint capture device is configured on the video display terminal to continuously collect the spatial coordinate data of the observer. The system sets a predetermined time statistics window and continuously extracts the vertical distance data between the observer's eye and the plane where the display terminal is located within the time statistics window. The minimum observation distance value appearing within the time statistics window is recorded and defined as... The maximum observed distance value within the synchronous recording statistical period is defined as... The spatial location constraint interval for observers is constructed by periodically updating two boundary distance parameters. The observation distance value acquired at the current acquisition moment is defined as... The above distance parameter calculation process uniformly adopts mm as the standard unit.
[0067] See attached document Figure 5 The illustrated proportional calculation method transforms absolute spatial distance into a relative proportional parameter, eliminating the interference of differences in absolute distance values on the mapping operation. (Appendix) Figure 5 A linear mapping relationship based on the geometric features of similar triangles is constructed between the distance parameter in the horizontal dimension and the depth parameter in the vertical dimension. A normalized position scaling factor is set relative to the minimum and maximum observation distance values. The calculation model is set as the ratio of the deviation of the observed distance value from the minimum observed distance value to the deviation of the maximum observed distance value from the observed distance value. The corresponding position scaling factor formula is: ; In the formula, Represents the normalized position scaling factor; This represents the observed distance value at the current moment; This represents the minimum observation distance value within the statistical period; This represents the maximum observation distance value within the statistical period. It occurs when the observer is at the furthest observation location. Equivalent to At this time, the expression triggers a division exception with a denominator of 0. To avoid arithmetic overflow errors, an algebraic transformation needs to be performed on the original proportional mapping relationship to convert the denominator terms containing dynamic variables into difference terms containing only boundary constants.
[0068] According to the appendix Figure 5 The linear and proportional relationship between the displayed spatial distance values and the image depth parameters can be established using the following algebraic mapping equation: ; In the formula, This represents the average depth of field of the target calculated at the current moment; Represents the lower limit benchmark for shallow depth of field; This represents the upper limit benchmark for large depth of field. It incorporates the normalized position scaling factor. Substituting the formula into the algebraic mapping equation, and after algebraic simplification to eliminate the dynamic distance variable in the denominator, the analytical formula for the target's average depth of field is derived: ; In this formula, the definitions of the symbols for each parameter are completely consistent with those in the aforementioned formula. A conditional decision branch is added to the computation unit, and the detection time statistics window... Numerical and The difference in values. If the difference is less than the preset distance resolution threshold, which is used as an empirical constant, it is determined that the observer's displacement has not met the system's recognition criteria, and the computation center skips the algebraic mapping calculation. Equivalent to Alternatively, it can maintain the calculation result from the previous sampling time. The addition of a conditional decision branch avoids computational anomalies caused by a denominator of 0. Based on the target average depth-of-field analytical formula, the computation unit completes the dynamic depth-of-field parameter solution process and outputs the calculated target average depth-of-field value to the rendering node, providing pre-processing data for 3D video reconstruction.
[0069] The target average depth of field value calculated in step S300 This embodiment describes step S400 of the system execution. The computing node receives multiple low-level media streams transmitted by the front-end acquisition device in parallel. The shallow depth-of-field video image frame matrix, after parameter calibration and geometric distortion correction processing of the inner layer camera array, is extracted and used as the inner layer image matrix parameter. The outer camera array's large depth-of-field video image frame matrix, after undergoing the same correction process, is extracted synchronously and used as the outer image matrix parameter. To meet the data requirements for subsequent weighted calculations, the underlying data preprocessing module performs timestamp verification on the two sets of image matrices to ensure that the inner and outer layer data involved in the fusion are sampled at the same time, thus completing the alignment operation of the time dimension of the multi-source media streams. Simultaneously, spatial resolution consistency resampling is performed on the inner and outer layer image matrices to ensure that they have the same matrix dimensional characteristics.
[0070] The target average depth of field value obtained by dynamic solution As a control variable, an interpolation weight allocation mechanism for inner and outer layer image frames is constructed. Based on the relative position of the target's average depth of field value within the overall depth of field range, the corresponding inner layer image matrix parameters are calculated. Inner layer weighting factors and corresponding outer layer image matrix parameters The outer weighting factor. The specific interpolation and fusion weighting mathematical allocation equation is set as follows: ; ; In the formula, Represents the inner layer weighting factor; Represents the outer layer weighting factor; This represents the average depth of field value of the target calculated at the current sampling time; This represents the lower limit of shallow depth of field benchmark extracted from the inner system calibration; This represents the upper limit of the depth of field benchmark extracted by the outer system calibration. Combined with the spatial topology conditions established in step S200, the shooting distance of the outer camera array is always greater than that of the inner camera array, which, under arithmetic conditions, necessarily satisfies... The differences in optical parameters caused by spatial layout constraints result in changes in the denominator of the formula. The constant value greater than 0 avoids arithmetic anomalies caused by a zero denominator in division operations. The weighting factor operation transforms the dynamic changes in spatial depth values into dimensionless interpolation allocation ratios.
[0071] After obtaining the dynamically calculated inner and outer weight factors, the video rendering unit performs pixel-level weighted interpolation on the inner and outer image matrices that are at the same timestamp and dimensionally aligned. The weighted fusion analytical equation is defined based on the inner and outer weight factors as follows: ; In the formula, A matrix representing the rendered 3D video image output; Represents the dimensionless inner weighting factor; Represents the inner image matrix; Represents the dimensionless outer weighting factor; This represents the outer image matrix. When the observation distance decreases and the target depth value approaches the lower limit, the computational equation guides the output image to present a higher proportion of inner shallow depth-of-field features, suitable for the terminal to reproduce local motion information. When the target depth value approaches the upper limit, the equation operation increases the proportion of outer large depth-of-field panoramic features, suitable for the terminal to present motion trajectories with larger spans. A 3D video image matrix is generated based on the weighted fusion analytical equation, and the 3D reconstructed video is output.
[0072] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.
Claims
1. A method for correcting large depth of field and multiple viewpoints, characterized in that, include: A multi-layer concentric circle camera array is constructed, which is divided into an inner layer camera group and an outer layer camera group; Multi-viewpoint parameter correction and depth-of-field reference calibration are performed on the multi-layer concentric circle camera array. The inner average depth-of-field value corresponding to the inner camera group and the outer average depth-of-field value corresponding to the outer camera group are extracted. The inner average depth-of-field value is set as the shallow depth-of-field lower limit reference parameter, and the outer average depth-of-field value is set as the large depth-of-field upper limit reference parameter. A proportional mathematical mapping model is established using the observation distance value obtained at the current acquisition time, the shallow depth of field lower limit benchmark parameter, and the large depth of field upper limit benchmark parameter to calculate the target average depth of field value matched by the observation distance value. The target average depth of field value obtained by dynamic calculation is set as a control variable, and interpolation calculation is performed on the bottom layer video images acquired by the inner layer camera group and the outer layer camera group to output the three-dimensional reconstructed video.
2. The large depth-of-field multi-viewpoint correction method according to claim 1, characterized in that, The construction of the multi-layer concentric circle camera array includes: Establish a coordinate space with its central point as the unified origin; The outer camera group is constructed by placing the first group of cameras at each vertex of a regular polygon. The second group of cameras is arranged on the tangent points where the inscribed circle of the regular polygon intersects with each of its edges to construct the inner camera group; The inner camera group is positioned so that its position completely coincides with the midpoint of each side of the regular polygon in spatial coordinates.
3. The large depth-of-field multi-viewpoint correction method according to claim 1, characterized in that, The multi-layer concentric circle camera array is extended outward to construct a multi-level physical acquisition array; Extending a multi-level concentric circle array relies on the rule of reusing the positions of vertices and midpoints in adjacent layers; The radius parameters of all concentric circles within the multi-level physical acquisition array form a geometric sequence.
4. The large depth-of-field multi-viewpoint correction method according to claim 1, characterized in that, The process of performing multi-viewpoint parameter correction and depth-of-field reference calibration includes: Trigger all nodes of the inner and outer camera groups to synchronously acquire chessboard image sequences from different angles; The internal and external parameter matrices of a single camera are calculated, and spatial geometric distortion correction is performed accordingly. Simultaneously perform color consistency correction on multi-viewpoint output images; Extract the depth-of-field data output by the inner camera group after calibration and correction, calculate the arithmetic mean value, and define the calculation result as the inner average depth-of-field value. Extract the depth-of-field data output by the outer camera group after calibration and correction, calculate the arithmetic mean value, and define the calculation result as the outer average depth-of-field value.
5. The large depth-of-field multi-viewpoint correction method according to claim 1, characterized in that, Before using the observation distance value obtained at the current acquisition time, the following is also included: A time statistics window with a pre-defined duration; Within the time statistics window, continuously extract the vertical distance data between the human eye and the plane where the display terminal is located; Record the minimum observation distance value that appears within the time statistics window; The maximum observation distance value that occurs within the statistical period is recorded synchronously.
6. The large depth-of-field multi-viewpoint correction method according to claim 5, characterized in that, Calculating the target average depth of field value matched by the observation distance includes: Set a normalized position scaling factor for the observation distance value relative to the minimum and maximum observation distance values; The proportional mathematical mapping model is set as the ratio of the deviation of the observed distance value from the minimum observed distance value to the deviation of the maximum observed distance value from the observed distance value; Substitute the formula for the normalized position scaling factor into the algebraic mapping equation; After algebraic simplification to eliminate the dynamic distance variable in the denominator, the analytical formula for the target average depth of field is derived. The dynamic depth-of-field parameter solution process is completed based on the target average depth-of-field analytical formula, and the target average depth-of-field value is calculated.
7. The large depth-of-field multi-viewpoint correction method according to claim 6, characterized in that, Before deriving the analytical formula for the target average depth of field, the following is also included: Detect the difference between the maximum observation distance value and the minimum observation distance value within the time statistics window; If the difference is less than the preset distance resolution threshold, which is an empirical constant, the algebraic mapping calculation is skipped, and the target average depth of field value is equal to the large depth of field upper limit reference parameter, or the calculation result of the previous sampling time is maintained.
8. The method for correcting large depth of field and multiple viewpoints according to claim 1, characterized in that, Before performing interpolation calculations on the bottom-layer video images acquired by the inner and outer camera groups, the process also includes: The shallow depth-of-field video image frame matrix after parameter calibration and geometric distortion correction of the inner camera group is extracted and used as the inner image matrix parameter. Simultaneously extract the large depth-of-field video image frame matrix of the outer camera group after the same correction processing, and use it as the outer image matrix parameter; The parameters of the inner image matrix and the parameters of the outer image matrix are verified by timestamp. Spatial resolution consistent resampling is performed on the parameters of the inner image matrix and the parameters of the outer image matrix.
9. The large depth-of-field multi-viewpoint correction method according to claim 8, characterized in that, The interpolation calculation process includes: Based on the relative position of the target average depth of field value within the overall depth of field range, calculate the inner layer weight factor corresponding to the inner layer image matrix parameters and the outer layer weight factor corresponding to the outer layer image matrix parameters respectively. The inner layer weight factor and the outer layer weight factor are obtained by setting an interpolation fusion weight mathematical allocation equation.
10. The large depth-of-field multi-viewpoint correction method according to claim 9, characterized in that, The output 3D reconstructed video includes: Perform pixel-level weighted interpolation on the inner and outer image matrix parameters that are at the same timestamp and dimensionally aligned; A weighted fusion analytical equation is defined based on the inner weight factor and the outer weight factor; A three-dimensional video image matrix is generated based on the weighted fusion analytical equation, and the three-dimensional reconstructed video is output.
