Method and device for outputting video picture of trinocular camera

Abstract

The invention relates to the technical field of stereo matching, in particular to a three-eye camera video picture output method and device, and the method comprises the following steps: extracting information from a video picture of a three-eye camera, calculating the Gaussian curvature and average curvature of multiple points in an image, calculating differential geometric characteristics, recording curvature attributes, and generating a curved surface feature set. According to the method, parallax adjustment is carried out on the edge area of the object by applying high-order curvature information, more accurate depth information can be provided at the edge of the object through the edge optimization disparity map, the method is particularly key to reconstruction of a fine structure in a scene, and in the generation process of the global optimization disparity map, through parallax comparison and correction between continuous scene frames, more accurate depth information can be provided at the edge of the object. According to the method, the moving object or scene change can be processed more effectively, the continuity and accuracy of the disparity map are ensured, and the quality of three-dimensional reconstruction and the practicability of applications such as automatic driving and video monitoring are remarkably improved in a dynamic environment.

Description

Technical Field

[0001] The present invention relates to the field of stereo matching technology, and in particular to a method and a device for outputting video images of a three-eye camera. Background Art

[0002] Stereo matching technology is an important branch in the field of computer vision. It is mainly used to reconstruct three-dimensional scenes from images taken from different perspectives by two or more cameras. By using the principle of parallax, the depth information of the object is determined by comparing multiple two-dimensional images of the same scene, and the corresponding three-dimensional model is constructed. Stereo matching technology has a wide range of applications in the fields of self-driving cars, robot navigation, augmented reality, and three-dimensional scene reconstruction. The key technologies include image alignment, parallax calculation, and three-dimensional reconstruction, and also involve a large number of image processing and pattern recognition methods.

[0003] Among them, the output method of the three-eye camera video screen refers to using three cameras to capture the video screen, and processing the screen through a specific algorithm to generate a video output with depth information. The three-eye camera configuration enhances the stereoscopic vision effect by adding a camera to improve the accuracy and reliability of parallax measurement. Uses include improving the performance of video surveillance systems, providing more accurate obstacle detection and avoidance in autonomous vehicles, and creating a more realistic three-dimensional experience in virtual reality. With more accurate depth information, complex scenes can be better understood and interpreted.

[0004] Although existing stereo matching technology has made progress in processing multi-view images, it faces challenges in processing dynamic scenes and complex edge information. For example, in the fields of autonomous driving and video surveillance, existing technologies have difficulty in quickly and accurately calculating stable disparity information in scenes with fast-moving objects or large changes in light, limiting performance in real-time applications. Traditional methods do not adequately process disparity at the edges, resulting in distortion of three-dimensional information at the outline of the object, affecting the accuracy of the entire three-dimensional reconstruction. The shortcomings are particularly evident when processing scenes with a high dynamic range, resulting in incoherence and error accumulation in the three-dimensional model, affecting the application effect, such as misidentification of obstacles in autonomous driving or unrealistic scene reconstruction in virtual reality. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings in the prior art and to propose a method and device for outputting video images of a three-eye camera.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme, a method for outputting a video image of a three-eye camera, comprising the following steps:

[0007] S1: Extract information from the video images of the trinocular camera, calculate the Gaussian curvature and mean curvature of multiple points in the image, calculate the differential geometric characteristics, record the curvature properties, and generate the surface feature set;

[0008] S2: using the surface feature set, by analyzing the directionality and intensity parameters of the differentiated curvature points, adjusting the vector field parameters according to the surface features of the object, and obtaining an initial disparity vector field;

[0009] S3: using the initial disparity vector field, applying high-order curvature information to the edge area of ​​the object in the video image to perform disparity adjustment, and obtaining an edge optimized disparity map by processing the edge pixel curvature;

[0010] S4: using the edge optimized disparity map, adjusting the local optimized disparity to a smooth disparity distribution in the entire image range, and generating a global optimized disparity map by comparing the disparities between consecutive scene frames and correcting the disparity values;

[0011] S5: analyzing the trend of the parallax change over time on the image plane by using the global optimization disparity map and using the wave equation, calculating and quantifying the dynamically changing parallax by tracking the fluctuation of the disparity value of each frame, and establishing a wave equation model;

[0012] S6: According to the wave equation model, by solving the real-time change of disparity, adjusting the global optimization disparity map according to the analysis result, updating the disparity adjustment strategy to match the scene and object movement, and obtaining the video picture output log.

[0013] As a further solution of the present invention, the surface feature set includes Gaussian curvature values, mean curvature values ​​and associated differential geometry characteristic data, the initial disparity vector field includes vector direction, intensity and distribution parameters adjusted according to the surface features of the object, the edge optimized disparity map includes the disparity adjustment results of the edge area, the optimized edge curvature values ​​and disparity distribution information, the global optimized disparity map includes the smoothed disparity distribution in the entire image, the disparity value correction results and the disparity continuity data, the wave equation model includes disparity fluctuation analysis parameters, time series change data and a quantitative model of dynamic change of disparity, and the video screen output log includes real-time update data of the disparity adjustment strategy, the matching results of the scene and object motion and the correction record of the global disparity map.

[0014] As a further solution of the present invention, the steps of extracting information from the video image of the trinocular camera, calculating the Gaussian curvature and mean curvature of multiple points in the image, calculating the differential geometric properties, recording the curvature attributes, and generating the surface feature set are specifically as follows:

[0015] S101: extract frame images from the video screen of the three-eye camera, use color correction to adjust the image color temperature and contrast, perform edge detection to strengthen the image edge, optimize the image resolution and clarity, and generate a processed image;

[0016] S102: Based on the processed image, locate feature points in the image using spatial coordinate transformation, apply a local curvature calculation formula to each feature point, calculate Gaussian curvature and average curvature, record the curvature value of each point, and generate curvature analysis data;

[0017] S103: using the curvature analysis data, comparing the curvature value differences of the differentiated feature points, analyzing the differential geometry characteristics, drawing the surface graph of each curvature point, and generating a surface feature set.

[0018] As a further solution of the present invention, the steps of using the surface feature set, analyzing the directionality and intensity parameters of the differentiated curvature points, adjusting the vector field parameters according to the surface features of the object, and obtaining the initial disparity vector field are specifically as follows:

[0019] S201: Based on the surface feature set, analyzing the directionality and intensity characteristics of each curvature point, recording the coordinates and curvature values ​​of the curvature point, and generating differentiated curvature point analysis results;

[0020] S202: adjusting the direction and strength of the curvature point vector according to the differentiated curvature point analysis result, verifying that the parameters reflect the physical characteristics of the curvature point, and generating adjusted vector field parameters;

[0021] S203: Using the adjusted vector field parameters, summarize and analyze the geometrical morphology of the surface of the object monitored by the three-eye camera, record the features of the surface of the object, and construct an initial disparity vector field.

[0022] As a further solution of the present invention, the initial disparity vector field is used to apply high-order curvature information to the edge area of ​​the object in the video screen for disparity adjustment, and the step of obtaining the edge optimized disparity map by processing the edge pixel curvature is specifically as follows:

[0023] S301: Based on the initial disparity vector field, identifying the edge region of the object in the video picture, extracting edge pixels in the region, and recording associated disparity vector information to generate disparity data of the edge region;

[0024] S302: using the edge area disparity data, performing high-order information analysis on the curvature of each edge pixel, adjusting the disparity value to match the real-time geometric curvature of the object, and generating adjusted edge disparity data;

[0025] S303: using the adjusted edge disparity data, comparing and analyzing the degree of deviation and the cause of deviation between the disparity in the video and the real-time picture, verifying the degree of disparity optimization in the edge area, and generating an edge optimized disparity map.

[0026] As a further solution of the present invention, the edge optimized disparity map is used to adjust the local optimized disparity to a smooth disparity distribution in the entire image range, and the steps of generating a global optimized disparity map by comparing the disparity between consecutive scene frames and correcting the disparity value are specifically as follows:

[0027] S401: Based on the edge optimized disparity map, analyzing the disparity distribution of the entire video frame, expanding the locally optimized disparity data to the entire image range, checking the continuity and smoothness of the global disparity, and generating global disparity initialization data;

[0028] S402: using the global disparity initialization data, comparing disparity data between consecutive scene frames, identifying a disparity abnormality area that needs to be corrected, adjusting the area, evaluating the consistency of the disparity, and generating corrected global disparity data;

[0029] S403: Using the corrected global disparity data, record information of multiple objects in the video panorama, verify the consistency of disparity distribution, and generate a global optimized disparity map.

[0030] As a further solution of the present invention, by using the global optimization disparity map and the wave equation, the trend of disparity change over time on the image plane is analyzed, and by tracking the fluctuation of the disparity value of each frame, the dynamically changing disparity is calculated and quantified. The steps of establishing the wave equation model are specifically as follows:

[0031] S501: Based on the global optimized disparity map, analyzing the distribution trend of disparity values ​​on the image plane, tracking the change of disparity values ​​over time, recording dynamic data of each frame disparity, and generating disparity time series data;

[0032] S502: using the disparity time series data, adopting the Fourier transform method, by decomposing the frequency components of the disparity data, quantitatively analyzing the fluctuation characteristics of the disparity value, calculating the disparity fluctuation rate of the differentiated time periods, and generating disparity fluctuation analysis data;

[0033] S503: According to the parallax fluctuation analysis data, the wave equation is applied to analyze and record the dynamic behavior of the parallax on the image plane according to time changes, verify the degree of analysis of the dynamic change of the parallax, and construct a wave equation model.

[0034] As a further solution of the present invention, the formula of the Fourier transform method is as follows:

[0035]

[0036] Among them, R Δ is the parallax volatility, d i represents the disparity data value at the i-th time period, represents the time series average of the disparity data, k1, k2 and k3 are the adjustment coefficient of the real-time value of the disparity data, the adjustment coefficient of the average value and the coefficient of the normalized disparity volatility, respectively, and n is the total number of data points.

[0037] As a further solution of the present invention, according to the wave equation model, by solving the real-time change of disparity, adjusting the global optimization disparity map according to the analysis result, updating the disparity adjustment strategy to match the scene and object movement, the steps of obtaining the video screen output log are specifically as follows:

[0038] S601: Based on the wave equation model, solving the real-time change of disparity in continuous video frames, analyzing the dynamic behavior of disparity between differentiated video frames, recording disparity analysis results, and generating real-time disparity analysis data;

[0039] S602: using the real-time disparity analysis data, adjusting the global optimization disparity map to match the scene changes and the movement of the object, verifying the consistency of the disparity map with the video content, adjusting the strategy to match the dynamic scene, and generating an adjusted disparity map;

[0040] S603: updating the disparity adjustment strategy according to the adjusted disparity map, analyzing the correlation between the real-time scene and the object motion, checking the continuous adaptability of the disparity adjustment, and generating a video screen output log.

[0041] A device for outputting a video picture of a three-eye camera, the device for outputting a video picture of a three-eye camera is used to execute the above-mentioned method for outputting a video picture of a three-eye camera, and the device comprises:

[0042] The attribute analysis module analyzes the Gaussian curvature and mean curvature of each pixel in the video image of the trinocular camera, extracts differential geometry characteristics from the curvature data, records the curvature attributes of the pixel points, and obtains the curvature attribute set;

[0043] The vector field construction module uses the curvature attribute set to analyze the directionality and intensity parameters of the differential curvature points, record the surface features of the object, and construct an initial disparity vector field;

[0044] The disparity adjustment module uses the initial disparity vector field to apply high-order curvature information to the edge area of ​​the object in the video screen to perform disparity adjustment, process the edge pixel curvature, and form an edge optimized disparity map;

[0045] The disparity comparison module uses the edge optimized disparity map to analyze the smooth disparity distribution within the entire image, and corrects the disparity value by comparing the disparity between consecutive scene frames to obtain a global optimized disparity distribution map;

[0046] The dynamic disparity analysis module analyzes the temporal variation trend of the disparity on the image plane according to the global optimized disparity distribution map, tracks the fluctuation of the disparity value of each frame, updates the disparity adjustment strategy to match the scene and object motion, and generates a video picture output log.

[0047] Compared with the prior art, the advantages and positive effects of the present invention are:

[0048] In the present invention, by refining and expanding the disparity vector field, Gaussian curvature and mean curvature calculations are used when extracting and analyzing curvature properties, so that the surface features of the object are captured more accurately. By applying high-order curvature information to the edge area of ​​the object for disparity adjustment, the edge-optimized disparity map can provide more accurate depth information at the edge of the object, which is particularly critical for the reconstruction of fine structures in the scene. In the process of generating the global optimized disparity map, by comparing and correcting the disparity between consecutive scene frames, moving objects or scene changes can be more effectively processed to ensure the consistency and accuracy of the disparity map. By analyzing the trend of disparity changes over time through wave equations, the adaptability to dynamic scenes is enhanced, and the consistency and stability of the disparity map in the entire video sequence are significantly improved. In a dynamic environment, the quality of three-dimensional reconstruction and the practicality of applications such as autonomous driving and video surveillance are significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a schematic diagram of the workflow of the present invention;

[0050] Figure 2 This is a detailed flow chart of S1 of the present invention;

[0051] Figure 3 This is a detailed flow chart of S2 of the present invention;

[0052] Figure 4 This is a detailed flow chart of S3 of the present invention;

[0053] Figure 5 This is a detailed flow chart of S4 of the present invention;

[0054] Figure 6 This is a detailed flow chart of S5 of the present invention;

[0055] Figure 7 This is a detailed flow chart of S6 of the present invention;

[0056] Figure 8 It is a flow chart of the device of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0058] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0059] See also Figure 1 The present invention provides a technical solution, a method for outputting a video image of a three-eye camera, comprising the following steps:

[0060] S1: Extract information from the video images of the trinocular camera, calculate the Gaussian curvature and mean curvature of multiple points in the image, calculate the differential geometric characteristics, record the curvature properties, and generate the surface feature set;

[0061] S2: Using the surface feature set, by analyzing the directionality and intensity parameters of the differentiated curvature points, the vector field parameters are adjusted according to the surface features of the object to obtain the initial disparity vector field;

[0062] S3: using the initial disparity vector field, applying high-order curvature information to the edge area of ​​the object in the video screen to perform disparity adjustment, and by processing the edge pixel curvature to avoid the error of disparity estimation, obtain an edge optimized disparity map;

[0063] S4: Use the edge optimized disparity map to perform global optimization of the trinocular camera video image, adjust the local optimized disparity to a smooth disparity distribution in the entire image range, compare the disparity between consecutive scene frames, and correct the disparity value to generate a global optimized disparity map;

[0064] S5: By globally optimizing the disparity map and using the wave equation, the trend of disparity change over time on the image plane is analyzed. By tracking the fluctuation of the disparity value of each frame, the disparity model is continuously updated, and the dynamically changing disparity is calculated and quantified to establish a wave equation model.

[0065] S6: According to the wave equation model, by solving the real-time change of disparity, adjusting the global optimized disparity map according to the analysis results, and synchronously updating the disparity adjustment strategy to match the scene and object movement, the video screen output log is obtained.

[0066] The surface feature set includes Gaussian curvature values, mean curvature values ​​and associated differential geometry characteristic data. The initial disparity vector field includes vector direction, intensity and distribution parameters adjusted according to the surface features of the object. The edge optimized disparity map includes the disparity adjustment results of the edge area, the optimized edge curvature values ​​and disparity distribution information. The global optimized disparity map includes the smoothed disparity distribution in the entire image, the disparity value correction results and the disparity continuity data. The wave equation model includes disparity fluctuation analysis parameters, time series change data and a quantitative model of dynamic changes in disparity. The video screen output log includes real-time update data of the disparity adjustment strategy, the matching results of the scene and object motion and the correction record of the global disparity map.

[0067] See also Figure 2 , extract information from the video screen of the trinocular camera, calculate the Gaussian curvature and mean curvature of multiple points in the image, calculate the differential geometric characteristics, record the curvature properties, and generate the surface feature set in the following steps:

[0068] S101: extract frame images from the video screen of the three-eye camera, use color correction to adjust the image color temperature and contrast, perform edge detection to strengthen the image edge, optimize the image resolution and clarity, and generate the processed image. The execution process is as follows;

[0069] When extracting frame images from the video screen of the three-eye camera, it is necessary to synchronize the video source to ensure the continuity and integrity of the frame. By adjusting the frequency and resolution of video acquisition, the frame extraction process is optimized, the data processing efficiency is improved, and the extracted image is color corrected, including adjusting the color temperature and contrast to improve the visual effect of the image. The adjustment is fine-tuned based on the lighting conditions in the specific scene and the color response characteristics of the camera. The adjustment process is implemented using color correction algorithms, such as white balance algorithms and gamma correction algorithms, to ensure the natural reality of the image color and the preservation of details. Edge detection and image resolution optimization are achieved through specific image processing algorithms, such as Canny edge detection and super-resolution technology, to enhance the clarity of the image edge and improve the overall image quality to generate a processed image.

[0070] S102: Based on the processed image, the feature points in the image are located by using spatial coordinate transformation, the local curvature calculation formula is applied to each feature point, the Gaussian curvature and the average curvature are calculated, the curvature value of each point is recorded, and the execution process of generating curvature analysis data is as follows;

[0071] Based on the processed image, the spatial coordinate transformation and positioning of the feature points are performed according to the formula Calculate the Gaussian curvature and mean curvature of the feature points. In the formula, E represents the E coefficient in the first basic form, F represents the mixed term, G represents the G coefficient in the first basic form, and L represents the L coefficient in the second basic form.

[0072] N b Represents N in the second fundamental form b Coefficient. Detailed explanation of the formula and the process of formula calculation: It is the quotient of the determinant in the form of the second-order partial derivative and the determinant in the first basic form, specifically Where M is the mixed term in the second basic form. In the calculation example, if E = 0.5, F = 0, G = 0.5, L = 0.1, N b =0.1, M = 0, then K G It can be calculated as (0.1×0.1-0 2 ) / (0.5×0.5-0 2 )=0.4. This indicates that the surface has a smaller local curvature at this point.

[0073] S103: Using the curvature analysis data, comparing the curvature value differences of the differentiated feature points, analyzing the differential geometry characteristics, drawing the surface graph of each curvature point, and generating the surface feature set. The execution process is as follows;

[0074] Using curvature analysis data, we compare the differences in curvature values ​​of differentiated feature points, including statistical analysis and classification of the curvature of each feature point. By quantitatively analyzing the curvature values ​​of different feature points, we can determine the differences in differential geometric properties. Data analysis requires sorting and clustering the curvature values ​​through algorithms in order to discover regularities or anomalies. The analyzed data will be used to generate surface graphics for each feature point, which requires surface modeling technology, such as Bezier surfaces or NURBS. Mathematical models are used to accurately depict the three-dimensional geometric shape of each feature point, and a surface feature set is generated through careful analysis.

[0075] See also Figure 3 , using the surface feature set, by analyzing the directionality and intensity parameters of the differentiated curvature points, adjusting the vector field parameters according to the surface features of the object, the steps to obtain the initial disparity vector field are as follows:

[0076] S201: Based on the surface feature set, the directionality and intensity characteristics of each curvature point are analyzed, the coordinates and curvature values ​​of the curvature point are recorded, and the execution process of generating the differentiated curvature point analysis results is as follows;

[0077] When analyzing the directionality and intensity characteristics of each curvature point based on the surface feature set, it is necessary to accurately measure the spatial position of each curvature point, including using three-dimensional coordinates to record the coordinates of each point, and determining the curvature value through advanced calculation methods. The process involves mathematical modeling and analytical solution of the local surface. The directionality and intensity characteristics of the curvature points can be clearly expressed. Analyzing the distribution of curvature points and the distribution of curvature size can better understand the geometric characteristics of the object surface, provide a scientific basis for subsequent physical property analysis and modeling, and generate differentiated curvature point analysis results.

[0078] S202: By differentiating the curvature point analysis results, adjusting the direction and strength of the curvature point vector, verifying that the parameters reflect the physical characteristics of the curvature point, and generating the adjusted vector field parameters. The execution process is as follows;

[0079] By differentiating the results of curvature point analysis and adjusting the direction and strength of the curvature point vector, a series of parameter adjustment and verification processes are involved. The parameters reflect the physical characteristics of the curvature point. For example, by changing the angle and size of the vector, the mechanical response of physical phenomena such as bending or twisting can be more accurately simulated. The adjusted vector field parameters need to be rigorously verified, including comparing the differences between actual measured data and simulated data to ensure the accuracy and adaptability of the vector field. The adjustment is achieved through complex mathematical models and calculation processes, which lays the foundation for the next step of geometric morphology analysis and generates adjusted vector field parameters.

[0080] S203: Using the adjusted vector field parameters, summarizing and analyzing the geometrical morphology of the surface of the object monitored by the three-eye camera, recording the features of the surface of the object, and constructing the initial disparity vector field. The execution process is as follows;

[0081] The adjusted vector field parameters are used to analyze the geometric shape of the surface of the object monitored by the three-eye camera. According to the formula V = S·N a , calculate the disparity vector field. In the formula, V represents the disparity vector, S represents the geometric parameters of the object surface, and N a Represents the normal vector. Detailed explanation of the formula and the process of formula calculation: In stereoscopic vision, the disparity vector field V is used to describe the geometric deviation of an object at different viewpoints, where S and N a Represent the surface features and surface normals of the object, respectively. The parameters can be extracted from the measured data of the object surface, such as images acquired using a trinocular camera. For example, if in an experiment, S is the differential area size of the surface 0.2 square meters, N a is the normal vector of the region, and the normal direction is the unit vector (1, 1, 1), then the disparity vector V can be calculated as 0.2×(1, 1, 1)=(0.2, 0.2, 0.2). This indicates that the disparity is evenly distributed in all directions, indicating that the surface of the object is relatively flat in the region.

[0082] See also Figure 4 , using the initial disparity vector field, applying high-order curvature information to the edge area of ​​the object in the video screen for disparity adjustment, and processing the edge pixel curvature to obtain the edge optimized disparity map are specifically as follows:

[0083] S301: Based on the initial disparity vector field, identify the edge area of ​​the object in the video picture, extract the edge pixels in the area, and record the associated disparity vector information. The execution process of generating disparity data of the edge area is as follows;

[0084] Identify the edge area of ​​the object in the video screen and record the associated disparity vector information according to the formula Calculate the disparity data of the edge area. In the formula, Represents the disparity data of the edge area, and represent the horizontal and vertical components of the disparity vector, respectively. and is the parallax vector reference value. Detailed explanation of the formula and the process of formula calculation: Parallax data It is calculated based on the distance between the disparity vectors at different positions in the video and the reference disparity vector, and is used to represent the position change of the edge image. For example, setting the reference disparity vector and are all 0 (i.e. the reference point is at the center of the image), while the disparity vector of the edge pixel is and is (0.3, 0.4), then Can be The results show that the disparity of edge pixels relative to the image center is 0.5 units, reflecting the relative depth between the edge of the object and the image center.

[0085] S302: using the edge area disparity data, performing high-order information analysis on the curvature of each edge pixel, adjusting the disparity value to match the real-time geometric curvature of the object, and generating the adjusted edge disparity data, the execution process is as follows;

[0086] When using edge area disparity data to perform high-order information analysis on the curvature of each edge pixel, it involves detailed mathematical processing and calculation of curvature-related geometric data. It is necessary to determine the curvature characteristics in three-dimensional space through the spatial distribution and disparity values ​​of edge pixels, including the application of advanced curvature analysis methods such as Gaussian curvature and mean curvature calculation methods. By accurately measuring the disparity values ​​and performing mathematical modeling, the disparity values ​​are adjusted to match the real-time geometric curvature of the object. The mathematical models and algorithms used in the adjustment process ensure that the disparity values ​​accurately reflect the physical properties of the object, thereby generating adjusted edge disparity data.

[0087] S303: using the adjusted edge disparity data, comparing and analyzing the deviation degree and deviation cause of the disparity in the video and the real-time picture, verifying the disparity optimization degree of the edge area, and generating the edge optimization disparity map. The execution process is as follows;

[0088] Using the adjusted edge disparity data, by calculating the statistical deviation of the disparity value, and using statistical methods to compare and analyze the deviation between the disparity data and the real-time picture, the process relies on batch image data processing technology and edge detection algorithm, and uses Canny or Sobel algorithm to detect the edge of the video frame, identify the main edges in the image, and improve the disparity accuracy of the area through edge enhancement technology. The statistical method includes the calculation of mean, variance and standard deviation, which is used to measure and optimize the degree of deviation, realize fine adjustment of the disparity, show the disparity optimization effect after technical improvement, and generate edge optimized disparity map.

[0089] See also Figure 5 , using the edge optimized disparity map, adjust the local optimized disparity to a smooth disparity distribution of the entire image range, compare the disparity between consecutive scene frames, and correct the disparity value to generate the global optimized disparity map in the following steps:

[0090] S401: Based on the edge optimized disparity map, analyze the disparity distribution of the entire video frame, expand the locally optimized disparity data to the entire image range, check the continuity and smoothness of the global disparity, and generate the global disparity initialization data. The execution process is as follows;

[0091] Analyzing the disparity distribution of the entire video frame based on the edge-optimized disparity map involves the process of expanding the locally optimized disparity data to the entire image, including evaluating the continuity and smoothness of the disparity in each video frame through an algorithm, and interpolating and smoothing the disparity data to ensure the uniform distribution of the disparity in the entire image. This requires the use of advanced image processing techniques and mathematical models, such as curve fitting and surface reconstruction techniques, which can effectively eliminate outliers and jumps in the disparity map, ensure the natural transition of disparity data across the entire image, and generate global disparity initialization data.

[0092] S402: using global disparity initialization data, by comparing disparity data between consecutive scene frames, identifying the disparity abnormal area that needs to be corrected, adjusting the area, evaluating the consistency of the disparity, and generating the corrected global disparity data. The execution process is as follows;

[0093] The global disparity initialization data is used to compare the disparity data between consecutive scene frames according to the formula Identify the parallax abnormality area that needs to be corrected. Where Δd represents the change in parallax, d x1 and d y1 Represents the disparity coordinates of the previous frame, dx2 and d y2 Represents the disparity coordinates of the next frame. Detailed explanation of the formula and the derivation process of the formula calculation: In the analysis of consecutive frames of the disparity map, the consistency evaluation of the disparity is very important. For example, if the disparity coordinates of the previous frame are (0.1, 0.2), and the disparity coordinates of the next frame become (0.3, 0.4), the change in disparity can be calculated as

[0094] Calculating the magnitude of the disparity change from one frame to the next helps identify anomalous areas in the disparity data that need to be adjusted to improve disparity consistency.

[0095] S403: Using the corrected global disparity data, recording the information of multiple objects in the video panorama, verifying the consistency of the disparity distribution, and generating a global optimized disparity map. The execution process is as follows;

[0096] The key to using the corrected global disparity data is to record the information of multiple objects in the video panorama and verify the consistency of the disparity distribution. The process requires the application of advanced image analysis technology. By comprehensively processing and optimizing the corrected global disparity data, the uniform distribution and accuracy of the disparity data in the entire video picture are ensured, including the use of image segmentation and object recognition technology to refine the boundaries and positions of each object, ensuring that the actual distance and position of the object can be accurately reflected in each part, providing higher accuracy and reliability for subsequent visual effect analysis and three-dimensional reconstruction, and generating a globally optimized disparity map.

[0097] See also Figure 6 ,Through global optimization of the disparity map, the wave equation is used to analyze the trend of the disparity on the image plane according to time, the dynamic change of the disparity value is calculated and quantified,by tracking the fluctuation of the disparity value of each frame. The specific steps of establishing the wave equation model are:

[0098] S501: Based on the global optimized disparity map, the distribution trend of the disparity value on the image plane is analyzed, the change of the disparity value over time is tracked, the dynamic data of each frame disparity is recorded, and the execution flow of generating the disparity time series data is as follows;

[0099] Based on the global optimized disparity map, when analyzing the distribution trend of disparity values ​​on the image plane, it is necessary to track the changes of disparity values ​​over time and record the dynamic data of each frame of disparity, including detailed analysis of the disparity time series through advanced data analysis and image processing techniques to determine the changing pattern of disparity values ​​in consecutive video frames. The process requires the use of time series analysis methods, such as autoregressive models or moving average technology, to accurately simulate and predict the time change trend of disparity values ​​and generate disparity time series data.

[0100] S502: Using the disparity time series data, adopting the Fourier transform method, by decomposing the frequency components of the disparity data, quantitatively analyzing the fluctuation characteristics of the disparity value, calculating the disparity fluctuation rate of the differentiated time period, and generating the disparity fluctuation analysis data, the execution process is as follows;

[0101] The formula for the Fourier transform method is as follows:

[0102]

[0103] Among them, R Δ is the parallax volatility, d i represents the disparity data value at the i-th time period, represents the time series average of the disparity data, k1, k2 and k3 are the adjustment coefficient of the real-time value of the disparity data, the adjustment coefficient of the average value and the coefficient of the normalized disparity volatility, respectively, and n is the total number of data points.

[0104] formula:

[0105]

[0106] Formula parameter settings and sources:

[0107] d i This is the disparity data value at the i-th time period, which comes from an actual disparity measurement device, such as a stereo camera or a laser scanner, which records disparity data at different time points.

[0108] is the time series average of the disparity data, calculated by adding all observed disparity values ​​d i Add up the total and divide by the total number of data points n.

[0109] k1, k2 and k3 are weights and adjustment coefficients used to adjust the impact of disparity data and normalize the data. The weights k1 and k2 are located according to the variability of the disparity data and are obtained through data analysis; k3 is to make the results have certain standardized or normalized characteristics to avoid difficulties in interpreting data due to different scales.

[0110] Parameter setting basis:

[0111] k1=1.2, considering d i The effect on the results is slightly larger to increase the sensitivity of the model to mutations.

[0112] k2=0.8, balances the impact of long-term average parallax, and is slightly lower to reduce the suppression of short-term fluctuations by long-term data.

[0113] k3=2.0, 2.0 is selected to ensure that the result is between 0 and 1, which is convenient for analysis and comparison.

[0114] Specific examples and derivation process:

[0115] Set to collect 5 disparity data points in a period of time, which are d=

[0116] 2.0, 2.5, 3.0, 2.8, 2.2], calculate the average:

[0117] 1. Calculate the adjustment value of each item using the given weights:

[0118]

[0119] 2. Calculate the parallax volatility R Δ :

[0120]

[0121] The results show that within a given time period, the average fluctuation rate of the disparity values ​​is 0.5, indicating that the disparity values ​​fluctuate by 50% relative to the average disparity. The results help to understand the stability and volatility of disparity data. The values ​​are further used to evaluate the performance of disparity measurement devices or to adjust parallax-related computational models.

[0122] S503: According to the parallax fluctuation analysis data, the wave equation is applied to analyze and record the dynamic behavior of the parallax on the image plane according to the time change, and the degree of analysis of the dynamic change of the parallax is verified. The execution process of constructing the wave equation model is as follows;

[0123] The wave equation is used to analyze the dynamic behavior of parallax changing with time on the image plane. According to the formula Analyze parallax fluctuations. Where ψ represents the parallax value function, is the second-order derivative of the parallax value function with respect to time, indicating the acceleration of the parallax over time, c is the wave propagation speed, is the spatial Laplace operator of the disparity value function. Formula explanation and formula calculation derivation process: The wave equation is a mathematical model that describes how the disparity value propagates over time on the image plane. For example, if the disparity value function ψ(x a ,y a ,t a ) at time t a and position (x a ,y a ) is known, the dynamic change of parallax value can be calculated by wave equation. Set the parallax value function to ψ(x a ,y a ,t a )=Asin(kx a -ωt a), where A is the amplitude, k is the wave number, and ω is the angular frequency. Performing quadratic time derivatives and spatial Laplace operations on the function can verify whether the solution to the wave equation is consistent with the actual observation data and verify the law of change of the parallax value over time and space. The process not only provides an in-depth understanding of the dynamic changes of the parallax, but also helps to build a more accurate parallax wave model.

[0124] See also Figure 7 According to the wave equation model, by solving the real-time change of disparity, adjusting the global optimization disparity map according to the analysis results, updating the disparity adjustment strategy to match the scene and object movement, the steps to obtain the video screen output log are as follows:

[0125] S601: Based on the wave equation model, solve the real-time change of disparity in continuous video frames, analyze the dynamic behavior of disparity between differentiated video frames, record the disparity analysis results, and generate real-time disparity analysis data. The execution process is as follows;

[0126] Based on the wave equation model, the real-time change of parallax in continuous video frames is solved. According to the formula Analyze the dynamic behavior of the disparity between differentiated video frames. Where ψ represents the disparity value function, is the first-order derivative of the parallax value function with respect to time, indicating the rate of change of the parallax over time, and v represents the parallax fluctuation speed. is the spatial gradient of the disparity value function. Formula explanation and formula calculation derivation process: The formula is in the form of a transport equation, which is used to describe how the disparity value is dynamically transmitted and changed with the video frame. Set at the initial time t a , the disparity value function ψ(x a ,y a ,t a ) is known, and the parallax fluctuation speed v is a constant, the change of parallax in consecutive frames can be simulated by solving the equation. For example, if the initial distribution of parallax values ​​is Where A and σ are the standard deviations of amplitude and spatial distribution, respectively. Through iterative calculation, the distribution of disparity values ​​at subsequent time points can be obtained, providing accurate analysis and prediction of dynamic changes in disparity, helping to generate real-time disparity analysis data, and providing a basis for video processing and enhancement.

[0127] S602: Using real-time disparity analysis data, adjusting the global optimization disparity map to match scene changes and object movements, verifying the consistency of the disparity map with the video content, adjusting the strategy to match dynamic scenes, and generating the adjusted disparity map. The execution process is as follows;

[0128] Using real-time disparity analysis data to adjust the global optimized disparity map to match scene changes and object movement involves complex image processing and data analysis techniques. By comparing the disparity analysis data with the actual video frame content, the consistency between the disparity map and the video content is evaluated. Advanced image registration and disparity adjustment algorithms, such as the optical flow method, are required to ensure that the disparity map accurately reflects the scene changes and object dynamics in the video. The strategy is adjusted to match the dynamic scene, ensure the global optimization and accuracy of the disparity map, and generate an adjusted disparity map.

[0129] S603: According to the adjusted disparity map, the disparity adjustment strategy is updated, the correlation between the real-time scene and the object movement is analyzed, the continuous adaptability of the disparity adjustment is checked, and the execution process of generating the video screen output log is as follows;

[0130] When updating the parallax adjustment strategy based on the adjusted parallax map, it is necessary to analyze the correlation between the real-time scene and object motion, and check the continuous adaptability of the parallax adjustment, including using dynamic tracking and machine learning technology to continuously analyze scene changes, adjust the parallax parameters to adapt to the changing scene and object motion, record all adjustments and results, and provide feedback on the adjustment effect and parallax. Figure 1 The system provides consistent and detailed feedback to ensure that the disparity map can continuously and accurately reflect the three-dimensional structure of the video content, which is of great significance for real-time video analysis and augmented reality applications, and generates video screen output logs.

[0131] See also Figure 8 , a device for outputting a video picture of a three-eye camera, the device for outputting a video picture of a three-eye camera is used to execute the above-mentioned method for outputting a video picture of a three-eye camera, and the device comprises:

[0132] The attribute analysis module analyzes the Gaussian curvature and mean curvature of each pixel in the video image of the trinocular camera, extracts differential geometry characteristics from the curvature data, records the curvature attributes of the pixel points, and obtains the curvature attribute set;

[0133] The vector field construction module uses the curvature attribute set to parse the directionality and intensity parameters of the differentiated curvature points, record the surface features of the object, and construct the initial disparity vector field;

[0134] The disparity adjustment module uses the initial disparity vector field, applies high-order curvature information to the edge area of ​​the object in the video screen to perform disparity adjustment, processes the edge pixel curvature, and forms an edge optimized disparity map;

[0135] The disparity comparison module uses edge-optimized disparity maps to analyze the smooth disparity distribution within the entire image. By comparing the disparity between consecutive scene frames, the disparity value is corrected to obtain a globally optimized disparity distribution map.

[0136] The dynamic disparity analysis module analyzes the temporal variation trend of the disparity on the image plane based on the global optimized disparity distribution map, tracks the fluctuation of the disparity value of each frame, updates the disparity adjustment strategy to match the scene and object motion, and generates a video screen output log.

[0137] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A method for outputting a video image of a three-eye camera, characterized in that: The following steps are involved: Extract information from the video images of the trinocular camera, calculate the Gaussian curvature and mean curvature of multiple points in the image, calculate the differential geometric characteristics, record the curvature properties, and generate a surface feature set; The surface feature set is used to analyze the directionality and intensity parameters of the differential curvature points, and the vector field parameters are adjusted according to the surface features of the object to obtain an initial disparity vector field; Using the initial disparity vector field, applying high-order curvature information to the edge region of the object in the video screen to perform disparity adjustment, and obtaining an edge optimized disparity map by processing the edge pixel curvature; Using the edge optimized disparity map, adjusting the local optimized disparity to a smooth disparity distribution in the entire image range, and generating a global optimized disparity map by comparing the disparities between consecutive scene frames and correcting the disparity values; By using the global optimization disparity map and the wave equation, the trend of the disparity change on the image plane according to time is analyzed, and by tracking the fluctuation of the disparity value of each frame, the dynamically changing disparity is calculated and quantified to establish a wave equation model; According to the wave equation model, by solving the real-time change of disparity, adjusting the global optimization disparity map according to the analysis result, updating the disparity adjustment strategy to match the scene and object movement, the video picture output log is obtained.

2. The method for outputting a three-eye camera video image according to claim 1, characterized in that: The surface feature set includes Gaussian curvature values, mean curvature values ​​and associated differential geometry characteristic data; the initial disparity vector field includes vector direction, intensity and distribution parameters adjusted according to the surface features of the object; the edge optimized disparity map includes the disparity adjustment result of the edge area, the optimized edge curvature value and disparity distribution information; the global optimized disparity map includes the smoothed disparity distribution in the entire image, the disparity value correction result and the disparity continuity data; the wave equation model includes disparity fluctuation analysis parameters, time series change data and a quantitative model of dynamic change of disparity; the video screen output log includes real-time update data of the disparity adjustment strategy, the matching results of the scene and object motion and the correction record of the global disparity map.

3. The method for outputting a three-eye camera video image according to claim 1, characterized in that: The steps of extracting information from the video images of the trinocular camera, calculating the Gaussian curvature and mean curvature of multiple points in the image, calculating the differential geometric characteristics, recording the curvature properties, and generating the surface feature set are as follows: Extract frame images from the video screen of the three-eye camera, use color correction to adjust the image color temperature and contrast, perform edge detection to enhance the image edge, optimize the image resolution and clarity, and generate a processed image; Based on the processed image, the feature points in the image are located by using spatial coordinate transformation, a local curvature calculation formula is applied to each feature point, Gaussian curvature and average curvature are calculated, the curvature value of each point is recorded, and curvature analysis data is generated; The curvature analysis data is used to compare the curvature value differences of the differentiated feature points, analyze the differential geometry characteristics, draw the surface graph of each curvature point, and generate a surface feature set.

4. The method for outputting a three-eye camera video image according to claim 1, characterized in that: The steps of using the surface feature set, analyzing the directionality and intensity parameters of the differentiated curvature points, and adjusting the vector field parameters according to the surface features of the object to obtain the initial disparity vector field are as follows: Based on the surface feature set, the directionality and intensity characteristics of each curvature point are analyzed, the coordinates and curvature values ​​of the curvature points are recorded, and differentiated curvature point analysis results are generated; Adjust the direction and strength of the curvature point vector through the differentiated curvature point analysis results, verify that the parameters reflect the physical characteristics of the curvature point, and generate adjusted vector field parameters; The adjusted vector field parameters are used to summarize and analyze the geometrical morphology of the surface of the object monitored by the three-eye camera, record the features of the surface of the object, and construct an initial disparity vector field.

5. The method for outputting a three-eye camera video image according to claim 1, characterized in that: The steps of using the initial disparity vector field, applying high-order curvature information to the edge region of the object in the video screen to perform disparity adjustment, and processing the edge pixel curvature to obtain the edge optimized disparity map are specifically as follows: Based on the initial disparity vector field, identifying the edge area of ​​the object in the video picture, extracting the edge pixels in the area, and recording the associated disparity vector information to generate disparity data of the edge area; Using the edge area disparity data, performing high-order information analysis on the curvature of each edge pixel, adjusting the disparity value to match the real-time geometric curvature of the object, and generating adjusted edge disparity data; The adjusted edge disparity data is used to compare and analyze the degree of deviation and the cause of deviation between the disparity in the video and the real-time picture, verify the degree of disparity optimization in the edge area, and generate an edge optimized disparity map.

6. The method for outputting a three-eye camera video image according to claim 1, characterized in that: The edge optimized disparity map is used to adjust the local optimized disparity to a smooth disparity distribution in the entire image range, and the steps of generating a global optimized disparity map by comparing the disparity between consecutive scene frames and correcting the disparity value are specifically as follows: Based on the edge-optimized disparity map, analyzing the disparity distribution of the entire video frame, expanding the locally optimized disparity data to the entire image range, checking the continuity and smoothness of the global disparity, and generating global disparity initialization data; Using the global disparity initialization data, by comparing the disparity data between consecutive scene frames, identifying the disparity abnormal area that needs to be corrected, adjusting the area, evaluating the consistency of the disparity, and generating corrected global disparity data; The corrected global disparity data is used to record information of multiple objects in the video panorama, verify the consistency of disparity distribution, and generate a global optimized disparity map.

7. The method for outputting a three-eye camera video image according to claim 1, characterized in that: By using the global optimization disparity map and the wave equation, the trend of disparity change over time on the image plane is analyzed. By tracking the fluctuation of the disparity value of each frame, the dynamically changing disparity is calculated and quantified. The specific steps of establishing the wave equation model are as follows: Based on the global optimized disparity map, analyzing the distribution trend of disparity values ​​on the image plane, tracking the change of disparity values ​​over time, recording the dynamic data of each frame disparity, and generating disparity time series data; Using the disparity time series data, adopting the Fourier transform method, by decomposing the frequency components of the disparity data, quantitatively analyzing the fluctuation characteristics of the disparity value, calculating the disparity fluctuation rate of the differentiated time periods, and generating disparity fluctuation analysis data; According to the parallax fluctuation analysis data, the wave equation is applied to analyze and record the dynamic behavior of the parallax on the image plane according to time changes, verify the degree of analysis of the dynamic change of the parallax, and build a wave equation model.

8. The method for outputting video images of a three-eye camera according to claim 7, characterized in that: The formula of the Fourier transform method is as follows: Among them, R Δ is the parallax volatility, d i represents the disparity data value at the i-th time period, represents the time series average of the disparity data, k1, k2 and k3 are the adjustment coefficient of the real-time value of the disparity data, the adjustment coefficient of the average value and the coefficient of the normalized disparity volatility, respectively, and n is the total number of data points.

9. The method for outputting a three-eye camera video image according to claim 1, characterized in that: According to the wave equation model, by solving the real-time change of disparity, adjusting the global optimization disparity map according to the analysis result, updating the disparity adjustment strategy to match the scene and object movement, the steps of obtaining the video screen output log are specifically as follows: Based on the wave equation model, solving the real-time change of disparity in continuous video frames, analyzing the dynamic behavior of disparity between differentiated video frames, recording disparity analysis results, and generating real-time disparity analysis data; Using the real-time disparity analysis data, adjusting the global optimization disparity map to match the scene changes and the movement of the object, verifying the consistency of the disparity map with the video content, adjusting the strategy to match the dynamic scene, and generating an adjusted disparity map; According to the adjusted disparity map, the disparity adjustment strategy is updated, the correlation between the real-time scene and the object motion is analyzed, the continuous adaptability of the disparity adjustment is checked, and the video screen output log is generated.

10. A device for outputting video images of a three-eye camera, characterized in that: According to the method for outputting a video image of a three-eye camera according to any one of claims 1 to 9, the device comprises: The attribute analysis module analyzes the Gaussian curvature and mean curvature of each pixel in the video image of the trinocular camera, extracts differential geometry characteristics from the curvature data, records the curvature attributes of the pixel points, and obtains the curvature attribute set; The vector field construction module uses the curvature attribute set to analyze the directionality and intensity parameters of the differential curvature points, record the surface features of the object, and construct an initial disparity vector field; The disparity adjustment module uses the initial disparity vector field to apply high-order curvature information to the edge area of ​​the object in the video screen to perform disparity adjustment, process the edge pixel curvature, and form an edge optimized disparity map; The disparity comparison module uses the edge optimized disparity map to analyze the smooth disparity distribution within the entire image, and corrects the disparity value by comparing the disparity between consecutive scene frames to obtain a global optimized disparity distribution map; The dynamic disparity analysis module analyzes the temporal variation trend of the disparity on the image plane according to the global optimized disparity distribution map, tracks the fluctuation of the disparity value of each frame, updates the disparity adjustment strategy to match the scene and object motion, and generates a video picture output log.

Images