Continuous zoom method of dual-camera network camera based on 3D reconstruction technology

Abstract

The present invention discloses a method for continuous zooming of a dual-camera network camera based on three-dimensional reconstruction technology, which belongs to the field of camera technology and includes: collecting and preprocessing dual-camera image data of the dual-camera network camera; extracting and matching features of the dual-camera image data of the dual-camera network camera, and determining corresponding points in the dual-camera image data of the dual-camera network camera; calculating disparity according to the corresponding points, and calculating the depth information of each pixel in the scene according to the disparity and constructing a depth map; generating a three-dimensional model of the scene according to the depth map and internal and external parameters of the dual-camera network camera; simulating image effects at different focal lengths based on the three-dimensional model of the scene to achieve continuous zooming. The present invention solves the problem that the existing dual-camera network camera cannot perform continuous zooming well, resulting in poor image quality and reduced user experience. The present invention can perform continuous zooming well on the dual-camera network camera, improve the image quality of the dual-camera network camera, and improve user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of cameras, and in particular to a continuous zoom method for a dual-camera network camera based on three-dimensional reconstruction technology. Background Art

[0002] A dual-camera network camera is a network surveillance device that integrates two cameras. By using both cameras simultaneously, you can monitor different areas. Compared to traditional single-camera cameras, the dual-camera design reduces blind spots and provides more comprehensive monitoring coverage.

[0003] Among them, Chinese patent application CN219536178U discloses a 180° wide-angle dual-camera network camera. The sliding rail is embedded in a hidden mechanism assembly and slidably installed inside the built-in 180° wide-angle downward adjustment mechanism. This changes the camera's wide-angle dual-camera's telescopic adjustment by using threads to a sliding structure, which facilitates later lowering and fine-tuning of the camera's wide-angle dual-camera angle, improving the image quality of the 180° wide-angle dual-camera downward adjustment. However, this patent has the following defects:

[0004] Existing technologies cannot perform continuous zooming on dual-camera network cameras, resulting in poor image quality captured by the dual-camera network cameras and reducing user experience. Summary of the Invention

[0005] The purpose of the present invention is to provide a dual-camera network camera continuous zoom method based on three-dimensional reconstruction technology, which can better continuously zoom the dual-camera network camera, improve the image shooting effect of the dual-camera network camera, and enhance the user experience, thereby solving the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A dual-camera continuous zoom method based on three-dimensional reconstruction technology includes:

[0008] Collect and pre-process dual-camera image data from dual-camera network cameras at different viewing angles in the same scene;

[0009] Performing feature extraction and matching on the dual-camera image data of the dual-camera network camera to determine corresponding points in the dual-camera image data of the dual-camera network camera;

[0010] Calculate the disparity based on the corresponding points, and calculate the depth information of each pixel in the scene based on the disparity and construct a depth map;

[0011] Generate a 3D model of the scene based on the depth map and the internal and external parameters of the dual-camera network camera;

[0012] Based on the three-dimensional model of the scene, the image effects at different focal lengths are simulated to achieve continuous zoom.

[0013] Preferably, feature extraction and matching are performed on the dual-camera image data of the dual-camera network camera, and the following operations are performed:

[0014] Normalizing the dual-camera image data of the dual-camera network camera to remove dimension differences in the dual-camera image data of the dual-camera network camera to form standardized dual-camera image data of the dual-camera network camera;

[0015] Performing contrast enhancement on dual-camera image data of a dual-camera network camera to highlight local details in the dual-camera image data of the dual-camera network camera;

[0016] Performing feature extraction on the dual-camera image data of the dual-camera network camera, extracting key features related to the continuous zoom of the dual-camera network camera from the dual-camera image data of the dual-camera network camera, and determining the dual-camera image features of the dual-camera network camera;

[0017] The corresponding points in the dual-camera image data of the dual-camera network camera are found through dual-camera image feature matching, and the corresponding relationship between the dual-camera image data of the dual-camera network camera is established.

[0018] Preferably, the three-dimensional model of the scene is generated according to the depth map and the internal and external parameters of the dual-camera network camera, and the following operations are performed:

[0019] Collect internal and external parameters of dual-camera network cameras, including focal length, principal point, distortion coefficient, rotation matrix and translation vector;

[0020] Based on the internal and external parameters of the dual-camera network, corresponding points, and depth maps, epipolar constraints are calculated to filter out mismatched corresponding points. The matched corresponding points are then calculated based on triangulation to determine their positions in 3D space.

[0021] The position information of all corresponding points of triangulation in three-dimensional space is collected to form three-dimensional point cloud data, and the three-dimensional model of the scene is reconstructed based on the three-dimensional point cloud data.

[0022] Preferably, disparity is calculated based on corresponding points, and depth information of each pixel in the scene is calculated based on the disparity and a depth map is constructed, and the following operations are performed:

[0023] Determine the position difference between the corresponding points based on the corresponding points in the dual-camera image data of the matched dual-camera network camera, and calculate the distance between the corresponding points in the horizontal direction to determine the parallax between the corresponding points;

[0024] The disparity between corresponding points is converted into depth information, where the depth information of each pixel in the scene is determined based on the disparity, the focal length of the camera, and the distance between the dual cameras. The depth information is presented in the form of a grayscale image to generate a depth map.

[0025] Preferably, to simulate image effects at different focal lengths based on the three-dimensional model of the scene to achieve continuous zoom, the following operations are performed:

[0026] The 3D rendering engine is used to render the 3D model of the scene into a 2D image. During the rendering process, the projection matrix is ​​calculated based on the internal and external parameters of the current dual-camera network camera, and the 3D model of the scene is projected onto the 2D image plane. Different zoom effects can be simulated by adjusting the focal length or by cropping different areas of the 3D model and adjusting the position and direction of the cropping plane to simulate different zoom effects, thereby achieving continuous zoom.

[0027] Preferably, to collect dual-camera image data of dual-camera network cameras at different viewing angles in the same scene, the following operations are performed:

[0028] The two cameras of the dual-camera network camera shoot the same scene at different angles to obtain images of the dual-camera network camera at different viewing angles of the same scene, thereby collecting dual-camera image data of the dual-camera network camera.

[0029] Preferably, the dual-camera image data of the dual-camera network camera is preprocessed by performing the following operations:

[0030] Cleaning the dual-camera image data of the dual-camera network camera to remove noise data that is worthless to the dual-camera network camera's continuous zoom, thereby reducing interference of the noise data in the dual-camera image data with the dual-camera network camera's continuous zoom;

[0031] Checking the dual-camera image data of the dual-camera network camera, identifying abnormal values ​​in the dual-camera image data of the dual-camera network camera, and evaluating the abnormal values ​​in the dual-camera image data of the dual-camera network camera to determine whether the abnormal values ​​in the dual-camera image data of the dual-camera network camera are valuable for continuous zoom of the dual-camera network camera;

[0032] If the abnormal value in the dual-camera image data of the dual-camera network camera is valuable for the continuous zoom of the dual-camera network camera, then correcting the abnormal value in the dual-camera image data of the dual-camera network camera;

[0033] If the abnormal value in the dual-camera image data of the dual-camera network camera is of no value for the dual-camera network camera continuous zoom, the abnormal value in the dual-camera image data of the dual-camera network camera is removed.

[0034] Preferably, the images captured by the dual-camera network camera after continuous zoom are monitored in real time, and the internal and external parameters and the cropping plane of the dual-camera network camera are adjusted according to the monitoring feedback to achieve the best zoom effect.

[0035] Preferably, the dual-camera continuous zoom method based on three-dimensional reconstruction technology further includes:

[0036] Dual-camera network cameras collect dual-camera image data of the same scene multiple times. After preprocessing, moving object recognition is performed. If there are multiple moving objects, the moving object closest to the center of the dual-camera lens viewing range is used as the moving object, and the position of the corresponding moving object in each collected dual-camera image data is determined;

[0037] Predicting the direction and speed of movement of the moving object based on the position of the moving object in the dual-camera image data collected at multiple consecutive time intervals, combined with the time intervals between the collection of each dual-camera image data;

[0038] Based on the predicted direction and speed of movement, combined with the zoom interval length of adjacent zooms during the continuous zoom process of the dual-camera network camera, the predicted position point of the moving object at the next zoom moment is predicted, and the dual-camera network camera is driven to implement zooming based on the predicted position point; by continuously predicting the predicted position point of the moving object and performing continuous zooming, tracking and shooting of the moving object is achieved.

[0039] Preferably, the dual-camera network camera is driven to perform zooming according to the predicted position point, and the following operations are performed:

[0040] Implement adaptive sparse feature extraction. Specifically, calculate the image gradient entropy of the dual-camera image data and adjust the number of FAST feature points based on the image gradient entropy. Determine the pyramid level of the image depth, determine the weight value of each pyramid level based on the signal-to-noise ratio of the pyramid level, and use the weight value to implement multi-scale feature fusion to obtain a fused depth map.

[0041] A Markov random field model is constructed based on the image visual range of the depth map. The energy function E(d) containing data terms and smoothing terms is used as the disparity energy function. Semi-global optimization is used in parallel on GPUs to achieve real-time solution of the disparity energy function.

[0042] The optimal focal length is calculated based on the depth map, and zoom is performed at the optimal focal length.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The present invention collects dual-camera image data from different perspectives of a same scene and performs preprocessing, removes noise data that is worthless for continuous zooming of the dual-camera image data from the dual-camera image data, extracts and matches features of the dual-camera image data, determines corresponding points in the dual-camera image data, establishes a corresponding relationship between the dual-camera image data of the dual-camera, calculates disparity based on the corresponding points, calculates depth information of each pixel in the scene based on the disparity, presents the depth information in the form of a grayscale image, thereby generating a depth map, generates a three-dimensional model of the scene based on the depth map and internal and external parameters of the dual-camera network camera, simulates image effects at different focal lengths based on the three-dimensional model of the scene, thereby achieving continuous zooming, monitors the images captured by the dual-camera network camera after continuous zooming in real time, adjusts the internal and external parameters of the dual-camera network camera and the clipping plane based on the monitoring feedback, so as to achieve the optimal zooming effect, thereby achieving better continuous zooming of the dual-camera network camera, improving the image effect captured by the dual-camera network camera, and enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The present invention is a flow chart of a method for continuous zooming of a dual-camera network camera based on three-dimensional reconstruction technology. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] In order to solve the problem that the existing dual-camera network camera cannot perform continuous zoom well, resulting in poor image quality of the dual-camera network camera and reduced user experience, please refer to Figure 1 , this embodiment provides the following technical solutions:

[0048] The invention discloses a continuous zoom method for a dual-camera network camera based on 3D reconstruction technology, comprising: dual-camera image acquisition, feature extraction and matching, parallax calculation and depth estimation, 3D reconstruction and continuous zoom.

[0049] Among them, dual-camera image data of dual-camera network cameras with different viewing angles in the same scene are collected and preprocessed.

[0050] In this embodiment, dual-camera image data from dual-camera network cameras with different viewing angles in the same scene are collected, and the following operations are performed:

[0051] Based on the two cameras of the dual-camera network camera shooting the same scene at different angles, the dual-camera network camera's images of different perspectives of the same scene are obtained. According to the images of different perspectives taken by the two cameras of the dual-camera network camera, the dual-camera image data of the dual-camera network camera is collected to provide a data basis for the subsequent continuous zoom of the dual-camera network camera.

[0052] In this embodiment, the dual-camera image data is pre-processed by performing the following operations:

[0053] Cleaning the dual-camera image data of the dual-camera network camera to remove noise data that is worthless to the dual-camera network camera's continuous zoom, thereby reducing interference of the noise data in the dual-camera image data with the dual-camera network camera's continuous zoom;

[0054] Checking the dual-camera image data of the dual-camera network camera, identifying abnormal values ​​in the dual-camera image data of the dual-camera network camera, and evaluating the abnormal values ​​in the dual-camera image data of the dual-camera network camera to determine whether the abnormal values ​​in the dual-camera image data of the dual-camera network camera are valuable for continuous zoom of the dual-camera network camera;

[0055] If the abnormal value in the dual-camera image data of the dual-camera network camera is valuable for the continuous zoom of the dual-camera network camera, then correcting the abnormal value in the dual-camera image data of the dual-camera network camera;

[0056] If the abnormal value in the dual-camera image data of the dual-camera network camera is of no value for the dual-camera network camera continuous zoom, the abnormal value in the dual-camera image data of the dual-camera network camera is removed.

[0057] It should be noted that, by cleaning the dual-camera image data of the dual-camera network camera, noise data and outliers that are worthless to the continuous zoom of the dual-camera network camera can be removed, and outliers that are valuable to the continuous zoom of the dual-camera network camera in the dual-camera image data of the dual-camera network camera can be corrected, which can improve the data quality of the dual-camera image data of the dual-camera network camera and ensure the accuracy of the continuous zoom of the dual-camera network camera.

[0058] The dual-camera network camera dual-camera image data is subjected to feature extraction and matching to determine corresponding points in the dual-camera network camera dual-camera image data.

[0059] In this embodiment, feature extraction and matching are performed on the dual-camera image data of the dual-camera network camera, and the following operations are performed:

[0060] Normalizing the dual-camera image data of the dual-camera network camera to convert the dual-camera image data into a unified data format, removing the dimension differences in the dual-camera image data of the dual-camera network camera, and forming standardized dual-camera image data of the dual-camera network camera, so as to facilitate subsequent feature extraction and analysis of the dual-camera image data of the dual-camera network camera;

[0061] Performing contrast enhancement on dual-camera image data of a dual-camera network camera to highlight local details in the dual-camera image data, thereby facilitating subsequent extraction of key features related to continuous zoom of the dual-camera network camera;

[0062] Feature extraction is performed on the dual-camera image data of the dual-camera network camera, key features related to the continuous zoom of the dual-camera network camera are extracted from the dual-camera network camera dual-camera image data, the dual-camera image features of the dual-camera network camera are determined, corresponding points in the dual-camera image data of the dual-camera network camera are found through dual-camera image feature matching, and a corresponding relationship between the dual-camera image data of the dual-camera network camera is established.

[0063] The disparity is calculated based on the corresponding points, and the depth information of each pixel in the scene is calculated based on the disparity to construct a depth map.

[0064] In this embodiment, disparity is calculated based on corresponding points, and depth information of each pixel in the scene is calculated based on the disparity to construct a depth map. The following operations are performed:

[0065] Determine the position difference between the corresponding points based on the corresponding points in the dual-camera image data of the matched dual-camera network camera, and calculate the distance between the corresponding points in the horizontal direction to determine the parallax between the corresponding points;

[0066] The disparity between corresponding points is converted into depth information, where the depth information of each pixel in the scene is determined based on the disparity, the focal length of the camera, and the distance between the dual cameras. The depth information is presented in the form of a grayscale image to generate a depth map.

[0067] It should be noted that parallax refers to the positional difference between corresponding points in images of the same scene from different perspectives. In binocular stereo vision, the horizontal distance between corresponding points in the left and right images is called parallax.

[0068] It should be noted that depth estimation is the distance from each pixel in the scene to the camera obtained by parallax calculation. The depth map is a two-dimensional representation of depth information, usually presented in the form of a grayscale image, where the grayscale value of each pixel represents its depth.

[0069] Therefore, a feature extraction algorithm is used to find the corresponding points in the left and right images of the dual-camera image data of the dual-camera network camera, and the disparity is calculated according to the positional relationship of the corresponding points. The disparity is converted into depth information and presented in the form of a grayscale image to generate a depth map.

[0070] Among them, a three-dimensional model of the scene is generated based on the depth map and the internal and external parameters of the dual-camera network camera.

[0071] In this embodiment, a 3D model of a scene is generated based on the depth map and the internal and external parameters of the dual-camera network camera by performing the following operations:

[0072] Collect internal and external parameters of dual-camera network cameras, including focal length, principal point, distortion coefficient, rotation matrix and translation vector;

[0073] Based on the internal and external parameters of the dual-camera network, corresponding points, and depth maps, epipolar constraints are calculated to filter out mismatched corresponding points. The matched corresponding points are then calculated based on triangulation to determine their positions in 3D space.

[0074] The position information of all corresponding points of triangulation in three-dimensional space is collected to form three-dimensional point cloud data, and the three-dimensional model of the scene is reconstructed based on the three-dimensional point cloud data. The three-dimensional model of the scene is denoised and optimized to improve the accuracy and visual effect of the three-dimensional model of the scene.

[0075] Among them, the image effects at different focal lengths are simulated based on the scene three-dimensional model to achieve continuous zoom.

[0076] In this embodiment, the image effects at different focal lengths are simulated based on the three-dimensional model of the scene to achieve continuous zooming, and the following operations are performed:

[0077] The 3D rendering engine is used to render the 3D model of the scene into a 2D image. During the rendering process, the projection matrix is ​​calculated based on the internal and external parameters of the current dual-camera network camera, and the 3D model of the scene is projected onto the 2D image plane. Different zoom effects can be simulated by adjusting the focal length or by cropping different areas of the 3D model and adjusting the position and direction of the cropping plane to simulate different zoom effects, thereby achieving continuous zoom.

[0078] In this embodiment, the images captured by the dual-camera network camera after continuous zoom are monitored in real time, and the internal and external parameters of the dual-camera network camera and the cropping plane are adjusted according to the monitoring feedback to achieve the best zoom effect.

[0079] In summary, dual-camera image data from different perspectives of a dual-camera network camera under the same scene is collected and preprocessed, and noise data that is worthless for continuous zooming of the dual-camera network camera is removed from the dual-camera image data. Feature extraction and matching are performed on the dual-camera image data to determine corresponding points in the dual-camera image data, and a corresponding relationship between the dual-camera image data of the dual-camera network camera is established. Disparity is calculated based on the corresponding points, and depth information of each pixel in the scene is calculated based on the disparity. The depth information is presented in the form of a grayscale image to generate a depth map. A three-dimensional model of the scene is generated based on the depth map and internal and external parameters of the dual-camera network camera. Image effects at different focal lengths are simulated based on the three-dimensional model of the scene to achieve continuous zooming. The images captured by the dual-camera network camera after continuous zooming are monitored in real time. The internal and external parameters of the dual-camera network camera and the cropping plane are adjusted according to the monitoring feedback to achieve the optimal zooming effect. The dual-camera network camera can be continuously zoomed well, the image effect of the dual-camera network camera is improved, and the user experience can be enhanced.

[0080] Based on the above embodiment, the dual-camera continuous zoom method based on 3D reconstruction technology also includes:

[0081] Dual-camera network cameras collect dual-camera image data of the same scene multiple times. After preprocessing, moving object recognition is performed. If there are multiple moving objects, the moving object closest to the center of the dual-camera lens viewing range is used as the moving object, and the position of the corresponding moving object in each collected dual-camera image data is determined;

[0082] Predicting the direction and speed of movement of the moving object based on the position of the moving object in the dual-camera image data collected at multiple consecutive time intervals, combined with the time intervals between the collection of each dual-camera image data;

[0083] Based on the predicted direction and speed of movement, combined with the zoom interval length of adjacent zooms during the continuous zoom process of the dual-camera network camera, the predicted position point of the moving object at the next zoom moment is predicted, and the dual-camera network camera is driven to implement zooming based on the predicted position point; by continuously predicting the predicted position point of the moving object and performing continuous zooming, tracking and shooting of the moving object is achieved.

[0084] In this embodiment, by capturing dynamic objects and adopting a bionic design, the "rapid scanning + fine focusing" mechanism of the human eye is imitated to achieve zero-delay zoom. In the dynamic tracking of moving objects, the tracking delay of moving objects is reduced to less than 1ms, which is particularly suitable for high-speed scenarios such as autonomous driving. The operation is simple, takes up less computing resources, has low operating power consumption, and improves endurance performance.

[0085] Based on the above embodiment, the dual-camera network camera is driven to perform zooming according to the predicted position point, and the following operations are performed:

[0086] Implement adaptive sparse feature extraction. Specifically, calculate the image gradient entropy of the dual-camera image data and adjust the number of FAST feature points based on the image gradient entropy. Determine the pyramid level of the image depth, determine the weight value of each level based on the signal-to-noise ratio of the pyramid level, and use the weight value to perform multi-scale feature fusion to obtain a fused depth map. For example, the depth map fusion of pyramid level 3 can be expressed as follows:

[0087]

[0088] in, Indicates the number of FAST feature points in the fused depth map, Indicates the The weight value of the level, Represents the depth map before fusion The number of FAST feature points at the level;

[0089] A Markov random field model is constructed based on the image visual range of the depth map. The energy function E(d) containing data terms and smoothing terms is used as the disparity energy function. Semi-global optimization is used in parallel on GPU to achieve real-time solution of the disparity energy function. The disparity energy function of the depth map is:

[0090]

[0091] in, represents two different points in the visual neighborhood, represents the first Point location coordinates, represents the first The energy value of the point, Indicates the distance The point gradient step size is The energy value at and represents the gradient step size of two different points in the visual neighborhood, represents the energy gradient smoothing coefficient, Representing the visual neighborhood of the depth map;

[0092] The optimal focal length is calculated based on the depth map, and zoom is performed at the optimal focal length. The optimal focal length calculation formula is as follows:

[0093]

[0094] in, represents the optimal focal length, Indicates the initial focal length before zooming. The image coordinate points representing the depth map, Indicates the number of FAST feature points in the fused depth map, represents the depth sensitivity coefficient, Indicates the image resolution of the depth map.

[0095] In this embodiment, image entropy is used for adaptive distribution of feature points, and dynamic feature point number (density) control is adopted. Through feature fusion and combined with dynamic disparity calculation, efficient adaptive matching is achieved, depth estimation error is reduced, and estimation accuracy is improved, thereby ensuring the zoom optimization effect. In addition, GPU parallel acceleration calculation is used to increase the disparity calculation speed and reduce zoom delay.

[0096] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0097] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A dual-camera continuous zoom method based on three-dimensional reconstruction technology, characterized in that: include: Collect and pre-process dual-camera image data from dual-camera network cameras at different viewing angles in the same scene; Performing feature extraction and matching on the dual-camera image data of the dual-camera network camera to determine corresponding points in the dual-camera image data of the dual-camera network camera; Calculate the disparity based on the corresponding points, and calculate the depth information of each pixel in the scene based on the disparity and construct a depth map; Generate a 3D model of the scene based on the depth map and the internal and external parameters of the dual-camera network camera; Simulate the image effects at different focal lengths based on the three-dimensional model of the scene to achieve continuous zoom; The dual-camera network camera is driven to perform zooming according to the predicted position point, the optimal focal length is calculated according to the depth map, and zooming is performed at the optimal focal length; Also includes: Dual-camera network cameras collect dual-camera image data of the same scene multiple times. After preprocessing, moving object recognition is performed. If there are multiple moving objects, the moving object closest to the center of the dual-camera lens viewing range is used as the moving object, and the position of the corresponding moving object in each collected dual-camera image data is determined; Predicting the direction and speed of movement of the moving object based on the position of the moving object in the dual-camera image data collected at multiple consecutive time intervals, combined with the time intervals between the collection of each dual-camera image data; Based on the predicted direction and speed of movement, combined with the zoom interval length of adjacent zooms during the continuous zooming process of the dual-camera network camera, the predicted position of the moving object at the next zoom moment is predicted, and the dual-camera network camera is driven to zoom according to the predicted position point; by continuously predicting the predicted position point of the moving object and performing continuous zooming, tracking and shooting of the moving object is achieved; Drive the dual-camera network camera to zoom based on the predicted location point and perform the following operations: Implement adaptive sparse feature extraction, calculate the image gradient entropy of dual-camera image data, and adjust the number of FAST feature points based on the image gradient entropy; determine the pyramid level of image depth, determine the weight value of each level based on the signal-to-noise ratio of the pyramid level, and use the weight value to implement multi-scale feature fusion to obtain the fused depth map; A Markov random field model is constructed based on the image visual range of the depth map. The energy function E(d) containing data terms and smoothing terms is used as the disparity energy function. Semi-global optimization is used in parallel on GPUs to achieve real-time solution of the disparity energy function. The optimal focal length is calculated based on the depth map, and zoom is performed at the optimal focal length.

2. The method for continuous zooming of a dual-camera network camera based on three-dimensional reconstruction technology according to claim 1, wherein: To extract and match features from dual-camera image data, perform the following operations: Normalizing the dual-camera image data of the dual-camera network camera to remove dimension differences in the dual-camera image data of the dual-camera network camera to form standardized dual-camera image data of the dual-camera network camera; Performing contrast enhancement on dual-camera image data of a dual-camera network camera to highlight local details in the dual-camera image data of the dual-camera network camera; Performing feature extraction on the dual-camera image data of the dual-camera network camera, extracting key features related to the continuous zoom of the dual-camera network camera from the dual-camera image data of the dual-camera network camera, and determining the dual-camera image features of the dual-camera network camera; The corresponding points in the dual-camera image data of the dual-camera network camera are found through dual-camera image feature matching, and the corresponding relationship between the dual-camera image data of the dual-camera network camera is established.

3. The method for continuous zooming of dual-camera network cameras based on three-dimensional reconstruction technology according to claim 2, wherein: Generate a 3D model of the scene based on the depth map and the internal and external parameters of the dual-camera network camera. Perform the following operations: Collect internal and external parameters of dual-camera network cameras, including focal length, principal point, distortion coefficient, rotation matrix and translation vector; Based on the internal and external parameters of the dual-camera network, corresponding points, and depth maps, epipolar constraints are calculated to filter out mismatched corresponding points. The matched corresponding points are then calculated based on triangulation to determine their positions in 3D space. The position information of all corresponding points of triangulation in three-dimensional space is collected to form three-dimensional point cloud data, and the three-dimensional model of the scene is reconstructed based on the three-dimensional point cloud data.

4. The method for continuous zooming of dual-camera network cameras based on three-dimensional reconstruction technology according to claim 3, wherein: Calculate the disparity based on the corresponding points, and calculate the depth information of each pixel in the scene based on the disparity and construct a depth map. Perform the following operations: Determine the position difference between the corresponding points based on the corresponding points in the dual-camera image data of the matched dual-camera network camera, and calculate the distance between the corresponding points in the horizontal direction to determine the parallax between the corresponding points; The disparity between corresponding points is converted into depth information, where the depth information of each pixel in the scene is determined based on the disparity, the focal length of the camera, and the distance between the dual cameras. The depth information is presented in the form of a grayscale image to generate a depth map.

5. The method for continuous zooming of dual-camera network cameras based on three-dimensional reconstruction technology according to claim 4, wherein: Based on the 3D model of the scene, simulate the image effects at different focal lengths to achieve continuous zoom. Perform the following operations: The 3D rendering engine is used to render the 3D model of the scene into a 2D image. During the rendering process, the projection matrix is ​​calculated based on the internal and external parameters of the current dual-camera network camera, and the 3D model of the scene is projected onto the 2D image plane. Different zoom effects can be simulated by adjusting the focal length or by cropping different areas of the 3D model and adjusting the position and direction of the cropping plane to simulate different zoom effects, thereby achieving continuous zoom.

6. The method for continuous zooming of a dual-camera network camera based on three-dimensional reconstruction technology according to claim 5, wherein: To collect dual-camera image data from a dual-camera network camera at different viewing angles in the same scene, perform the following operations: The two cameras of the dual-camera network camera shoot the same scene at different angles to obtain images of the dual-camera network camera at different viewing angles of the same scene, thereby collecting dual-camera image data of the dual-camera network camera.

7. The method for continuous zooming of dual-camera network cameras based on three-dimensional reconstruction technology according to claim 6, wherein: To preprocess the dual-camera image data, perform the following operations: Cleaning the dual-camera image data of the dual-camera network camera to remove noise data that is worthless to the dual-camera network camera's continuous zoom, thereby reducing interference of the noise data in the dual-camera image data with the dual-camera network camera's continuous zoom; Checking the dual-camera image data of the dual-camera network camera, identifying abnormal values ​​in the dual-camera image data of the dual-camera network camera, and evaluating the abnormal values ​​in the dual-camera image data of the dual-camera network camera to determine whether the abnormal values ​​in the dual-camera image data of the dual-camera network camera are valuable for continuous zoom of the dual-camera network camera; If the abnormal value in the dual-camera image data of the dual-camera network camera is valuable for the continuous zoom of the dual-camera network camera, then correcting the abnormal value in the dual-camera image data of the dual-camera network camera; If the abnormal value in the dual-camera image data of the dual-camera network camera is of no value for the dual-camera network camera continuous zoom, the abnormal value in the dual-camera image data of the dual-camera network camera is removed.

8. The method for continuous zooming of dual-camera network cameras based on three-dimensional reconstruction technology according to claim 7, wherein: The dual-camera image data of the dual-camera network camera after continuous zoom is monitored in real time, and the internal and external parameters and cropping plane of the dual-camera network camera are adjusted according to the monitoring feedback to achieve the best zoom effect.

Images