Stereoscopic shooting focusing method and system for stereoscopic image shooting

Abstract

The invention discloses a stereo camera shooting focusing method and system for stereo image shooting. The method comprises the following steps: firstly, preprocessing at least one group of original image data collected by at least two camera shooting devices at the same time to obtain a focusing image sequence; inputting the focusing image sequence into the depth information algorithm mathematical model to obtain a focusing depth map; performing target imaging area identification and labeling on a group of original image data in the focusing image sequence according to the focusing depth map to obtain synchronous target area image data; and finally, automatically adjusting focusing parameters of each camera device synchronously according to the synchronous target area image data. The depth information acquisition technology in visual imaging is applied to the stereoscopic photographing system, and the focusing parameters of the multiple photographing devices are dynamically coordinated according to the consistency of the depth information, so that the definition and the three-dimensional feeling of stereoscopic images are improved, and the user experience is improved in a mode of enhancing a real-time feedback mechanism for a user.

Description

Technical Field

[0001] The present application relates to the technical field of stereoscopic photography, and in particular to a stereoscopic camera focusing method and system for stereoscopic image shooting. Background Art

[0002] Stereoscopic image shooting refers to the use of special equipment and technology to shoot images with a sense of depth and three-dimensionality, which can be used for playback of VR glasses or other stereoscopic imaging devices, and can bring a more immersive viewing experience to the audience. In the field of advertising, stereoscopic images can highlight the three-dimensionality and realism of products; in the field of education, stereoscopic images can be used to make teaching models to help students better understand abstract concepts; in the field of scientific research, stereoscopic images can be used to record and analyze three-dimensional space structures; in the field of games and virtual social networking, stereoscopic images can bring users a more immersive physical experience. Generally, stereoscopic images are shot by shooting two images from different viewpoints, each viewed by the corresponding eyes, to simulate a three-dimensional effect. Generally, a standard stereo camera can be used or a stereo attachment lens can be installed on a single-lens camera to achieve this. Using an ordinary camera, it is necessary to move left and right to shoot stereoscopic images (time-sharing asynchronous shooting). The imaging effect of the stereoscopic images obtained by the above shooting method is poor. Due to the limitations of the shooting equipment and the limitations of the camera focus range, the image in the non-focal area is blurred (poor clarity). How to shoot high-quality three-dimensional stereo images is still the basic direction of the research and development of stereoscopic camera equipment at this stage. Summary of the invention

[0003] The technical problem to be solved by the present application is how to enhance and improve the three-dimensional effect and image quality of stereoscopic images.

[0004] According to the first aspect, an embodiment provides a stereo camera focusing method for stereo image shooting, comprising: Preprocessing at least one set of raw image data collected simultaneously by at least two camera devices to obtain a focus image sequence; Inputting the focusing image sequence into a preset depth information algorithm mathematical model to obtain a focusing depth map output by the depth information algorithm mathematical model; According to the focus depth map, a group of the original image data in the focus image sequence are respectively identified and labeled with a target imaging area to obtain synchronized target area image data; the synchronized target area image data includes distance information between the identified and labeled targets and respective camera devices; Focus parameters of each of the camera devices are automatically adjusted synchronously according to the synchronization target area image data.

[0005] In one embodiment, the depth information algorithm mathematical model includes a binocular vision depth estimation algorithm mathematical model, a stereo matching algorithm mathematical model or an optical flow method to obtain depth information mathematical model.

[0006] In one embodiment, when the depth information algorithm mathematical model is a stereo matching algorithm mathematical model, preprocessing at least one set of raw image data simultaneously collected by at least two camera devices to obtain a focus image sequence includes: The two camera devices that acquire the original image data form a binocular camera system, and one of each set of original image data is used as a left image, and the other is used as a right image.

[0007] In one embodiment, the step of preprocessing at least one set of raw image data collected simultaneously by at least two camera devices to obtain a focus image sequence includes: Each piece of the original image data is subjected to denoising and / or enhancement processing to obtain an image element in the focused image sequence; the denoising processing is used to remove noise points generated during the acquisition process of the original image data, and the enhancement processing is used to apply data enhancement technology to flip, crop, scale and / or adjust the illumination of the region of interest in the original image data.

[0008] In one embodiment, the focus adjustment parameters include an exposure factor, an aperture factor, a shutter speed factor and / or a noise control factor.

[0009] In one embodiment, the automatically adjusting the focus parameters of each of the camera devices synchronously based on the synchronization target area image data includes: Maintaining the consistency of the focusing parameters of each of the camera devices in real time; Alternatively, the focus parameters are independently set according to the difference in viewing angles of the respective camera devices.

[0010] In one embodiment, the stereo camera focusing method further includes: According to the consistency of the depth information in the image data of the synchronized target area, the image data obtained after the camera device synchronizes and automatically adjusts the focusing parameters is subjected to image post-processing; the image post-processing is performed by interactively compensating the grouped original image data for focal length correction and / or parallax compensation.

[0011] According to the second aspect, an embodiment provides a computer-readable storage medium, on which a program is stored. The program can be executed by a processor to implement the stereoscopic camera focusing method as described in the first aspect.

[0012] According to a third aspect, a computer program product includes a computer program and / or instructions, wherein when the computer program and / or instructions are executed by a processor, the stereoscopic camera focusing method as described in the first aspect is implemented.

[0013] According to a fourth aspect, an embodiment provides a stereo camera focusing system for stereo image shooting, which is used to apply the stereo camera focusing method according to the first aspect, and the stereo camera focusing system includes: A focusing image acquisition unit, used for preprocessing at least one set of raw image data collected simultaneously by at least two camera devices to obtain a focusing image sequence; A focusing depth map acquisition unit, used for inputting the focusing image sequence into a preset depth information algorithm mathematical model to acquire a focusing depth map output by the depth information algorithm mathematical model; a target area recognition unit, configured to respectively recognize and mark a target imaging area for a group of the original image data in the focus image sequence according to the focus depth map, so as to obtain synchronized target area image data; the synchronized target area image data includes distance information between the recognized and marked targets and respective camera devices; A focusing unit is used to automatically adjust the focusing parameters of each of the camera devices according to the synchronization target area image data.

[0014] According to the stereoscopic camera focusing method of the above embodiment, since the depth information acquisition technology in visual imaging is applied to the stereoscopic photography system, the focus parameters of multiple camera devices are dynamically coordinated according to the consistency of the depth information, thereby improving the clarity and three-dimensional sense of the stereoscopic image, and enhancing the real-time feedback mechanism for the user to improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a flow chart of a stereo camera focusing method in an embodiment; Figure 2 1 is a structural block diagram of a stereo camera focusing system in an embodiment. DETAILED DESCRIPTION

[0016] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0017] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0018] The serial numbers assigned to the components herein, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0019] In existing stereo photography technology, dual cameras are often used for shooting with a fixed focal length. This method is prone to focus mismatch in complex depth scenes, resulting in distorted three-dimensional effects and reduced image clarity. For example, in a three-dimensional visual scene, when moving objects are operated (the types of camera movements include surround, push-pull, pan, and follow-up camera techniques), using a fixed focal length to shoot will cause the shooting to be out of focus due to the inability to synchronize with the moving object. The main reasons why the automatic focus function is not applied to the stereo photography process at this stage are as follows: 1. It is difficult to accurately obtain and process the depth information of the image in real time when shooting any moment; 2. It is difficult to adjust the focal length of multiple cameras synchronously and the algorithm is complex; 3. The user interaction feedback mechanism during the entire focusing process is not clear, that is, when the focus target needs to be changed, the replacement focus target cannot be quickly confirmed; 4. The data consistency and compensation mechanism after multiple focusing adjustments are not yet perfect.

[0020] In the embodiment of the present application, the depth information acquisition technology in visual imaging is applied to the stereoscopic photography system to dynamically coordinate the focus parameters of the dual-camera devices according to the consistency of the depth information in the image data, thereby solving the focus mismatch problem of multiple camera devices. The automatic focus function can also adjust the focal length in real time for complex scenes, improve image clarity and three-dimensional sense, and thus enhance the real-time feedback mechanism for users and improve user experience.

[0021] Embodiment 1: Please refer to Figure 1 , is a flow chart of a stereo camera focusing method in an embodiment, which is used to realize the automatic focusing function of stereo image shooting, and specifically includes: Step 101: Acquire a focus image sequence.

[0022] At least one set of raw image data collected simultaneously by at least two camera devices is preprocessed to obtain a focus image sequence. In one embodiment, each piece of raw image data is subjected to denoising and / or enhancement processing to obtain an image element in the focus image sequence, wherein the denoising processing is used to remove noise points generated during the acquisition of the raw image data, and the enhancement processing is used to apply data enhancement technology to flip, crop, scale and / or adjust the illumination of the region of interest in the raw image data.

[0023] Step 102: Obtain a focus depth map.

[0024] The focused image sequence is input into a preset depth information algorithm mathematical model to obtain a focused depth map output by the depth information algorithm mathematical model. In one embodiment, the depth information algorithm mathematical model includes a binocular vision depth estimation algorithm mathematical model, a stereo matching algorithm mathematical model, or an optical flow method to obtain depth information mathematical model. In one embodiment, when the depth information algorithm mathematical model is a stereo matching algorithm mathematical model, two camera devices that obtain raw image data form a binocular camera system, and one of each set of raw image data is used as a left image and the other is used as a right image.

[0025] Step 103, acquiring synchronization target area image data.

[0026] According to the focusing depth map, a group of original image data in the focusing image sequence are respectively identified and labeled with target imaging areas to obtain synchronized target area image data, wherein the synchronized target area image data includes distance information between the identified and labeled targets and respective camera devices.

[0027] Step 104: automatically adjust the focus parameters.

[0028] The focus parameters of each camera device are automatically adjusted synchronously according to the synchronization target area image data. In one embodiment, the focus parameters include exposure factor, aperture factor, shutter speed factor and / or noise control factor.

[0029] In one embodiment, the focus parameters of each camera device are kept consistent in real time. In one embodiment, the focus parameters are set independently according to the difference in viewing angles of the respective camera devices.

[0030] In one embodiment, the automatic focusing method for video recording further includes: Step 105: post-processing of image data.

[0031] According to the consistency of the depth information in the synchronized target area image data, the image data acquired after the camera device synchronizes and automatically adjusts the focus parameters is subjected to image post-processing. In one embodiment, the image post-processing is to perform focus correction and / or parallax compensation on the grouped original image data in an interactive compensation manner.

[0032] Please refer to Figure 2 , is a structural block diagram of a stereo camera focusing system in an embodiment. In one embodiment of the present application, a stereo camera focusing system is also disclosed for applying the stereo camera focusing method as described above, specifically comprising a focusing image acquisition unit 10, a focusing depth map acquisition unit 20, a target area recognition unit 30 and a focusing unit 40. The focusing image acquisition unit 10 is used to pre-process at least one group of raw image data collected simultaneously by at least two camera devices to obtain a focusing image sequence. The focusing depth map acquisition unit 20 is used to input the focusing image sequence into a preset depth information algorithm mathematical model to obtain a focusing depth map output by the depth information algorithm mathematical model. The target area recognition unit 30 is used to identify and mark the target imaging area of ​​a group of raw image data in the focusing image sequence according to the focusing depth map, so as to obtain synchronized target area image data. The synchronized target area image data includes the distance information between the identified and marked targets and the respective camera devices. The focusing unit 40 is used to automatically adjust the focusing parameters of each camera device synchronously according to the synchronized target area image data.

[0033] In the stereo camera focusing method disclosed in the embodiment of the present application, at least one group of raw image data collected simultaneously by at least two camera devices is first preprocessed to obtain a focusing image sequence; then the focusing image sequence is input into the depth information algorithm mathematical model to obtain a focusing depth map; then, according to the focusing depth map, a group of raw image data in the focusing image sequence are respectively identified and labeled with the target imaging area to obtain synchronized target area image data; finally, according to the synchronized target area image data, the focusing parameters of each camera device are automatically adjusted synchronously. Since the depth information acquisition technology in visual imaging is applied to the stereo photography system, the focusing parameters of multiple camera devices are dynamically coordinated according to the consistency of the depth information, thereby improving the clarity and three-dimensional sense of the stereo image, and enhancing the real-time feedback mechanism for the user to improve the user experience.

[0034] The automatic focusing method for video recording disclosed in the embodiment of the present application dynamically adjusts the focal length of the video recording device according to the depth information of the scene image extracted in real time, thereby avoiding the limitations of the traditional fixed focal length. Further, an efficient stereo matching algorithm is adopted to ensure the accuracy of the depth information extraction. Further, according to the depth distribution of the scene image or the specified area preset by the user, the distance of the target object is dynamically determined as the basis for focusing. For example, in a complex scene, the area with the most significant depth distribution range is preferentially selected, or the target object is manually specified according to the user input. Among them, the focal length of the video recording device can be dynamically adjusted according to the depth information. The consistency mode is that the left and right video recording devices adjust the focal length synchronously, which is suitable for the scene where the target object is centered. The independent mode is that the left and right cameras adjust the focal length independently according to the difference in viewing angles, which is suitable for the scene where the target object is asymmetric. Further, a focusing feedback mechanism is added to display the focusing information, the current target object depth, focusing parameters and image clarity evaluation indicators in real time through the user interface, and the user can adjust the mode or target area at any time. The focusing feedback mechanism can optimize the user operation experience while improving the intuitiveness and control accuracy of focusing.

[0035] The following describes the application of the automatic focusing method for video recording disclosed in the present application through a specific embodiment, which specifically includes: a. Image data acquisition.

[0036] A high-precision dual-camera system is used to synchronously capture images and input them into the depth analysis module. b. In-depth information analysis.

[0037] The image is processed using stereo matching technology to extract the depth distribution information of the scene.

[0038] c. Coordinated focusing of both eyes.

[0039] According to the target depth, the focal length of the left and right cameras is dynamically adjusted, and the viewing angle is optimized simultaneously. d. User interaction and feedback.

[0040] The focus adjustment status is displayed in real time through the user interface, supporting user manual intervention and real-time control.

[0041] e. Data interaction compensation.

[0042] The image data after multiple focusing adjustments is corrected for consistency and compensated for parallax through algorithms to ensure the stability of image quality and three-dimensional effects.

[0043] The camera automatic focusing method based on depth information as described above not only improves the image quality and three-dimensional effect of stereoscopic photography, but also provides new feasibility for the development of future stereoscopic photography technology.

[0044] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0045] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art, according to the concept of the present invention, some simple deductions, modifications or substitutions can be made.

Claims

1. A stereo camera focusing method for stereo image shooting, characterized in that: include: Preprocessing at least one set of raw image data collected simultaneously by at least two camera devices to obtain a focus image sequence; Inputting the focusing image sequence into a preset depth information algorithm mathematical model to obtain a focusing depth map output by the depth information algorithm mathematical model; According to the focus depth map, a group of the original image data in the focus image sequence are respectively identified and labeled with a target imaging area to obtain synchronized target area image data; the synchronized target area image data includes distance information between the identified and labeled targets and respective camera devices; Focus parameters of each of the camera devices are automatically adjusted synchronously according to the synchronization target area image data.

2. The stereo camera focusing method according to claim 1, wherein: The depth information algorithm mathematical model includes a binocular vision depth estimation algorithm mathematical model, a stereo matching algorithm mathematical model or an optical flow method to obtain depth information mathematical model.

3. The stereo camera focusing method according to claim 2, wherein: When the depth information algorithm mathematical model is a stereo matching algorithm mathematical model, the preprocessing of at least one set of original image data collected simultaneously by at least two camera devices to obtain a focus image sequence includes: The two camera devices that acquire the original image data form a binocular camera system, and one of each set of original image data is used as a left image, and the other is used as a right image.

4. The stereo camera focusing method according to claim 1, wherein: The step of preprocessing at least one set of raw image data collected simultaneously by at least two camera devices to obtain a focus image sequence comprises: Each piece of the original image data is subjected to denoising and / or enhancement processing to obtain an image element in the focused image sequence; the denoising processing is used to remove noise points generated during the acquisition process of the original image data, and the enhancement processing is used to apply data enhancement technology to flip, crop, scale and / or adjust the illumination of the region of interest in the original image data.

5. The stereo camera focusing method according to claim 1, wherein: The focusing parameters include an exposure factor, an aperture factor, a shutter speed factor and / or a noise control factor.

6. The stereo camera focusing method according to claim 1, wherein: The automatically adjusting the focus parameters of each of the camera devices synchronously according to the synchronization target area image data comprises: Maintaining the consistency of the focusing parameters of each of the camera devices in real time; Alternatively, the focus parameters are independently set according to the difference in viewing angles of the respective camera devices.

7. The stereo camera focusing method according to claim 1, wherein: Also includes: According to the consistency of the depth information in the image data of the synchronized target area, the image data obtained after the camera device synchronizes and automatically adjusts the focusing parameters is subjected to image post-processing; the image post-processing is performed by interactively compensating the grouped original image data for focal length correction and / or parallax compensation.

8. A computer program product comprising a computer program and / or instructions, characterized in that: When the computer program and / or the instructions are executed by a processor, the stereoscopic camera focusing method according to any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium, characterized in that: The medium stores a program, and the program can be executed by a processor to implement the stereoscopic camera focusing method according to any one of claims 1 to 7.

10. A stereo camera focusing system for stereo image shooting, characterized in that: Used to apply the stereo camera focusing method according to any one of claims 1 to 7, the stereo camera focusing system comprises: A focusing image acquisition unit, used for preprocessing at least one set of raw image data collected simultaneously by at least two camera devices to obtain a focusing image sequence; A focusing depth map acquisition unit, used for inputting the focusing image sequence into a preset depth information algorithm mathematical model to acquire a focusing depth map output by the depth information algorithm mathematical model; a target area recognition unit, configured to respectively recognize and mark a target imaging area for a group of the original image data in the focus image sequence according to the focus depth map, so as to obtain synchronized target area image data; the synchronized target area image data includes distance information between the recognized and marked targets and respective camera devices; A focusing unit is used to automatically adjust the focusing parameters of each of the camera devices according to the synchronization target area image data.

Images