Shooting sequence focusing method and system for continuous shooting snapshot function

Abstract

The invention discloses a shooting sequence focusing method and system for a continuous shooting snapshot function. The method comprises the following steps: firstly, responding to a first operation of a user, switching shooting equipment to enter a continuous shooting mode; preprocessing the pre-shot image data to obtain a focusing image sequence; inputting the focusing image sequence into a focal length sequence to obtain a neural network mathematical model so as to obtain continuous shooting image sequence feature change data, and presetting a continuous shooting image focal length parameter according to the continuous shooting parameter and the continuous shooting image sequence feature change data; and when a second operation of the user is responded, executing continuous shooting according to the continuous shooting image focal length parameter. The neural network mathematical model is acquired through the focal length sequence to analyze the image feature parameters in the pre-shot focusing image sequence, so that the focusing parameters during shooting of each image during continuous shooting are preset in real time, intelligent automatic focusing in continuous shooting is realized, the imaging quality and shooting efficiency of a continuous shooting snapshot function are remarkably improved, and the shooting efficiency of the continuous shooting snapshot function is improved. And the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of photographic technology, and in particular to a photographic sequence focusing method and system for a continuous snapshot function. Background Art

[0002] With the popularization and development of camera equipment, users have increasingly higher requirements for the quality of captured images. Traditional autofocus technology mainly relies on image sensors and reflector components. Compared with professional camera operators, these methods often have poor autofocus performance in complex scenes (such as low-light environments, multi-object scenes or motion scenes). With the advancement of machine learning algorithm technology, especially in the field of image processing technology, it has shown significant advantages. Due to the improvement of the computing power and image processing capabilities of camera terminal hardware equipment, how to effectively combine machine learning algorithms with the focusing technology of camera equipment is a major direction of camera equipment research and development at this stage. Among them, the continuous snapshot function is an important shooting function of camera equipment. For example, if a smart mobile terminal (such as a mobile phone) is used to record a certain motion process, the smart mobile terminal can use the continuous snapshot function to continuously shoot the motion process to obtain multiple image sequences arranged in time. However, the entire image sequence is shot according to the same preset focusing parameters, so some element images in some image sequences will be out of focus, and it cannot be guaranteed that each element image in the entire image sequence is clear and usable. Especially when continuously shooting a target that moves rapidly over a large range in front of or behind a camera device, the number of element images with poor shooting effects will be greater. Summary of the invention

[0003] The technical problem to be solved by this application is how to improve and perfect the continuous snapshot function.

[0004] According to a first aspect, an embodiment provides a photographic sequence focusing method for a continuous snapshot function, comprising: In response to a first user operation, switching the camera device to enter a continuous shooting mode; Preprocessing at least N pre-shot image data marked with time node information collected according to a preset continuous shooting parameter to obtain a focus image sequence, wherein N is a natural number not less than 1; Inputting the focus image sequence into a preset focal length sequence acquisition neural network mathematical model to obtain continuous shooting image sequence feature change data output by the focal length sequence acquisition neural network mathematical model; Presetting the continuous shooting image focal length parameter according to the continuous shooting parameter and the continuous shooting image sequence characteristic change data; the continuous shooting image focal length parameter is used to set the focus parameter of each image taken by the camera device when performing continuous shooting; In response to a second user operation, continuous shooting is performed according to the continuous shooting image focal length parameter.

[0005] In one embodiment, the preprocessing of at least N pre-shot image data marked with time node information collected according to a preset continuous shooting parameter includes: Each of the pre-shot image data is subjected to denoising and / or enhancement processing to obtain an element image in the focused image sequence; the denoising processing is used to remove noise points generated during the acquisition process of the pre-shot image data, and the enhancement processing is used to apply data enhancement technology to flip, crop, scale and / or adjust the illumination of the region of interest in the pre-shot image data.

[0006] In one embodiment, the focal length sequence acquisition neural network mathematical model includes a recurrent neural network, a long short-term memory network and / or a transformer model; the recurrent neural network is used to extract the temporal dynamic characteristics of each element image in the focusing image sequence on the time axis; the long short-term memory network is used to extract the dynamic movement trend characteristics of the target object in each element image in the focusing image sequence on the time axis; the transformer model extracts the global contextual correlation information characteristics of each element image in the focusing image sequence through a self-attention mechanism.

[0007] In one embodiment, the focusing image sequence is input into a preset focal length sequence to obtain a neural network mathematical model, including: The focusing image sequence is sequentially input into the recurrent neural network, the long short-term memory network and the transformer model to obtain feature change data of the continuous shooting image sequence.

[0008] In one embodiment, the photographic sequence focusing method further includes: When the camera device is performing continuous shooting, at least one verification image data is obtained through the auxiliary camera device; wherein the auxiliary camera device and the camera device performing continuous shooting are synchronously photographed; Focus parameters of a camera device that performs continuous shooting are verified and fine-tuned based on the verification image data.

[0009] In one embodiment, the step of verifying and fine-tuning the focus parameters of a camera device that performs continuous shooting according to the verification image data includes: Performing imaging quality scoring on the verification image data and the continuous-shooting image data taken by the camera device respectively; the imaging quality includes image clarity, depth of field accuracy and / or target position accuracy; When the imaging quality score of the verification image data is greater than the imaging quality score of the continuous-shooting image data, the focusing parameter of the camera device is fine-tuned according to the focusing parameter of the auxiliary camera device.

[0010] In one embodiment, the continuous shooting parameters include continuous shooting frequency, continuous shooting quantity and scene type; The scene types include static scenes, dynamic scenes, special light source scenes, character scenes or landscape scenes, and the special light source scenes include low light environments, strong light environments or backlight environments; The focusing parameters include exposure factor, aperture factor, shutter speed factor and / or noise control factor; The continuous-shooting image sequence feature change data includes image definition change feature data, depth of field change feature data and / or target movement trajectory feature data on the time axis of each element image in the focus image sequence.

[0011] According to the second aspect, an embodiment provides a computer program product, including a computer program and / or instructions, which, when executed by a processor, implements the photographic sequence focusing method as described in the first aspect.

[0012] According to a third aspect, an embodiment provides a computer-readable storage medium, on which a program is stored, and the program can be executed by a processor to implement the photography sequence focusing as described in the first aspect.

[0013] According to a fourth aspect, an embodiment provides a photographic sequence focusing system for a continuous snapshot function, which is used to apply the photographic sequence focusing method according to the first aspect, and the photographic sequence focusing system includes: A sequence acquisition unit, used for pre-processing at least N pre-shot image data marked with time node information collected according to a preset continuous shooting parameter to obtain a focus image sequence, wherein N is a natural number not less than 1; A data processing unit, used for inputting the focus image sequence into a preset focal length sequence acquisition neural network mathematical model to obtain continuous shooting image sequence feature change data output by the focal length sequence acquisition neural network mathematical model; The parameter setting unit is used to preset the continuous shooting image focal length parameters according to the continuous shooting parameters and the continuous shooting image sequence characteristic change data; the continuous shooting image focal length parameters are used to set the focusing parameters of each image taken by the camera device when performing continuous shooting.

[0014] According to the photographic sequence focusing method of the above embodiment, the image feature parameters in the pre-shot focusing image sequence are analyzed by the neural network mathematical model through the focal length sequence to preset the focusing parameters for each image taken during continuous shooting in real time, thereby realizing intelligent automatic focusing in continuous shooting, significantly improving the imaging quality and shooting efficiency of the continuous snapshot function, and thus improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a process flow of a photographic sequence focusing method in an embodiment; Figure 2 1 is a structural block diagram of a photographic sequence focusing system in an embodiment. DETAILED DESCRIPTION

[0016] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0017] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0018] The serial numbers assigned to the components herein, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0019] Existing automatic focusing technology has problems such as slow focusing speed and poor imaging effect in continuous shooting scenes. In the embodiment of this application, the sequence model and intelligent image analysis technology are introduced to provide an efficient and fast focusing method for continuous shooting. By analyzing the subtle change parameters in the continuously shot image sequence (such as clarity gradient, depth of field change amplitude and target movement trend), combined with the sequence model to predict the focal length adjustment parameters in real time, intelligent focusing in continuous shooting is achieved, which significantly improves the imaging quality and shooting efficiency, and is particularly suitable for shooting needs in dynamic scenes and complex lighting conditions.

[0020] Embodiment 1: Please refer to Figure 1, is a flow chart of a photographic sequence focusing method in an embodiment, the photographic sequence focusing method is used to improve and perfect the continuous snapshot function, and specifically includes: Step 101, switch to continuous shooting mode.

[0021] In response to the user's first operation, the camera device is switched to enter the continuous shooting mode. For example, when shooting with a mobile phone, the camera enters the preview mode (the camera starts working), and then the continuous shooting mode is selected.

[0022] Step 102: Acquire a focus image sequence.

[0023] Pre-processing is performed on at least N pre-shot image data marked with time node information collected according to a preset continuous shooting parameter to obtain a focused image sequence, wherein N is a natural number not less than 1. The acquisition of the focused image sequence is automatically performed in the background of the camera. In one embodiment, denoising and / or enhancement processing is performed on each pre-shot image data to obtain an element image in the focused image sequence. De-noising processing is used to remove noise generated during the pre-shot image data acquisition process, and enhancement processing is used to apply data enhancement technology to flip, crop, scale and / or adjust the lighting of the area of ​​interest in the pre-shot image data. The purpose of pre-processing the pre-shot image data is to highlight the target features of each image to improve the efficiency of image post-processing.

[0024] In one embodiment, in order to ensure focusing accuracy, certain imaging conditions are required for the pre-captured image data (to ensure imaging quality), for example, the exposure value is maintained within a set range (to avoid overexposure or underexposure), the shutter speed is fast enough to avoid motion blur, and the light uniformity meets a minimum threshold to avoid image imbalance.

[0025] Step 103: Acquire image sequence feature change data.

[0026] The focusing image sequence is input into a preset focal length sequence acquisition neural network mathematical model to obtain the continuous shooting image sequence feature change data output by the focal length sequence acquisition neural network mathematical model. Among them, the continuous shooting image sequence feature change data includes image clarity change feature data, depth of field change feature data and / or target movement trajectory feature data on the time axis of each element image in the focusing image sequence. In one embodiment, the focal length sequence acquisition neural network mathematical model includes a recurrent neural network, a long short-term memory network and / or a transformer model. The recurrent neural network (RNN) can capture the temporal dynamics between each element image by processing sequence image data, and the recurrent neural network is used to extract the temporal dynamic features of each element image in the focusing image sequence on the time axis. The long short-term memory network (LSTM) can capture the temporal dynamics between video frames and understand the temporal relationship in the image sequence, thereby improving the accuracy of image processing. The long short-term memory network is used to extract the dynamic movement trend features of the target object in the focusing image sequence. The transformer model extracts the global context information features of each element image in the focusing image sequence through a self-attention mechanism. In one embodiment, the focus image sequence is sequentially input into a recurrent neural network, a long short-term memory network and a transformer model to obtain feature change data of the continuous shooting image sequence.

[0027] Step 104: preset continuous shooting focus parameters.

[0028] The continuous shooting image focal length parameters are preset according to the continuous shooting parameters and the continuous shooting image sequence feature change data. The continuous shooting image focal length parameters are used to set the focus parameters of each image taken by the camera device during continuous shooting. Among them, the focus parameters include exposure factor, aperture factor, shutter speed factor and / or noise control factor, and the continuous shooting parameters include continuous shooting frequency, continuous shooting quantity and scene type. The scene types include static scenes, dynamic scenes, special light source scenes, character scenes or landscape scenes, and the special light source scenes include low light environment, strong light environment or backlight environment.

[0029] Step 105, executing continuous shooting.

[0030] In response to the second user operation, continuous shooting is performed according to the continuous shooting image focal length parameter.

[0031] In one embodiment, the automatic focusing method for video recording further includes: Step 106, continuous shooting parameter verification.

[0032] When the camera device is performing continuous shooting, at least one verification image data is obtained through the auxiliary camera device, and the focus parameters of the camera device performing continuous shooting are verified and fine-tuned according to the verification image data. The auxiliary camera device and the camera device performing continuous shooting shoot synchronously.

[0033] In one embodiment, the process of verifying and fine-tuning the focus parameters of a camera device that performs continuous shooting includes: The verification image data and the continuous image data taken by the camera device are scored for imaging quality, and the imaging quality includes image clarity, depth of field accuracy and / or target position accuracy. When the imaging quality score of the verification image data is greater than the imaging quality score of the continuous image data, the focusing parameters of the camera device are fine-tuned according to the focusing parameters of the auxiliary camera device.

[0034] Please refer to Figure 2 , is a structural block diagram of a photographic sequence focusing system in an embodiment. In one embodiment of the present application, a photographic sequence focusing system is also disclosed for applying the photographic sequence focusing method as described above, specifically comprising a sequence acquisition unit 10, a data processing unit 20 and a parameter setting unit 30. The sequence acquisition unit 10 is used to pre-process at least N pre-shot image data marked with time node information collected according to a preset continuous shooting parameter to obtain a focused image sequence, wherein N is a natural number not less than 1. The data processing unit 20 is used to input the focused image sequence into a preset focal length sequence acquisition neural network mathematical model to obtain continuous shooting image sequence feature change data output by the focal length sequence acquisition neural network mathematical model. The parameter setting unit 30 is used to preset the continuous shooting image focal length parameters according to the continuous shooting parameters and the continuous shooting image sequence feature change data, and the continuous shooting image focal length parameters are used to set the focusing parameters of each image taken when the camera device performs continuous shooting.

[0035] The photographic sequence focusing method disclosed in the embodiment of the present application first responds to the user's first operation to switch the camera device to enter the continuous shooting mode; then pre-processes the pre-shot image data to obtain the focusing image sequence; then inputs the focusing image sequence into the focal length sequence to obtain the neural network mathematical model to obtain the continuous shooting image sequence feature change data, and presets the continuous shooting image focal length parameters according to the continuous shooting parameters and the continuous shooting image sequence feature change data; when responding to the user's second operation, the continuous shooting is performed according to the continuous shooting image focal length parameters. Since the image feature parameters in the pre-shot focusing image sequence are analyzed by the neural network mathematical model obtained by the focal length sequence, the focusing parameters when each image is shot during continuous shooting are preset in real time, thereby realizing intelligent automatic focusing in continuous shooting, significantly improving the imaging quality and shooting efficiency of the continuous shooting snapshot function, and thus improving the user experience.

[0036] In the embodiment of the present application, the neural network mathematical model is obtained through the focal length sequence to define and extract features of the subtle changes in the focusing image sequence. Among them, the subtle changes in the image sequence include changes in clarity gradient (capturing changes in image edge clarity by calculating image gradients and Laplace operators), changes in depth of field (evaluating the change in the degree of blur between the target object and the background) and the target object motion trend (the trajectory of target position change can be estimated based on the optical flow method). The mathematical model of the neural network extracted from the focal length sequence predicts the focusing parameters based on the subtle changes in the features of the image sequence. In one embodiment, the model is trained using a wide range of image data sets, covering dynamic scenes, low light conditions and complex background environments. The model selection is based on the focal length adjustment effect in each scene, that is, a mapping between model training and focusing parameters is established. The sequence model establishes a mapping relationship between image change features and focal length adjustment values ​​through supervised learning. The training set includes real captured images and simulated images, involving different focal length, clarity and depth of field parameter combinations. The focal length parameters output by the model are applied in real time by the photographic device. The following describes the application of the automatic focusing method for video recording disclosed in the present application through a specific embodiment, which specifically includes: The automatic focusing method of the camera in the embodiment of the present application combines intelligent image recognition technology to dynamically adjust the focus priority by analyzing the subject categories (such as people, animals, vehicles, etc.) in the image sequence. For example, portrait shooting scenes (focusing on the face area first), motion scenes (dynamically adjusting the follow focus according to the speed of the target object) and image recognition systems provide data support for focus adjustment to improve the focusing effect.

[0037] In one embodiment, the automatic focusing method for video is applied to a multi-camera system. By synchronously acquiring image sequences of different viewing angles and collaboratively analyzing the depth of field differences of each camera, the focusing strategy can be optimized. For example, it is applied to real-time shooting of sports games to capture high-speed moving targets (such as athletes and vehicles). The main camera is used for focus tracking, and the auxiliary camera (i.e., auxiliary camera device) corrects the focal length deviation.

[0038] In one embodiment, different focus parameter settings are switched for different shooting scenes. For example, when the image subject changes, the focus is determined by detecting the change in the size and position of the target in the sequence. When the ambient light changes, the aperture and focus are adjusted in combination with the brightness histogram. When switching the shooting task type, the parameters are preset according to the shooting task (such as portrait priority, distant view priority).

[0039] The automatic focusing method for video recording disclosed in the embodiments of the present application is applicable to devices such as smart phones, cameras, and drones. The sequence model can be embedded in the chip of the shooting device, and real-time calculation can be achieved through hardware acceleration. In actual tests, the focusing speed of continuous shooting is increased by more than 50% compared with the traditional method, significantly improving the shooting success rate in dynamic scenes.

[0040] The actual test environment includes: Test 1: static portrait shooting in low light environment.

[0041] When shooting portraits in indoor low-light scenes, the system uses a sequence model to analyze the clarity of the facial area, adjust the focus and exposure, and obtain high-quality images.

[0042] Test 2: Following high-speed moving targets in sports competitions.

[0043] A multi-camera system is used to capture the target's motion trajectory. The main camera quickly focuses and the secondary camera assists in correction to ensure shooting clarity and continuity.

[0044] Test three, applied to drone monitoring. In multi-target monitoring tasks, the UAV system analyzes image sequences through sequence models and dynamically adjusts the focal length to continuously track key targets.

[0045] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer programs, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0046] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art, according to the concept of the present invention, some simple deductions, modifications or substitutions can be made.

Claims

1. A photographic sequence focusing method for a continuous snapshot function, characterized in that: include: In response to a first user operation, switching the camera device to enter a continuous shooting mode; Preprocessing at least N pre-shot image data marked with time node information collected according to a preset continuous shooting parameter to obtain a focus image sequence, wherein N is a natural number not less than 1; Inputting the focus image sequence into a preset focal length sequence acquisition neural network mathematical model to obtain continuous shooting image sequence feature change data output by the focal length sequence acquisition neural network mathematical model; Presetting the continuous shooting image focal length parameter according to the continuous shooting parameter and the continuous shooting image sequence characteristic change data; the continuous shooting image focal length parameter is used to set the focus parameter of each image taken by the camera device when performing continuous shooting; In response to a second user operation, continuous shooting is performed according to the continuous shooting image focal length parameter.

2. The photographic sequence focusing method according to claim 1, wherein: The preprocessing of at least N pre-shot image data marked with time node information collected according to a preset continuous shooting parameter includes: Each of the pre-shot image data is subjected to denoising and / or enhancement processing to obtain an element image in the focused image sequence; the denoising processing is used to remove noise points generated during the acquisition process of the pre-shot image data, and the enhancement processing is used to apply data enhancement technology to flip, crop, scale and / or adjust the illumination of the region of interest in the pre-shot image data.

3. The photographic sequence focusing method according to claim 1, characterized in that: The focal length sequence acquisition neural network mathematical model includes a recurrent neural network, a long short-term memory network and / or a transformer model; the recurrent neural network is used to extract the temporal dynamic characteristics of each element image in the focusing image sequence on the time axis; the long short-term memory network is used to extract the dynamic movement trend characteristics of the target object in each element image in the focusing image sequence on the time axis; The transformer model extracts global contextual correlation information features of each element image in the focused image sequence through a self-attention mechanism.

4. The photographic sequence focusing method according to claim 3, wherein: Inputting the focus image sequence into a preset focal length sequence to obtain a neural network mathematical model includes: The focusing image sequence is sequentially input into the recurrent neural network, the long short-term memory network and the transformer model to obtain feature change data of the continuous shooting image sequence.

5. The photographic sequence focusing method according to claim 1, characterized in that: Also includes: When the camera device is performing continuous shooting, at least one verification image data is obtained through the auxiliary camera device; wherein the auxiliary camera device and the camera device performing continuous shooting are synchronously photographed; Focus parameters of a camera device that performs continuous shooting are verified and fine-tuned based on the verification image data.

6. The photographic sequence focusing method according to claim 5, characterized in that: The step of verifying and fine-tuning the focus parameters of the camera device that performs continuous shooting according to the verification image data includes: Performing imaging quality scoring on the verification image data and the continuous-shooting image data taken by the camera device respectively; the imaging quality includes image clarity, depth of field accuracy and / or target position accuracy; When the imaging quality score of the verification image data is greater than the imaging quality score of the continuous-shooting image data, the focusing parameter of the camera device is fine-tuned according to the focusing parameter of the auxiliary camera device.

7. The photographic sequence focusing method according to claim 1, wherein: The continuous shooting parameters include continuous shooting frequency, continuous shooting quantity and scene type; The scene types include static scenes, dynamic scenes, special light source scenes, character scenes or landscape scenes, and the special light source scenes include low light environments, strong light environments or backlight environments; The focusing parameters include exposure factor, aperture factor, shutter speed factor and / or noise control factor; The continuous-shooting image sequence feature change data includes image definition change feature data, depth of field change feature data and / or target movement trajectory feature data on the time axis of each element image in the focus image sequence.

8. A computer program product comprising a computer program and / or instructions, characterized in that: When the computer program and / or instructions are executed by a processor, the photography sequence focusing method according to any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium, characterized in that: The medium stores a program, which can be executed by a processor to implement the photography sequence focusing method according to any one of claims 1 to 7.

10. A photographic sequence focusing system for a continuous snapshot function, characterized in that: Used to apply the photographic sequence focusing method according to any one of claims 1 to 7, the photographic sequence focusing system comprising: A sequence acquisition unit, used for pre-processing at least N pre-shot image data marked with time node information collected according to a preset continuous shooting parameter to obtain a focus image sequence, wherein N is a natural number not less than 1; A data processing unit, used for inputting the focus image sequence into a preset focal length sequence acquisition neural network mathematical model to obtain continuous shooting image sequence feature change data output by the focal length sequence acquisition neural network mathematical model; The parameter setting unit is used to preset the continuous shooting image focal length parameters according to the continuous shooting parameters and the continuous shooting image sequence characteristic change data; the continuous shooting image focal length parameters are used to set the focusing parameters of each image taken by the camera device when performing continuous shooting.

Images