Image processing method, electronic equipment and storage medium

Abstract

The invention discloses an image processing method, electronic equipment and a storage medium, in response to a shooting instruction, an image acquisition device is controlled based on a focal length group to change from an Nth focal length value to an Mth focal length value to continuously obtain multiple frames of acquisition images, and the Nth focal length value and the Mth focal length value are different and belong to the focal length group; an output streaming media file is generated based on the multiple frames of collection images, the streaming media file comprises multiple frames of output images, the size of a target object in each frame of multi-frame output image is unchanged, and a background part in each frame of multi-frame output image is matched with a background part of each frame of multi-frame collection image; wherein the continuously obtained multiple frames of acquisition images are multiple frames of acquisition images acquired by the electronic equipment with the image acquisition device at the same position.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically to an image processing method, electronic device, and storage medium. Background Art

[0002] Slide zoom is a special shooting method that creates a unique visual effect by simultaneously moving the camera and the subject. If the photographer cannot move the camera, this effect cannot be achieved, reducing shooting accuracy and visual quality, thereby degrading the user experience. Summary of the Invention

[0003] In view of this, this application provides the following technical solutions:

[0004] An image processing method comprising

[0005] In response to a shooting instruction, controlling the image acquisition device to change from an Nth focal length value to an Mth focal length value based on the focal length group to continuously acquire multiple frames of acquired images, wherein the Nth focal length value and the Mth focal length value are different and belong to the focal length group;

[0006] generating an output streaming media file based on the multiple frames of acquired images, the streaming media file including the multiple frames of output images, the size of the target object in each frame of the multiple frames of output images being constant, and the background portion of each frame of the multiple frames of output images matching the background portion of each frame of the multiple frames of acquired images;

[0007] The continuously obtained multiple frames of captured images are multiple frames of captured images captured by an electronic device having the image capture device at the same position.

[0008] Optionally, in response to the shooting instruction, controlling the image acquisition device to change from the Nth focal length value to the Mth focal length value based on the focal length group to continuously obtain multiple frames of captured images includes:

[0009] In response to the shooting instruction, an initial size of the target object corresponding to the shooting initial frame is obtained; the focal length value corresponding to the shooting initial frame is the Nth focal length value in the focal length group;

[0010] Determining an Mth focal length value in the focal length group based on a correspondence between a current size of the target object in the current frame and the initial size;

[0011] The image acquisition device is controlled to change from the Nth focal length value to the Mth focal length value to continuously obtain multiple frames of acquired images.

[0012] Optionally, determining the Mth focal length value in the focal length group based on the correspondence between the current size of the target object in the current frame and the initial size includes:

[0013] Determine a size change ratio between a current size of the target object in the current frame and the initial size;

[0014] If the size change ratio is less than a target threshold, determining an Mth focal length value in the focal length group, the Mth focal length value being greater than the Nth focal length value;

[0015] If the size change ratio is greater than the target threshold, an Mth focal length value is determined in the focal length group, and the Mth focal length value is smaller than the Nth focal length value.

[0016] Optionally, determining the Mth focal length value in the focal length group based on the correspondence between the current size of the target object in the current frame and the initial size includes:

[0017] determining a positional relationship between the target object and the electronic device based on a correspondence between a current size of the target object in the current frame and the initial size;

[0018] If the positional relationship indicates that the target object is far away from the electronic device, determining an Mth focal length value in the focal length group, the Mth focal length value being greater than the Nth focal length value;

[0019] If the positional relationship indicates that the target object is close to the electronic device, an Mth focal length value is determined in the focal length group, and the Mth focal length value is smaller than the Nth focal length value.

[0020] Optionally, generating an output streaming media file based on the multiple frames of captured images includes:

[0021] Determining, based on the focal length value corresponding to each frame, processing parameters for the target object in the captured image corresponding to the current frame;

[0022] Processing the target object based on the processing parameters so that the size of the target object in each frame of output image remains unchanged;

[0023] fusing the processed target object with a background portion of the acquired image corresponding to the frame to obtain an output image corresponding to the frame, wherein the background portion represents an image region in the acquired image excluding the target object;

[0024] Based on the multiple frames of output images, an output streaming media file is generated.

[0025] Optionally, determining, based on the focal length value corresponding to each frame, a processing parameter for the target object in the captured image corresponding to the current frame includes:

[0026] Determine a size change ratio of a current size of the target object in the current frame to an initial size of the target object in the initial frame;

[0027] Based on the focal length change ratio between the focal length value of the current frame and the focal length value of the initial frame and the size change ratio, a processing parameter for the target object in the captured image corresponding to the current frame is determined.

[0028] Optionally, generating an output streaming media file based on the multiple frames of the output image includes:

[0029] Performing motion compensation processing on the target object between adjacent frames in the multiple frames of output images to obtain a processed target object;

[0030] performing feature point compensation processing on background portions between adjacent frames in the multi-frame output image to obtain processed background portions;

[0031] An output streaming media file is generated based on the processed multiple frames of output images, wherein the processed multiple frames of output images are determined based on the processed target object and the processed background portion.

[0032] An electronic device, comprising:

[0033] Image acquisition device and processing device; wherein,

[0034] The image acquisition device is configured to continuously acquire multiple frames of acquired images in response to a shooting instruction, changing from an Nth focal length value to an Mth focal length value, wherein the Nth focal length value and the Mth focal length value are different and belong to the focal length group;

[0035] The processing device is configured to generate an output streaming media file based on the multiple frames of acquired images, wherein the streaming media file includes the multiple frames of output images, the size of the target object in each frame of the multiple frames of output images remains unchanged, and the background portion of each frame of the multiple frames of output images matches the background portion of each frame of the multiple frames of acquired images;

[0036] The continuously obtained multi-frame captured images are multi-frame captured images captured by the electronic device at the same position.

[0037] Optionally, the processing device is further configured to:

[0038] In response to the shooting instruction, an initial size of the target object corresponding to the shooting initial frame is obtained; the focal length value corresponding to the shooting initial frame is the Nth focal length value in the focal length group;

[0039] Determining an Mth focal length value in the focal length group based on a correspondence between a current size of the target object in the current frame and the initial size;

[0040] A control instruction is generated, where the control instruction is used to control the image acquisition device to change from the Nth focal length value to the Mth focal length value to continuously acquire multiple frames of acquired images.

[0041] A storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements:

[0042] In response to a shooting instruction, controlling the image acquisition device to change from an Nth focal length value to an Mth focal length value based on the focal length group to continuously acquire multiple frames of acquired images, wherein the Nth focal length value and the Mth focal length value are different and belong to the focal length group;

[0043] generating an output streaming media file based on the multiple frames of acquired images, the streaming media file including the multiple frames of output images, the size of the target object in each frame of the multiple frames of output images being constant, and the background portion of each frame of the multiple frames of output images matching the background portion of each frame of the multiple frames of acquired images;

[0044] The continuously obtained multiple frames of captured images are multiple frames of captured images captured by an electronic device having the image capture device at the same position. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0046] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of a target object during image acquisition in an application scenario provided by an embodiment of the present application;

[0048] Figure 3 A schematic diagram of a processing flow for an application scenario provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0051] In this application, the terms "first" and "second" are used to distinguish between different objects, not to describe a specific order. Furthermore, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements and may include steps or elements that are not listed.

[0052] The embodiments of the present application provide an image processing method that can be applied to scenarios such as image production and real-time video processing, and can be applied to electronic devices with image acquisition devices, such as smartphones, tablets, digital cameras, sports cameras, drones, virtual reality devices, etc. The method can, when the electronic device is fixed, respond to shooting instructions and control the image acquisition device to continuously change from corresponding focal length values ​​within a focal length group, acquiring multiple frames of captured images. Based on these captured images, the method generates an output streaming media file with a target object of unchanged size and a background that matches the captured images, achieving an automatic sliding zoom effect. This solves the problems of traditional sliding zoom that require the photographer to move the camera, cannot be automatically shot, and is difficult to achieve for single-person selfies. The method allows users to automatically shoot videos with a sliding zoom effect without changing the camera position, improving shooting quality and convenience.

[0053] See also Figure 1 , is a flow chart of an image processing method provided in an embodiment of the present application, which may include the following steps:

[0054] S101 . In response to a shooting instruction, control an image acquisition device based on a focal length group to change from an Nth focal length value to an Mth focal length value to continuously acquire multiple frames of acquired images.

[0055] In the embodiment of the present application, the continuously acquired multiple frames of captured images are multiple frames of captured images captured by an electronic device having an image capture device at the same position. The image capture device may be a camera of the electronic device (such as a front camera or a rear camera of the electronic device, or a connected external camera, etc.), and the shooting instruction may be a shooting instruction issued by the user of the electronic device, or a shooting instruction automatically generated by the electronic device under preset conditions, such as automatically starting the camera to shoot at a specified time. After the electronic device responds to the shooting instruction, the image capture device of the electronic device begins to perform the image capture task.

[0056] If the subject object (such as a user) photographed by the electronic device is in a stationary state, a fixed focal length value that matches its current state can be used for shooting. When it is detected that the subject object (such as a user) photographed by the electronic device is in a dynamic state, the image acquisition device will perform zoom shooting, that is, the shooting frames when the image acquisition device captures images all have corresponding focal length values, and image capture is performed based on the focal length value to obtain a captured image corresponding to the current shooting frame. In the embodiment of the present application, the focal length value of the current shooting frame will be determined based on the focal length value in the focal length group and the motion state of the current photographed subject object. Correspondingly, the zoom trajectory can also be pre-set to define a smooth change curve of the focal length from the first parameter to the second parameter. The user's movement will be detected during the shooting process, and the user's movement state will trigger the start of zooming. The focal length value corresponding to each shooting frame is determined based on the smooth change curve.

[0057] The focal length group contains multiple different focal length values, with the Nth focal length value being the initial focal length. During the capture process, the image acquisition device continuously changes from the Nth focal length value to the Mth focal length value (N≠M), continuously acquiring multiple frames of captured images. Because the electronic device is fixed in position, the movement of the target object (such as the user's face) in the captured multiple frames will cause its size to change in the image, and the background will also change accordingly.

[0058] S102: Generate an output streaming media file based on multiple frames of captured images.

[0059] The streaming media file includes multiple frames of output images. The size of the target object in each of the multiple frames of output images remains unchanged, and the background portion of each of the multiple frames of output images matches the background portion of each of the multiple frames of captured images. The target object can represent a moving object that appears in the captured image during the capture process. For example, the target object can be the main subject in the captured image. If the captured image includes a target user, the target object can represent the target user's main features, such as the target user's face or a specific area of ​​the target user. If the captured image includes multiple moving people, the target object can also be a specific person in the image, etc.

[0060] For each captured image frame, the target object is processed so that its size in the output image remains constant while the background matches that of the captured image. For example, as the target object approaches the camera, its size in the captured image increases. The system shrinks the target object so that the output image is the same size as the original, while the background remains the same as it was at the time of capture. This creates a sliding zoom effect with a relative background change.

[0061] For example, a user might be shooting a video on their phone alone and want a sliding zoom effect. With this feature enabled, the user moves in front of the camera, while the phone remains stationary. As the user moves closer to the phone, the camera automatically zooms out. The system processes the user's size in the video while the background shrinks, creating the effect of the background overwhelming the subject. As the user moves away from the phone, the camera automatically zooms in, magnifying the background and creating the illusion of the background moving away from the subject. This allows for seamless filming without the need for assistance from others.

[0062] The present application provides an image processing method that, while an electronic device is fixed in position, responds to a capture instruction and controls an image acquisition device to continuously change from the Nth focal length value to the Mth focal length value within a focal length group, acquiring multiple frames of captured images. Based on these captured images, an output streaming media file is generated with the target object unchanged in size and the background matching the captured images, thereby achieving an automatic sliding zoom effect. This method addresses the issues of traditional sliding zoom, such as requiring the photographer to move the camera, the inability to automatically capture images, and the difficulty of taking selfies by a single user. It allows users to automatically capture videos with a sliding zoom effect, and even allows single users to take selfies.

[0063] The following describes the image processing method according to the embodiment of the present application in conjunction with specific application scenarios.

[0064] In the embodiments of the present application, the focal length can be adjusted based on the movement of the target object, making the sliding zoom effect more in line with actual needs and improving the accuracy and adaptability of the focal length control. In one embodiment, in response to a shooting instruction, the process of controlling the image acquisition device to change from the Nth focal length value to the Mth focal length value based on the focal length group to continuously obtain multiple frames of captured images includes:

[0065] In response to a capture instruction, an initial size of a target object corresponding to an initial capture frame is obtained; based on a correspondence between the current size of the target object in the current frame and the initial size, an Mth focal length value is determined in a focal length group; and an image acquisition device is controlled to change from the Nth focal length value to the Mth focal length value to continuously acquire multiple frames of captured images. The focal length value corresponding to the initial capture frame is the Nth focal length value in the focal length group.

[0066] When a user triggers a capture command, the electronic device first captures an initial frame. At this point, the focal length is the Nth focal length value in the focal length group. The initial size of the target object in the initial frame is detected and recorded. For example, if the target object is the user's face, the initial size of the target object can be determined based on information such as the pixel area, width, or height of the face. During subsequent capture, the current frame image is captured in real time, and the current size of the target object in the current frame is detected. The Mth focal length value is determined by comparing the current size with the initial size. For example, the ratio of the current size to the initial size can be calculated. Alternatively, the correspondence between the current and initial sizes can be determined based on the difference between the two. If the target object continues to move away from the electronic device, its current size will be smaller than the initial size. In this case, the Mth focal length value is determined to be greater than the Nth focal length value, causing the camera lens to zoom in. If the target object continues to approach, its current size will be larger than the initial size. In this case, the Mth focal length value is determined to be less than the Nth focal length value, causing the camera lens to zoom out. After determining the Mth focal length value, the image capture device is controlled to continuously vary between the Nth and Mth focal length values ​​to capture multiple frames of images. This allows the focal length to be adjusted more accurately according to the movement of the target object, making the sliding zoom effect more in line with actual needs and improving the accuracy and adaptability of focal length control.

[0067] For example, when a teacher moves in front of a podium while recording a teaching video, the recording device records the initial size of the teacher's face and adjusts the focal length based on the changes in face size caused by the teacher's movement. This ensures that the teacher's size in the video remains consistent, and the background scales accordingly. This allows students to focus more clearly on the teacher without being distracted by background changes.

[0068] When the target object is in motion, the focus determination may be affected by unstable conditions caused by changes in the target object's posture. In the embodiment of the present application, a corresponding processing mode can be used to make the focus adjustment more stable and reliable. In one embodiment, the corresponding relationship of the target object size change is characterized by the target object size change ratio. Correspondingly, based on the corresponding relationship between the current size and the initial size of the target object in the current frame, the process of determining the Mth focus value in the focus group may include:

[0069] Determine a size change ratio between a current size and an initial size of a target object in a current frame; if the size change ratio is less than a target threshold, determine an Mth focal length value in the focal length group, and the Mth focal length value is greater than an Nth focal length value; if the size change ratio is greater than the target threshold, determine an Mth focal length value in the focal length group, and the Mth focal length value is less than the Nth focal length value.

[0070] The initial capture frame can be the first capture frame after the image acquisition device of the electronic device responds to the capture command, or it can be the capture frame after the user reaches a stable initial state after receiving the capture command. After obtaining the initial size of the target object in the capture initial frame, for the subsequent current frame, the size change ratio between the current size and the initial size is calculated (for example, if the current size is 0.8 times the initial size, the change ratio is 0.8). A target threshold (such as 0.9) is set. When the size change ratio is less than the target threshold, it indicates that the target object may be steadily moving away from the electronic device. At this time, the Mth focal length value in the focal length group is determined to be greater than the Nth focal length value, and the camera lens is zoomed in; when the size change ratio is greater than the target threshold, it indicates that the target object may be steadily approaching. The Mth focal length value is determined to be less than the Nth focal length value, and the camera lens is zoomed out. In this way, some unstable size changes caused by short-term posture changes (such as slight shaking of the user's head) are filtered out, avoiding frequent and meaningless adjustments of the focus, reducing the waste of processing resources, and ensuring the stability of the sliding zoom effect.

[0071] See also Figure 2 , which shows a schematic diagram of a target object in an image acquisition process in an application scenario provided by an embodiment of the present application, Figure 2 In the figure, only a part of the target object (i.e., a pet in motion) in the captured image is extracted as an example to illustrate the process of determining the focal length value. Figure 2 When shooting a moving subject outdoors with a fixed camera, such as a running puppy, the puppy may sway while running, causing its size in the image to temporarily change. After setting a target threshold, the focus is adjusted only when the size change ratio consistently exceeds or falls below the threshold. For example, if the running puppy is actually moving away from the camera and its size change ratio remains below the threshold, the camera lens will maintain a stable zoom. This prevents incorrect focus adjustments caused by temporary size changes such as the puppy's running movement. This ensures that the puppy in the video is stable in size and the background scales naturally.

[0072] In one embodiment of the present application, the focal length value for each captured frame during image capture can also be determined based on the motion characteristics of the target object, so that the sliding zoom effect more closely matches the actual movement of the target object, thereby improving the smoothness and naturalness of the generated video. Accordingly, based on the correspondence between the current size and the initial size of the target object in the current frame, the process of determining the Mth focal length value in the focal length group includes:

[0073] Based on the correspondence between the current size and the initial size of the target object in the current frame, the positional relationship between the target object and the electronic device is determined; if the positional relationship indicates that the target object is far away from the electronic device, the Mth focal length value is determined in the focal length group, and the Mth focal length value is greater than the Nth focal length value; if the positional relationship indicates that the target object is close to the electronic device, the Mth focal length value is determined in the focal length group, and the Mth focal length value is less than the Nth focal length value.

[0074] First, the initial size of the target object is recorded in the initial frame of the shot. In subsequent shots, the current size of the target object in the current frame is obtained, and by comparing the current size with the initial size, it is determined whether the target object is approaching or moving away from the electronic device. For example, if the current size is smaller than the initial size, it means that the target object is moving away; if the current size is larger than the initial size, it means that the target object is approaching. Furthermore, the moving speed of the target object can also be determined. The moving speed (such as the proportion of size change per unit time) can be calculated by the rate of change of the target object size in multiple consecutive frames. Then, according to the preset mapping relationship between the moving speed and the focal length value, the corresponding Mth focal length value is determined. For example, the faster the moving speed, the larger the change in the corresponding focal length value; the slower the moving speed, the smaller the change in the focal length value. In this way, when the target object moves away quickly, the camera lens zooms in quickly; when it approaches slowly, the lens zooms out slowly, so that the focal length adjustment is more in line with the actual movement of the target object.

[0075] For example, a fixed camera captures an athlete running on a track, with the athlete's distance from the lens constantly changing. The system determines whether the athlete is moving away from or toward the lens based on the athlete's size changes in the image and calculates their movement speed. When the athlete moves away from the lens quickly, their movement speed is high. Based on the mapping between movement speed and focal length, the system determines a larger focal length (Mth). The camera lens quickly zooms in to ensure the athlete's size remains constant in the frame, while the background quickly scales in and out to emphasize the athlete's motion. When the athlete slows down or approaches the lens, the focal length adjusts accordingly to create a natural effect.

[0076] In each of the above embodiments, the processing process of dynamic zoom during image acquisition by the image acquisition device of the electronic device is described, so that the dynamic zoom process of the image acquisition device is more in line with the movement characteristics of the target object and the stability of the zoom effect is ensured.

[0077] The following describes the process of generating an output streaming media file in an embodiment of the present application. In one embodiment of the present application, the process of generating an output streaming media file based on multiple frames of acquired images includes: determining processing parameters for a target object in the acquired image corresponding to the current frame based on the focal length value corresponding to each frame; processing the target object based on the processing parameters so that the size of the target object in each frame of the output image remains unchanged; fusing the processed target object with the background portion of the acquired image corresponding to the frame to obtain an output image corresponding to the frame, wherein the background portion represents the image area in the acquired image other than the target object; and generating an output streaming media file based on the multiple frames of output images.

[0078] In a typical processing mode, if an electronic device is located at the same position and captures multiple frames of captured images, the image capture device (such as a camera or a mobile phone camera) directly outputs the original captured image, and the change in focal length causes all contents in the image to be scaled synchronously. For example, if the lens is zoomed in (the focal length is increased), the target object and the background in the image will both become larger. However, in the embodiment of the present application, it is necessary to process the image captured by the image capture device, and process the size of the target in the captured image so that the size of the target object in each frame of the output image remains unchanged. The background portion in the output image can use the background portion in the captured image without being processed, thereby preserving the perspective change caused by the zoom.

[0079] For each captured image frame, the corresponding focal length value is first determined. Then, based on this focal length value and the size relationship between the target object in the initial and current frames, processing parameters (such as the scaling ratio) are determined. For example, if the current frame's focal length value is greater than the initial frame's, this indicates that the camera lens is zoomed in, and the target object's size in the captured image may have become smaller due to distance. To maintain the same size of the target object in the output image, the target object must be enlarged by a certain ratio. Based on the determined processing parameters, the target object is processed, such as by scaling and cropping, so that its size in the output image is consistent with that in the initial frame. After processing is complete, the processed target object is fused with the background portion (the area excluding the target object) of the current frame's captured image to obtain the output image for that frame. During the fusion process, the transition between the target object and the background is ensured to be natural, with no obvious stitching artifacts. Finally, the multiple output image frames are sequentially combined to generate the output streaming media file.

[0080] For example, when shooting a product display video, the camera position can be fixed, with the product as the target object, and different angles can be displayed by moving the product. The system determines the processing parameters based on the focal length value of each frame and the change in product size, and processes the product so that its size remains consistent in the video. The background scales accordingly as the product moves and the focal length changes, highlighting the product details and allowing the audience to clearly observe each part of the product. At the same time, the change in background also makes the video more enjoyable.

[0081] In one implementation of the embodiment of the present application, based on the focal length value corresponding to each frame, a process of determining processing parameters for a target object in a captured image corresponding to a current frame may include the following steps:

[0082] Determine a size change ratio between a current size of a target object in a current frame and an initial size of a target object in an initial frame; and determine processing parameters for the target object in a captured image corresponding to the current frame based on a focal length change ratio and a size change ratio between a focal length value of the current frame and a focal length value of the initial frame.

[0083] The processing parameters are determined by combining the ratio of the target object's current size to its initial size in the current frame, as well as the ratio of the focal length change in the current frame to the focal length in the initial frame. This approach establishes a constraint relationship between the two, resulting in better inverse scaling, more accurately ensuring that the target object's size remains unchanged in the output image, and improving the accuracy and effectiveness of image processing.

[0084] When capturing the initial frame, record the initial size of the target object and the corresponding initial focal length value (Nth focal length value). For the current frame, obtain the current size of the target object and the corresponding current focal length value. Calculate the size change ratio between the current size and the initial size, as well as the focal length change ratio between the current focal length value and the initial focal length value. For example, if the initial size is S0 and the current size is S1, the size change ratio is S1 / S0; if the initial focal length value is F0 and the current focal length value is F1, the focal length change ratio is F1 / F0. Then, determine the processing parameters based on these two ratios. A function or rule can be set to combine these two ratios to calculate the processing parameters, such as if the processing parameter is equal to a certain functional relationship (such as a product, ratio, etc.) between the size change ratio and the focal length change ratio, thereby determining the ratio for reverse scaling of the target object so that the size of the processed target object in the output image is consistent with the initial size, thereby improving the accuracy and effect of image processing.

[0085] In the embodiment of the present application, it is necessary to generate an output streaming media file. Since the target object in each frame is processed and fused with the background portion of the captured image, in order to ensure the output effect of the output streaming media file, the embodiment of the present application performs relevant processing during image fusion, thereby making the video image smoother and more stable, and improving the viewing experience. Accordingly, the process of generating the output streaming media file based on multiple frames of the output image may include the following steps:

[0086] Performing motion compensation processing on a target object between adjacent frames in a multi-frame output image to obtain a processed target object; performing feature point compensation processing on a background portion between adjacent frames in the multi-frame output image to obtain a processed background portion; and generating an output streaming media file based on the processed multi-frame output image, wherein the processed multi-frame output image is determined based on the processed target object and the processed background portion.

[0087] After generating multiple frames of output images, motion compensation is performed on the target object between adjacent frames. By analyzing the changes in the target object's position, posture, and other aspects in adjacent frames, interpolation and prediction algorithms are used to compensate for the target object's motion, making the transition between adjacent frames smoother and reducing the jitter and discontinuity caused by the target object's movement. At the same time, feature point compensation is performed on the background between adjacent frames. Feature points in the background (such as corners and edges) are detected, and based on the changes in the feature points in adjacent frames, the background is transformed and compensated, making the transition between adjacent frames natural and eliminating background jitter and misalignment caused by slight changes in focal length and shooting angle. Finally, the processed target object and background are combined into a processed multi-frame output image, and these images are then sequentially generated into output streaming media files to ensure the overall smoothness and stability of the video, with good visual effects.

[0088] The following describes an application scenario in which a user is photographed at a fixed camera position to obtain a sliding zoom video as an example to illustrate the image processing method of an embodiment of the present application. The target object is the user's face area. When the user starts shooting, the size of the user's face area is recorded to determine whether the user is close to or far away from the camera during shooting, and automatically controls the camera lens to perform continuous and smooth zooming. When the camera performs continuous and smooth zooming, the image zoom factor is calculated based on the size of the face area in each frame image and the recorded face area size. The user area image in the frame image is scaled in reverse and then fused with other area images. In this way, the size of the user in the new image remains unchanged, but the background is scaled, thus achieving the Hitchcock zoom effect. The user can automatically shoot a video with a sliding zoom effect, and the photographer can shoot a video with a sliding zoom effect for himself.

[0089] See also Figure 3 , is a schematic diagram of a processing flow of an application scenario provided by an embodiment of the present application. The process may include the following steps:

[0090] S301: When the user starts shooting, the size of the user's face area in the start frame capture image is recorded.

[0091] S302. In response to user movement, determine whether the user is close to or far away from the camera based on the size of the user's face area in the current frame captured image. If the user is close to the camera, execute step S303; if the user is far away from the camera, execute step S304.

[0092] S303: Control the camera lens to zoom out continuously and smoothly.

[0093] S304: Control the camera lens to zoom in continuously and smoothly.

[0094] S305 . For each frame of captured image, calculate a scaling parameter for the face region size according to the face region size in the current image and the recorded initial face region size.

[0095] S306 : Process the face area based on the scaling parameter and fuse it with the background area image to obtain an output image of the current frame.

[0096] S307: Generate a video based on the processed output images corresponding to the frames.

[0097] For example, the face area A0 in the initial frame is 100*100 pixels;

[0098] As the user approaches the camera, the size of the face area in the current frame changes to A1 = 150 * 150 pixels. At this time, the scaling parameter R = A0 / A1 ≈ 0.67, which means that the face area in the current frame needs to be reduced to 100 * 100 pixels compared to the initial frame. At this time, the background area is enlarged by 1.5 compared to the initial frame due to the current focal length. As the user moves away from the camera, the size of the face area in the current frame changes to A1 = 80 * 80 pixels. At this time, the scaling parameter R = A0 / A1 = 1.25, which means that the face area in the current frame needs to be enlarged to 100 * 100 pixels compared to the initial frame. At this time, the background area is reduced by 0.8 times compared to the initial frame due to the current focal length. In the resulting new image, the size of the user's face area remains unchanged, but the background is scaled, achieving the Hitchcock zoom effect.

[0099] In the embodiment of the present application, an electronic device is also provided. Figure 4 , the electronic device comprises:

[0100] Image acquisition device 401 and processing device 402.

[0101] An image acquisition device 401 is configured to continuously acquire multiple frames of acquired images in response to a shooting instruction, changing from an Nth focal length value to an Mth focal length value, wherein the Nth focal length value and the Mth focal length value are different and belong to the focal length group;

[0102] a processing device 402 configured to generate an output streaming media file based on the multiple frames of acquired images, wherein the streaming media file includes the multiple frames of output images, the size of the target object in each frame of the multiple frames of output images being constant, and the background portion of each frame of the multiple frames of output images matching the background portion of each frame of the multiple frames of acquired images;

[0103] The continuously obtained multi-frame captured images are multi-frame captured images captured by the electronic device at the same position.

[0104] Optionally, the processing device 402 is further configured to:

[0105] In response to the shooting instruction, an initial size of the target object corresponding to the shooting initial frame is obtained; the focal length value corresponding to the shooting initial frame is the Nth focal length value in the focal length group;

[0106] Determining an Mth focal length value in the focal length group based on a correspondence between a current size of the target object in the current frame and the initial size;

[0107] A control instruction is generated, where the control instruction is used to control the image acquisition device to change from the Nth focal length value to the Mth focal length value to continuously acquire multiple frames of acquired images.

[0108] Optionally, determining the Mth focal length value in the focal length group based on the correspondence between the current size of the target object in the current frame and the initial size includes:

[0109] Determine a size change ratio between a current size of the target object in the current frame and the initial size;

[0110] If the size change ratio is less than a target threshold, determining an Mth focal length value in the focal length group, the Mth focal length value being greater than the Nth focal length value;

[0111] If the size change ratio is greater than the target threshold, an Mth focal length value is determined in the focal length group, and the Mth focal length value is smaller than the Nth focal length value.

[0112] Optionally, determining the Mth focal length value in the focal length group based on the correspondence between the current size of the target object in the current frame and the initial size includes:

[0113] determining a positional relationship between the target object and the electronic device based on a correspondence between a current size of the target object in the current frame and the initial size;

[0114] If the positional relationship indicates that the target object is far away from the electronic device, determining an Mth focal length value in the focal length group, the Mth focal length value being greater than the Nth focal length value;

[0115] If the positional relationship indicates that the target object is close to the electronic device, an Mth focal length value is determined in the focal length group, and the Mth focal length value is smaller than the Nth focal length value.

[0116] Optionally, generating an output streaming media file based on the multiple frames of captured images includes:

[0117] Determining, based on the focal length value corresponding to each frame, processing parameters for the target object in the captured image corresponding to the current frame;

[0118] Processing the target object based on the processing parameters so that the size of the target object in each frame of output image remains unchanged;

[0119] fusing the processed target object with a background portion of the acquired image corresponding to the frame to obtain an output image corresponding to the frame, wherein the background portion represents an image region in the acquired image excluding the target object;

[0120] Based on the multiple frames of output images, an output streaming media file is generated.

[0121] Optionally, determining, based on the focal length value corresponding to each frame, a processing parameter for the target object in the captured image corresponding to the current frame includes:

[0122] Determine a size change ratio of a current size of the target object in the current frame to an initial size of the target object in the initial frame;

[0123] Based on the focal length change ratio between the focal length value of the current frame and the focal length value of the initial frame and the size change ratio, a processing parameter for the target object in the captured image corresponding to the current frame is determined.

[0124] Optionally, generating an output streaming media file based on the multiple frames of the output image includes:

[0125] Performing motion compensation processing on the target object between adjacent frames in the multiple frames of output images to obtain a processed target object;

[0126] performing feature point compensation processing on background portions between adjacent frames in the multi-frame output image to obtain processed background portions;

[0127] An output streaming media file is generated based on the processed multiple frames of output images, wherein the processed multiple frames of output images are determined based on the processed target object and the processed background portion.

[0128] In another embodiment of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the computer program implements:

[0129] In response to a shooting instruction, controlling the image acquisition device to change from an Nth focal length value to an Mth focal length value based on the focal length group to continuously acquire multiple frames of acquired images, wherein the Nth focal length value and the Mth focal length value are different and belong to the focal length group;

[0130] generating an output streaming media file based on the multiple frames of acquired images, the streaming media file including the multiple frames of output images, the size of the target object in each frame of the multiple frames of output images being constant, and the background portion of each frame of the multiple frames of output images matching the background portion of each frame of the multiple frames of acquired images;

[0131] The continuously obtained multiple frames of captured images are multiple frames of captured images captured by an electronic device having the image capture device at the same position.

[0132] In the embodiments of the present application, for example, a mobile phone with a camera can capture a video file with a sliding zoom visual effect on the subject when the operator is motionless or without an operator (Selfie). In other words, in the embodiments of the present application, the operator maintains a constant relative position to the mobile phone during the video capture process, and a video file with a sliding zoom visual effect is captured on the subject.

[0133] It should be noted that the specific implementation of the processor in this embodiment can refer to the corresponding content in the previous text and will not be described in detail here.

[0134] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0135] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0136] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0137] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, comprising: In response to a shooting instruction, controlling the image acquisition device to change from an Nth focal length value to an Mth focal length value based on the focal length group to continuously acquire multiple frames of acquired images, wherein the Nth focal length value and the Mth focal length value are different and belong to the focal length group; generating an output streaming media file based on the multiple frames of acquired images, the streaming media file including the multiple frames of output images, the size of the target object in each frame of the multiple frames of output images being constant, and the background portion of each frame of the multiple frames of output images matching the background portion of each frame of the multiple frames of acquired images; in, The continuously obtained multi-frame captured images are multi-frame captured images captured by an electronic device having the image capture device at the same position.

2. The method according to claim 1, wherein, in response to a shooting instruction, controlling the image acquisition device to change from an Nth focal length value to an Mth focal length value based on the focal length group to continuously acquire multiple frames of captured images comprises: In response to a shooting instruction, obtaining an initial size of a target object corresponding to an initial shooting frame; The focal length value corresponding to the shooting initial frame is the Nth focal length value in the focal length group; Determining an Mth focal length value in the focal length group based on a correspondence between a current size of the target object in the current frame and the initial size; The image acquisition device is controlled to change from the Nth focal length value to the Mth focal length value to continuously obtain multiple frames of acquired images.

3. The method according to claim 2, wherein determining the Mth focal length value in the focal length group based on the correspondence between the current size of the target object in the current frame and the initial size comprises: Determine a size change ratio between a current size of the target object in the current frame and the initial size; If the size change ratio is less than a target threshold, determining an Mth focal length value in the focal length group, the Mth focal length value being greater than the Nth focal length value; If the size change ratio is greater than the target threshold, an Mth focal length value is determined in the focal length group, and the Mth focal length value is smaller than the Nth focal length value.

4. The method according to claim 2, wherein determining the Mth focal length value in the focal length group based on the correspondence between the current size of the target object in the current frame and the initial size comprises: determining a positional relationship between the target object and the electronic device based on a correspondence between a current size of the target object in the current frame and the initial size; If the positional relationship indicates that the target object is far away from the electronic device, determining an Mth focal length value in the focal length group, the Mth focal length value being greater than the Nth focal length value; If the positional relationship indicates that the target object is close to the electronic device, an Mth focal length value is determined in the focal length group, and the Mth focal length value is smaller than the Nth focal length value.

5. The method according to claim 1, wherein generating an output streaming media file based on the multiple frames of captured images comprises: Determining, based on the focal length value corresponding to each frame, processing parameters for the target object in the captured image corresponding to the current frame; Processing the target object based on the processing parameters so that the size of the target object in each frame of output image remains unchanged; fusing the processed target object with a background portion of the acquired image corresponding to the frame to obtain an output image corresponding to the frame, wherein the background portion represents an image region in the acquired image excluding the target object; Based on the multiple frames of output images, an output streaming media file is generated.

6. The method according to claim 5, wherein determining the processing parameters for the target object in the captured image corresponding to the current frame based on the focal length value corresponding to each frame comprises: Determine a size change ratio of a current size of the target object in the current frame to an initial size of the target object in the initial frame; Based on the focal length change ratio between the focal length value of the current frame and the focal length value of the initial frame and the size change ratio, a processing parameter for the target object in the captured image corresponding to the current frame is determined.

7. The method according to claim 5, wherein generating an output streaming media file based on the multiple frames of the output image comprises: Performing motion compensation processing on the target object between adjacent frames in the multiple frames of output images to obtain a processed target object; performing feature point compensation processing on background portions between adjacent frames in the multi-frame output image to obtain processed background portions; An output streaming media file is generated based on the processed multiple frames of output images, wherein the processed multiple frames of output images are determined based on the processed target object and the processed background portion.

8. An electronic device comprising: Image acquisition device and processing device; wherein, The image acquisition device is configured to continuously acquire multiple frames of acquired images in response to a shooting instruction, changing from an Nth focal length value to an Mth focal length value, wherein the Nth focal length value and the Mth focal length value are different and belong to the focal length group; The processing device is configured to generate an output streaming media file based on the multiple frames of acquired images, wherein the streaming media file includes the multiple frames of output images, the size of the target object in each frame of the multiple frames of output images remains unchanged, and the background portion of each frame of the multiple frames of output images matches the background portion of each frame of the multiple frames of acquired images; The continuously obtained multi-frame captured images are multi-frame captured images captured by the electronic device at the same position.

9. The electronic device according to claim 8, wherein the processing device is further configured to: In response to the shooting instruction, an initial size of the target object corresponding to the shooting initial frame is obtained; the focal length value corresponding to the shooting initial frame is the Nth focal length value in the focal length group; Determining an Mth focal length value in the focal length group based on a correspondence between a current size of the target object in the current frame and the initial size; A control instruction is generated, where the control instruction is used to control the image acquisition device to change from the Nth focal length value to the Mth focal length value to continuously acquire multiple frames of acquired images.

10. A storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements: In response to a shooting instruction, controlling the image acquisition device to change from an Nth focal length value to an Mth focal length value based on the focal length group to continuously acquire multiple frames of acquired images, wherein the Nth focal length value and the Mth focal length value are different and belong to the focal length group; generating an output streaming media file based on the multiple frames of acquired images, the streaming media file including the multiple frames of output images, the size of the target object in each frame of the multiple frames of output images being constant, and the background portion of each frame of the multiple frames of output images matching the background portion of each frame of the multiple frames of acquired images; in, The continuously obtained multi-frame captured images are multi-frame captured images captured by an electronic device having the image capture device at the same position.

Images