Image Processing Method and Apparatus, Electronic Device, Storage Medium

Abstract

The present disclosure relates to an image processing method and apparatus, an electronic device, and a storage medium. Among them, the method includes: obtaining a plurality of image frames captured for a subject and a plurality of shooting state parameters respectively used when shooting the plurality of image frames, where the subject is located at the same position in each image frame; determining respectively the scaling ratios corresponding to the plurality of shooting state parameters according to a preset correspondence between shooting state parameters and scaling ratios; wherein, after scaling the image frame captured under the corresponding shooting state parameter by the scaling ratio in any one of the correspondences, the size of the subject in the scaled target image frame is the same as the size of the subject in the image reference frame; scaling the corresponding image frames according to the determined scaling ratios to obtain a plurality of target image frames, and encoding the obtained plurality of target image frames into a video file.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular, to an image processing method, an apparatus, an electronic device, and a storage medium. Background Art

[0002] The Hitchcock zoom, also known as the dolly zoom or the sliding zoom, is an advanced shooting technique in the field of video shooting. It mainly changes the focal length while pushing or pulling the camera position, so that in the captured video, the subject remains roughly unchanged, but the background space is compressed or enlarged, thus creating emotions such as anxiety and panic for the viewer. This video special effect generated by the Hitchcock zoom is called the Hitchcock effect.

[0003] In the related art, if a video frame with the Hitchcock effect needs to be captured, the shooting technique of the Hitchcock zoom must be strictly followed. The change rate of the object distance between the camera position and the subject and the change rate of the focal length of the camera must both be kept stable, and the change rates of the two satisfy a certain mathematical relationship. Usually, only professional people who master advanced photography skills can meet the above requirements, and a large number of shooting tools are required, resulting in a high shooting cost. Summary of the Invention

[0004] The present disclosure provides an image processing method, an apparatus, an electronic device, and a storage medium, which can obtain a video file with the Hitchcock effect without using the Hitchcock zoom.

[0005] According to a first aspect of the present disclosure, there is provided an image processing method, including:

[0006] Obtaining a plurality of image frames captured for a subject and a plurality of shooting state parameters respectively used when shooting the plurality of image frames, where the subject is located at the same position in each image frame;

[0007] According to a preset correspondence between the shooting state parameters and the scaling ratios, respectively determining the scaling ratios corresponding to the plurality of shooting state parameters; wherein, after scaling the image frame captured under the corresponding shooting state parameter by the scaling ratio in any correspondence, the size of the subject in the scaled target image frame is the same as the size of the subject in the image reference frame;

[0008] Scaling the corresponding image frames according to the determined scaling ratios to obtain a plurality of target image frames, and encoding the obtained plurality of target image frames into a video file.

[0009] According to a second aspect of the present disclosure, there is provided an image processing apparatus, including:

[0010] An acquisition unit that acquires a plurality of image frames captured for a subject and a plurality of shooting state parameters respectively used when shooting the plurality of image frames, where the subject is located at the same position in each image frame;

[0011] A determination unit that respectively determines scaling ratios corresponding to the plurality of shooting state parameters according to a preset correspondence between shooting state parameters and scaling ratios; wherein, after scaling the image frame captured under the corresponding shooting state parameter by the scaling ratio in any one of the correspondences, the size of the subject in the scaled target image frame is the same as the size of the subject in the image reference frame;

[0012] A scaling unit that scales the corresponding image frames according to the determined scaling ratios to obtain a plurality of target image frames, and encodes the obtained plurality of target image frames into a video file.

[0013] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0014] A processor;

[0015] A memory for storing instructions executable by the processor;

[0016] Wherein, the processor realizes the method as described in the first aspect by running the executable instructions.

[0017] According to a fourth aspect of the present disclosure, there is provided a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method as described in the first aspect are realized.

[0018] In the technical solution of the present disclosure, after obtaining a plurality of image frames and the shooting state parameters respectively used when shooting the plurality of image frames, the scaling ratios corresponding to the plurality of shooting state parameters can be respectively determined according to a preset correspondence between shooting state parameters and scaling ratios, and then the corresponding image frames are scaled by the determined scaling ratios to obtain a plurality of target image frames. Among them, after scaling the image frame captured under the corresponding shooting state parameter by the scaling ratio in any one of the correspondences, the size of the subject in the scaled target image frame is the same as the size in the image reference frame.

[0019] It should be understood that after the plurality of obtained image frames are scaled by the corresponding scaling ratios, the size of the subject in the obtained plurality of target image frames is the same as the size of the subject in the image reference frame, which is equivalent to the size of the subject in the plurality of target image frames being the same. Therefore, after encoding the plurality of target image frames into a video file, the size of the subject in the video picture remains unchanged.

[0020] In actual operation, the photographer can change the shooting state parameters to cause changes in the background space, and then offset the size change of the subject during the previous shooting process by scaling the image frames as described above, so that the subject in the finally processed video remains unchanged and the background space changes, achieving the Hitchcock effect.

[0021] Compared with the related art, which realizes the Hitchcock effect through the shooting technique of Hitchcock zoom, the technical solution of the present disclosure no longer relies on the offset effect of the focal length change and the object distance change on the imaging change. Instead, starting from the presentation effect of the Hitchcock effect, multiple image frames with continuously changing background spaces are obtained through the previous shooting, and the size change of the subject during the previous shooting process is offset by the later scaling method. This makes the technical solution of the present disclosure have extremely low requirements for the previous shooting skills and shooting equipment, avoiding the problems of excessive cost caused by the need for additional shooting equipment in the related art and the high technical threshold for the photographer. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure.

[0023] Figure 1 It is a schematic diagram of an imaging principle shown in an exemplary embodiment of the present disclosure;

[0024] Figure 2 It is a flowchart of an image processing method shown in an exemplary embodiment of the present disclosure;

[0025] Figure 3 It is a flowchart of another image processing method shown in an exemplary embodiment of the present disclosure;

[0026] Figure 4 It is a flowchart of yet another image processing method shown in an exemplary embodiment of the present disclosure;

[0027] Figure 5 It is a flowchart of a method for realizing the Hitchcock effect shown in an exemplary embodiment of the present disclosure;

[0028] Figure 6 It is a block diagram of an image processing device shown in an exemplary embodiment of the present disclosure;

[0029] Figure 7 It is a schematic diagram of the structure of an electronic device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0031] The terms used in the present disclosure are for the purpose of describing particular embodiments only and are not intended to limit the present disclosure. The singular forms "a", "the", and "said" used in the present disclosure and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0032] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0033] The Hitchcock zoom, also known as the moving zoom or sliding zoom, is an advanced shooting technique in the field of video shooting. It mainly creates a visual effect in the captured video where the subject remains roughly the same, but the background space is compressed or enlarged by changing the focal length while pushing or pulling the camera position, thereby creating emotions such as anxiety and panic for the viewer. This video special effect produced by the Hitchcock zoom is called the Hitchcock effect.

[0034] Since this video special effect first appeared in the 1958 film Vertigo directed by the suspense master Alfred Hitchcock, it was named the Hitchcock effect, and the corresponding shooting technique is called the Hitchcock zoom.

[0035] When actually using the Hitchcock zoom, if the camera position is continuously approaching the subject, the cameraman needs to reduce the focal length of the camera at a certain rate according to the speed of the camera movement; while when the camera position is continuously moving away from the subject, the cameraman needs to increase the focal length of the camera at a certain rate according to the speed of the camera movement. The theoretical basis is as follows:

[0036] If the object height of the subject is h, in the initial state, the object distance is u1, the image distance is v1, the focal length of the camera's lens is f1, and the image height of the subject's image is h1'. According to the imaging formula, we can get:

[0037]

[0038] After public conversion, we can get:

[0039]

[0040] The imaging magnification M1 at this time is:

[0041]

[0042] And because the magnification can also be expressed as the ratio of image height to object height, that is,

[0043]

[0044] Then from formulas (3) and (4), we can get:

[0045]

[0046] Because the object distance is much larger than the camera focal length, it can be approximated as:

[0047]

[0048] When the camera operator changes the focal length of the camera head from f1 to f2, and the camera operator and the camera move forward and backward on the slide rail, the object distance of the subject changes from u1 to u2, and the image height of the subject image changes from h1' to h2'. Similar to formula (5), we can get:

[0049]

[0050] Then the change in image height caused by the change in camera focal length, that is, the change in the size of the subject image, is

[0051]

[0052] The characteristic of the Hitchcock camera effect is to keep the image size of the subject unchanged, even if Δh in formula (7) is equal to 0. Thus, the following relationship can be obtained:

[0053]

[0054] That is, the ratio of the focal length before and after the change is proportional to the ratio of the object distance. In other words, when the Hitchcott zoom is actually used, if the object distance is increased, the focal length of the camera needs to be increased as well.

[0055] Next, combine Figure 1Furthermore, introduce the theoretical basis for the change in the image size of the background in the picture while keeping the image size of the subject unchanged. And from the above introduction, it can be seen that as long as the change ratio of the object distance is exactly proportional to the change ratio of the focal length, the image size of the subject can be kept from changing significantly.

[0056] As Figure 1 shown, there are two object points in the scene. S is an object point on the subject, and B is an object point on the background. When the camera moves from position P1 to position P2, the object distance of the subject object point S changes from SP1 = u1 to SP2 = u2, and the object distance of the background object point B changes from BP1 = w1 to BP2 = w2. The included angle SP1B = θ1, and the included angle SP2B = θ2. From the geometric relationship, we can get:

[0057] u 1 = w 1 cosθ 1 (9)

[0058] u 2 = w 2 cosθ 2 (10)

[0059] Combined with formula (8), we can get (11):

[0060]

[0061] Because θ 1 < θ 2 , so,

[0062]

[0063] The image height of the background object point is g, and the change in the image height before and after is Δg'. Imitating formula (7), we can get the image height change formula (13) of the background object point.

[0064]

[0065] Combined with formula (12), it can be seen that formula (13) is not equal to 0, that is

[0066] Δg'≠0 (14)

[0067] Thus, it is proved that when the image height of the object point on the subject does not change, the image height of the object point in the background space will necessarily change.

[0068] However, it should be noted that the basis of the above proof process is that the subject must be located at the center of the picture, and the object distance between the subject and the camera should not be too small to avoid the object point on the subject deviating too much from the center of the picture.

[0069] As shown in formula (11), since the background object point B is not at the center of the screen, when the object distance of the camera changes, the viewing angle of the background object point B changes. Therefore, the change in the image height caused by the change in the object distance and the change in the focal length cannot cancel each other out. For the subject to be photographed, only a small part of the object points are located exactly at the center of the screen, and most of the subject object points also have a certain viewing angle. However, since the viewing angle of this part of the object points is relatively small, the change in the viewing angle caused by the change in the object distance is not significant and is approximately 0. Therefore, it is approximately considered that the change in the image height is canceled out by each other.

[0070] As can be seen from the above introduction, when shooting a video screen with the Hitchcock effect, the shooting method of the Hitchcock zoom must be strictly adhered to. The change rate of the object distance between the camera position and the subject to be photographed and the change rate of the focal length of the camera must both be kept stable, and the change rates of the two satisfy a certain mathematical relationship, that is, the ratio of the object distances before and after the change and the ratio of the focal lengths always maintain a proportional relationship. Usually, only professional people who master advanced photography techniques can meet the above requirements, and a large number of shooting tools are needed, resulting in a high shooting cost.

[0071] For example, in a related technology, a slide rail needs to be used to control the camera to approach the subject to be photographed at a stable rate. And while moving the camera position, the focal length of the camera is adjusted accordingly by manually controlling the focal length. In another related technology, the camera is kept approaching the subject to be photographed at a stable rate by means of the stable flight of a drone, but the change rate of the focal length of the camera needs to be calculated and set in advance.

[0072] Therefore, the present disclosure proposes an image processing method to avoid the problems in related technologies that when shooting a video screen with the Hitchcock effect, additional instruments are needed, resulting in a high hardware cost, and the problem of high technical requirements for the shooter.

[0073] Figure 2 An image processing method shown in an exemplary embodiment of the present disclosure. As Figure 2 shown, the method may include the following steps:

[0074] Step 202, obtaining a plurality of image frames photographed for the subject to be photographed and a plurality of shooting state parameters respectively used when shooting the plurality of image frames, where the subject to be photographed is located at the same position in each image frame.

[0075] As can be seen from the above introduction to the Hitchcock zoom, in the related art, when shooting a video frame with the Hitchcock effect, the following principle is utilized: when the focal length change and the object distance change satisfy a certain mathematical relationship, the effects of the two on the imaging change of the subject located at the center of the frame cancel each other out. Based on this, the shooter can use equipment and / or techniques to make the focal length change and the object distance change during the shooting process satisfy the above mathematical relationship as much as possible, so that the resulting video frame contains the Hitchcock effect.

[0076] It is not difficult to understand that the related art starts from the generation principle of the Hitchcock effect and obtains a video frame with the Hitchcock effect through pure pre-shooting. Therefore, the related art has relatively high requirements for both the shooting equipment and the professionalism of the shooter.

[0077] In view of this, the present disclosure no longer relies on the cancellation effect of the focal length change and the object distance change on the imaging change, but starts from the presentation effect of the Hitchcock effect, and proposes an image processing method combining pre-shooting and post-processing to avoid the high requirements of the Hitchcock zoom for shooting equipment and shooters.

[0078] Specifically, the presentation effect of the Hitchcock effect is that the size of the subject remains unchanged, but the background space produces a visual effect of compression or magnification. On this basis, the present disclosure still obtains a video with a compression phenomenon and / or a magnification phenomenon in the background space through pre-shooting, and makes the sizes of the subjects in multiple image frames of the video consistent by scaling multiple image frames respectively.

[0079] It should be understood that since the technical solution of the present disclosure no longer relies on the cancellation effect of the object distance change and the image distance change on the imaging change, but makes the sizes of the subjects in multiple image frames consistent by scaling the image frames in the later stage, therefore, through the technical solution of the present disclosure, the shooter only needs to keep the object to be shot at a fixed position in the frame during pre-shooting (i.e., the subject is located at the same position in each image frame), which greatly reduces the operation requirements for pre-shooting. It can be seen that through the technical solution of the present disclosure, a video with the Hitchcock effect can be obtained without the need to use additional shooting equipment as in the related art, and there is no high technical threshold for the shooter.

[0080] Step 204: Determine the scaling ratios corresponding to the multiple shooting state parameters respectively according to the corresponding relationship between the preset shooting state parameters and the scaling ratios; wherein, after scaling the image frame obtained by shooting under the corresponding shooting state parameter by the scaling ratio in any corresponding relationship, the size of the subject in the scaled target image frame is the same as the size of the subject in the image reference frame.

[0081] It should be understood that the technical solution of the present disclosure is equivalent to obtaining a changing background space through pre-shooting and then offsetting the size change of the subject during pre-shooting through post-scaling. Therefore, in order to make the background space change during pre-shooting in the present disclosure, the photographer can continuously adjust at least one shooting state parameter during pre-shooting. The change direction and change rate of the shooting state parameter affect the change direction and change rate of the size of the subject in the picture. For example, when the electronic device continuously approaches the subject, the size of the subject in the picture will continuously increase, and the increasing rate is positively correlated with the rate at which the electronic device approaches the subject. In other words, there is a certain correspondence between the shooting state parameter and the size of the subject in the picture. Correspondingly, there must also be a certain correspondence between the scaling ratio set later to offset the size change of the subject and the shooting state parameter. Therefore, in the present disclosure, at least one set of correspondence between the shooting state parameter and the scaling ratio can be preset for determining the scaling ratio to be used for each image frame during post-processing.

[0082] In the present disclosure, the photographer can obtain multiple image frames by adjusting different shooting state parameters during pre-shooting.

[0083] In one embodiment, during pre-shooting, the photographer can shoot a video by pushing and pulling (moving back and forth) the electronic device to ensure that the subject is always located at a fixed position in the picture. During this process, it is equivalent to changing the device motion parameter of the electronic device so that the background space in the captured video image continuously changes. Therefore, in this embodiment, the correspondence between the device motion parameter and the scaling ratio should be preset. Then, after obtaining multiple image frames in the captured video and the device motion parameters used when shooting multiple image frames in this embodiment, the scaling ratio corresponding to each device motion parameter can be determined according to the preset correspondence, and then the corresponding image frame can be scaled by the determined scaling ratio so that the size of the subject in the scaled target image frame is the same as the size of the subject in the image reference frame.

[0084] It should be understood that for the photographer, there is no need to master professional camera knowledge to know the impact of pushing, pulling or moving the electronic device forward and backward on the video screen. For example, the photographer can determine that when the electronic device is close to the subject, the background space in the video screen will continue to expand, and when the electronic device is far away from the subject, the background space in the video screen will continue to compress, based on the law of enlargement and reduction of the object observed by the eyes when the photographer approaches or moves away from the object. It can be seen that through the technical solution of this embodiment, the user only needs to master the relationship between the direction of movement and the imaging change (obviously common sense in life) to accurately determine the shooting method of the required video special effects (mainly referring to the moving direction during shooting), avoiding the situation where reshooting is required due to incorrect shooting methods. For example, when it is necessary to continuously enlarge the background space in the video screen, you can shoot by continuously approaching the subject; when it is necessary to continuously compress the background space in the video screen, you can shoot by continuously moving away from the subject.

[0085] In this embodiment, the device motion parameters respectively used when capturing multiple image frames may be obtained in a variety of ways.

[0086] In one case, the motion information of the electronic device when shooting multiple image frames can be detected by a motion sensor installed in the electronic device, such as a gyroscope, a magnetic sensor, etc., and the acquired motion information can be analyzed to determine the device motion parameters used when shooting the multiple image frames. In this case, there are certain hardware requirements for the electronic device, but the device motion information used when shooting multiple image frames can be detected simply and directly, which is highly efficient.

[0087] In another case, the electronic device can obtain the proportion of the subject in each image frame, so as to compare the proportion of the subject in any image frame with the proportion of the subject in the adjacent frames of any image frame when determining the device motion parameters used to shoot any image frame, and determine the device motion parameters used when shooting any image frame based on the change information of the proportion of the subject. For example, after shooting multiple image frames, the proportion of the subject in each image frame can be compared with the proportion of the subject in the previous image frame (the previous image frame in the shooting time sequence), so as to determine the device motion parameters of each image frame based on the change information of the proportion.

[0088] Of course, the above examples are merely illustrative, and how to determine the device motion parameters corresponding to each image frame can be determined by those skilled in the art according to actual needs, and the present disclosure does not impose any limitation on this.

[0089] In another embodiment, during the initial shooting process, the photographer can keep the electronic device stationary and obtain a continuously changing video frame of the background space by adjusting the focal length parameter of the camera. During this process, since the electronic device remains stationary, it can ensure that the subject to be photographed is always located at a fixed position in the captured video. In this embodiment, it is equivalent to changing the focal length to make the background space in the video frame continuously change. Therefore, in this embodiment, the corresponding relationship between the camera focal length parameter and the zoom ratio should be preset. Then, after obtaining multiple image frames in the captured video and the multiple camera focal length parameters respectively used when shooting the multiple image frames, the zoom ratios corresponding to the multiple camera focal length parameters can be determined according to the preset corresponding relationship, and then the corresponding image frames can be zoomed by the determined zoom ratios so that the size of the subject to be photographed in the scaled target image frame is consistent with the size of the subject to be photographed in the image reference frame.

[0090] In this embodiment, although the photographer needs to master certain photography knowledge, that is, the relationship between focal length change and imaging change, in order to accurately determine the video special effects required for shooting and the shooting method adopted (mainly referring to the direction of focal length adjustment during shooting: increasing or decreasing). However, this photography knowledge belongs to entry-level knowledge and is relatively easy to master. And in actual shooting, the user does not need to move forward and backward or push and pull, but only needs to keep the electronic device stationary to shoot a video frame with Hitchcock special effects, which further simplifies the operation of obtaining Hitchcock special effects.

[0091] It is not difficult to understand that the above two embodiments respectively perform initial shooting by changing the focal length and the object distance, and offset the size change of the subject to be photographed caused by the change of the focal length or the object distance by zooming the image frame in the later stage, and both can achieve the Hitchcock special effect that the size of the subject to be photographed remains unchanged and the background space is compressed or enlarged.

[0092] In the present disclosure, an image reference frame needs to be determined in advance to use this image reference frame as the standard for enlarging and reducing each image frame. Specifically, after each image frame is scaled by the corresponding zoom ratio, the size of the subject to be photographed in this image frame is consistent with the size in the image reference frame.

[0093] In actual operation, different image frames can be determined as this image reference frame according to actual needs.

[0094] In one embodiment, the multiple acquired image frames have a unified image reference frame. For example, one of the multiple image frames obtained by shooting can be selected as the image reference frame. For another example, the first image frame or the last image frame among the multiple acquired image frames can be used as the image reference frame, which is equivalent to using the first frame image or the last frame image in the shot video as the image reference frame.

[0095] In another embodiment, each of the multiple acquired image frames has its corresponding image reference frame. For example, each image frame can use its adjacent frame as its image reference frame. In one case, each image frame among the multiple image frames can use the previous image frame in the preset arrangement order as its image reference frame; or each image frame among the multiple image frames can use the next image frame in the preset arrangement order as its image reference frame. In this case, since each image frame uses the adjacent frame in the same direction of the preset arrangement order as its image reference frame, that is, after each image frame is scaled, the size of the subject in the image is consistent with that of the adjacent frame in the same direction, it is certain that the size of the subject in each image frame can be kept consistent. In actual operation, the above preset arrangement order mostly adopts the shooting time order of the multiple image frames, and the specific order to be used as the preset arrangement order can be determined by those skilled in the art according to the actual situation, and the present disclosure does not limit this. It should be emphasized that for the first image frame in the preset arrangement order, there is no previous image frame. Therefore, when using the previous image frame as the image reference frame, the first image frame can default itself as its own image reference frame; the same is true for the last image frame in the preset arrangement order, which will not be elaborated here.

[0096] Of course, the above examples are only illustrative, and how to determine the image reference frame can be determined by those skilled in the art according to the actual situation, and the present disclosure does not limit this.

[0097] It should be noted that the purpose of the corresponding relationship between the preset shooting state parameters and the scaling ratio is: after the image frame shot under the corresponding shooting state parameters is scaled by the corresponding scaling ratio, the size of the subject in the image frame is consistent with the size in the image reference frame. It is not difficult to understand from this that the corresponding relationship between the shooting state parameters and the scaling ratio is set with reference to the selected image reference frame. It can be seen that the setting of this corresponding relationship is necessarily related to the selection method of the image reference frame.

[0098] For example, in an embodiment where the shooting state parameter is the device motion parameter, the above device motion parameter can be the motion speed, the motion distance, or other motion parameters of the device. Which specific motion parameter of the device it is usually depends on the selected reference frame. Usually, when multiple acquired image frames have a unified image reference frame, the device motion parameter is usually the "motion distance between the device position when each image frame is shot and the device position when the image reference frame is shot". Correspondingly, the pre-set relationship is the corresponding relationship between the "device motion distance and the scaling ratio"; when multiple image frames have their respective corresponding image reference frames, for example, when the reference frame corresponding to each image frame is the previous image frame of the corresponding image frame, the device motion parameter is usually the "device motion speed when this image frame is shot". Correspondingly, the pre-set relationship is the corresponding relationship between the "device motion speed and the scaling ratio". Of course, the shooting state parameters respectively adopted in the above various image frame selection methods are only illustrative. Specifically, the shooting state parameters respectively adopted in various image frame selection methods, as well as the set corresponding relationships, can all be set by those skilled in the art according to the actual situation, and the present disclosure does not limit this.

[0099] For another example, in an embodiment where the shooting state parameter is the focal length parameter of the camera, the above focal length parameter of the camera can be the focal length change speed of the camera, the focal length value of the camera, or other parameters related to the camera focal length of the device. Which specific parameter the focal length parameter is usually depends on the selected reference frame. Usually, when multiple acquired image frames have a unified image reference frame, the camera focal length parameter is usually the "focal length difference between the camera focal length value when each image frame is shot and the camera focal length value when the image reference frame is shot". Correspondingly, the pre-set relationship is the corresponding relationship between the "camera focal length difference and the scaling ratio"; when multiple image frames have their respective corresponding image reference frames, for example, when the reference frame corresponding to each image frame is the previous image frame of the corresponding image frame, the camera focal length parameter is usually the "focal length change speed when this image frame is shot". Correspondingly, the pre-set relationship is the corresponding relationship between the "focal length change speed and the scaling ratio". Of course, the shooting state parameters respectively adopted in the above various image frame selection methods are only illustrative. Specifically, the shooting state parameters respectively adopted in various image frame selection methods, as well as the set corresponding relationships, can all be set by those skilled in the art according to the actual situation, and the present disclosure does not limit this.

[0100] Step 206: Scale the corresponding image frames according to the determined scaling ratio to obtain multiple target image frames, and encode the obtained multiple target image frames into a video file.

[0101] After obtaining multiple target image frames, the present disclosure can encode the multiple target image frames into a video file in various ways.

[0102] In one embodiment, the present disclosure can preferentially determine the time sequence of capturing multiple image frames, and encode the obtained multiple target image frames into a video file according to this time sequence.

[0103] For example, assume that 4 image frames A, B, C, and D are captured, and the time sequence of capturing these 4 image frames is "A→B→C→D"; the 4 target image frames obtained after scaling the 4 image frames are a, b, c, and d. Then, the 4 target image frames can be encoded into a video file in the order of "a→b→c→d".

[0104] In another embodiment, the present disclosure can preferentially determine the scaling ratios respectively adopted by the obtained multiple target image frames, and sort these multiple target image frames according to a predefined scaling ratio sorting rule. On this basis, the multiple target image frames can be encoded into a video file according to the sorting result.

[0105] For example, assume that the scaling ratio sorting rule is: sort the target image frames in the order of increasing scaling ratio. Assume that the scaling ratios corresponding to the 4 obtained target image frames A, B, C, and D are: 0.9, 0.8, 0.7, and 0.6. Then, the order of the multiple target image frames after sorting is: target image frame D, C, B, A. On this basis, the multiple target image frames can be encoded into a video file in the order of "D→C→B→A".

[0106] In actual operation, the above scaling ratio sorting rule can be set according to actual needs. For example, if it is necessary to achieve the visual effect of continuously enlarging the background space, then, as in the above example, the multiple target image frames can be sorted in the order of increasing scaling ratio. The specific scaling ratio sorting rule to be adopted can be determined by those skilled in the art according to actual needs. For example, it can also be sorted in the way of increasing first and then decreasing to produce the visual effect of the background space enlarging first and then compressing, creating a tense and depressing atmosphere for the viewer. The present disclosure does not limit this.

[0107] It should be noted that since the present disclosure obtains target image frames by scaling image frames, the sizes of the multiple target image frames obtained by scaling are actually different. Therefore, before encoding the target image frames into a video file, the multiple target image frames can also be cropped to make the sizes of the cropped multiple target image frames consistent.

[0108] In addition, different from the related art which utilizes the cancellation effect of object distance change and focal length change on imaging change (the subject to be photographed must be at the center of the frame to utilize this cancellation effect to keep the size of the subject to be photographed unchanged) to obtain the Hitchcock effect, when implementing the Hitchcock effect using the technical solution of the present disclosure, the subject to be photographed does not have to be located at the center of the frame. It only needs to be at the same position in the multiple acquired image frames. This same position can be the center position of the corresponding image frame as in the related art, and then the same Hitchcock effect as in the related art can be obtained (the subject to be photographed is at the center of the frame and its size remains unchanged; the background space shows compression or magnification). This same position can also be different from that in the related art and can be any position deviating from the center position in the corresponding image frame. For example, it can be a position to the left in the corresponding image frame. Then, a Hitchcock effect different from that in the related art can be obtained (the size of the subject to be photographed remains unchanged, but it is not at the center of the frame; the background space shows compression or magnification).

[0109] It should be emphasized that the technical solution of the present disclosure can be applied to any type of electronic device as long as the electronic device is equipped with a camera and has an image acquisition function. For example, the electronic device can be a mobile terminal such as a smart phone or a tablet computer, or a fixed terminal such as a smart TV or a PC (Personal Computer) equipped with a camera. Which specific type of electronic device is used as the execution subject of the technical solution of the present disclosure can be determined by those skilled in the art according to actual needs, and the present disclosure does not limit this.

[0110] As can be seen from the above technical solution, after the present disclosure obtains multiple image frames and the shooting state parameters respectively used when shooting multiple image frames, it can respectively determine the scaling ratios corresponding to the multiple shooting state parameters according to the preset correspondence between the shooting state parameters and the scaling ratios, and then scale the corresponding image frames by the determined scaling ratios to obtain multiple target image frames. Among them, after scaling the image frame shot under the corresponding shooting state parameter by the scaling ratio in any correspondence, the size of the subject to be photographed in the scaled target image frame is the same as its size in the image reference frame.

[0111] It should be understood that after the multiple acquired image frames are scaled by the corresponding scaling ratios, the size of the subject to be photographed in the obtained multiple target image frames is the same as its size in the image reference frame, which is equivalent to the size of the subject to be photographed being the same in the multiple target image frames. Therefore, after encoding the multiple target image frames into a video file, the size of the subject to be photographed in the video picture remains unchanged.

[0112] In actual operation, the photographer can change the shooting state parameters to make the background space change, and then offset the size change of the subject during the previous shooting process by scaling the image frames as described above, so that the subject in the finally processed video remains unchanged and the background space changes, achieving the Hitchcock effect.

[0113] Compared with the method of achieving the Hitchcock effect by the shooting technique of Hitchcock zoom in the related art, the technical solution of the present disclosure no longer relies on the offset effect of the focal length change and the object distance change on the imaging change. Instead, starting from the presentation effect of the Hitchcock effect, multiple image frames with continuously changing background space are obtained through the previous shooting, and the size change of the subject during the previous shooting process is offset by the later scaling method. This makes the technical solution of the present disclosure have extremely low requirements for the previous shooting skills and shooting equipment, avoiding the problems of too high cost caused by the need for additional shooting equipment in the related art and the problem of high technical threshold for the photographer.

[0114] In addition, in the related art, since the change amplitude of the imaging is related to the distance from the imaging position to the center of the screen (that is, the above-mentioned offset phenomenon of the focal length change and the object distance change on the imaging change only occurs in the central area of the screen), therefore, during the Hitchcock zoom process, the subject must be kept in the center of the screen. In the video screen with the Hitchcock effect obtained in this way, the subject is also necessarily in the center of the screen. However, through the technical solution of the present disclosure, since it no longer relies on the offset effect of the image distance change and the object distance change on the imaging change, therefore, during the previous shooting process, it is no longer required that the subject be in the center of the screen. In other words, through the technical solution of the present disclosure, in addition to obtaining the traditional Hitchcock effect with the size of the subject in the center of the screen remaining unchanged, it is also possible to obtain the Hitchcock effect where the subject is not in the center of the screen. For example, the subject can be in the left half or the right half of the screen, etc.

[0115] Next, taking a smart phone (hereinafter simply referred to as a mobile phone) as an example, the technical solution of the present disclosure will be introduced by means of specific embodiments.

[0116] Figure 3 It is a flowchart of another image processing method shown in an exemplary embodiment of the present disclosure. As Figure 3 shown, the method includes the following steps:

[0117] Step 301, obtain multiple image frames captured by the user by moving the mobile phone.

[0118] In this embodiment, the user can push or walk forward and backward, so that the subject to be photographed always remains in a fixed position in the picture during the process of taking pictures with the mobile phone. This shooting method is equivalent to changing the object distance, so that the background in the captured picture keeps changing.

[0119] In actual operation, after the user opens the image shooting application installed on the mobile phone, the video effects that can be achieved can be shown to the user. When the user selects the Hitchcock effect, the prompt message "Please keep the subject to be photographed in a fixed position in the picture during the video shooting process" can be further displayed in the shooting interface to remind the user to keep the position of the subject to be photographed unchanged in the picture.

[0120] Step 302, obtain the motion information detected by the gyroscope when shooting multiple image frames.

[0121] In this embodiment, the gyroscope originally assembled in the mobile phone can be used to detect the motion information when shooting multiple image frames, so as to determine the device motion parameters respectively adopted when shooting multiple images according to the obtained motion information.

[0122] Step 303, determine the motion speed of the mobile phone when shooting each image frame according to the obtained motion information.

[0123] In this embodiment, the corresponding relationship between the motion speed and the scaling ratio can be preset, so as to scale the image frame after obtaining the image to ensure that the size of the subject to be photographed in each image frame remains the same.

[0124] In actual operation, technicians can obtain the relationship between the motion speed and the scaling multiple of the subject to be photographed through experiments, and determine the corresponding relationship between the motion speed and the scaling ratio according to this relationship.

[0125] For example, assume that the experiment measures that when shooting a certain image frame, the forward motion speed of the mobile phone is 3 cm / s, and the size of the subject to be photographed in this image frame is enlarged by 4 / 3 times compared with the subject to be photographed in the previous image frame. Then, in order to keep the size of the subject to be photographed the same as that in the previous image frame after scaling this image frame, the scaling ratio can be set to 3 / 4.

[0126] In this example, it is equivalent to taking the previous image frame of each captured image frame as the respective image reference frame, and taking the motion speed of the mobile phone as the device motion parameter. Since the size of the subject to be photographed in each frame of the image remains the same as that of the subject to be photographed in the previous frame after scaling, it can be ensured that the size of the subject to be photographed remains the same in multiple target image frames after scaling.

[0127] Step 304: Determine the scaling ratio corresponding to each image frame according to the preset correspondence between the motion speed and the scaling ratio.

[0128] Continuing with the above example, assume that the relationship between the motion speed and the scaling ratio preset by the technician is as shown in Table 1 below:

[0129] Moving speed (cm / s) Scaling ratio …… …… -3 4 / 3 -2 5 / 4 -1 6 / 5 0 1 +1 5 / 6 +2 4 / 5 +3 3 / 4 …… ……

[0130] Table 1

[0131] In the above table, the motion speed is positive, indicating that the motion direction is towards the "subject being photographed"; the motion speed is negative, indicating that the motion direction is away from the "subject being photographed".

[0132] Step 305: Scale the corresponding image frames based on the determined scaling ratios.

[0133] Continuing with the above example, assume that 4 image frames A, B, C, and D are obtained by shooting, and the motion speed of the device when shooting these 4 image frames is as shown in Table 2 below:

[0134]

[0135]

[0136] Table 2

[0137] On this basis, Table 2 can be compared with Table 1, and then it can be determined that the scaling ratio corresponding to image frame A is 1; the scaling ratio corresponding to image frame B is 5 / 6; the scaling ratio corresponding to image frame C is 4 / 5; the scaling ratio corresponding to image frame D is 3 / 4. In other words, based on Table 2, Table 3 below can be further obtained

[0138] Image frame Moving speed (cm / s) Scaling ratio A 0 1 B +1 5 / 6 C +2 4 / 5 D +3 3 / 4

[0139] Table 3

[0140] After obtaining the scaling ratios corresponding to multiple image frames respectively, the multiple image frames can be scaled respectively. Specifically, since the scaling ratio of image frame A is 1, image frame A is not scaled and is directly used as the target image frame A'; image frame B is scaled to 5 / 6 of the original to obtain the target image frame B'; image frame C is scaled to 4 / 5 of the original to obtain the target image frame C'; image frame D is scaled to 3 / 4 of the original to obtain the target image frame D'.

[0141] Step 306: Encode the multiple target image frames obtained by scaling into a video file in the shooting order.

[0142] In this embodiment, multiple target image frames can be directly encoded into a video file in the order of the captured image frames.

[0143] Continuing with the above example, since the time sequence of capturing multiple image frames is "A → B → C → D", the target image frames A', B', C', and D' can be encoded into a video file in the order of "A' → B' → C' → D'".

[0144] It should be noted that the above example is only illustrative. It is only an example of taking the shooting state parameter as the moving speed and the image reference frame of each image frame being the previous image frame of its own to introduce the technical solution of the present disclosure. In actual applications, any of the above-introduced methods can be adopted, and no further examples will be elaborated here.

[0145] As can be seen from the above technical solution, through the technical solution of this embodiment, the user only needs to push or move forward and backward to ensure that the subject to be photographed in the multiple captured image frames is located at a fixed position to achieve the Hitchcock effect. The user neither needs to strictly control the moving speed nor adjust the camera focal length while moving, greatly simplifying the operation of achieving the Hitchcock effect and reducing the technical threshold for achieving the Hitchcock effect.

[0146] It should be understood that for the user, there is no need to master professional photography knowledge to know the impact of pushing or moving the mobile phone forward and backward on the video picture. For example, when the photographer approaches or moves away from an object, the photographer can determine that when the mobile phone approaches the subject to be photographed, the background space in the video picture will continuously expand, and when the mobile phone moves away from the subject to be photographed, the background space in the video picture will continuously compress according to the law of the object magnifying and shrinking observed by the eyes. It can be seen that through the technical solution of this embodiment, the user only needs to master the relationship between the moving direction and the imaging change (which is obviously common sense in life) to accurately determine the shooting method of the required video special effect, avoiding the situation of having to reshoot due to incorrect shooting methods. For example, when it is necessary to continuously expand the background space in the video picture, the shooting can be carried out by continuously approaching the subject to be photographed; when it is necessary to continuously compress the background space in the video picture, the shooting can be carried out by continuously moving away from the subject to be photographed. Figure 4 It is a flowchart of another image processing method shown in an exemplary embodiment of the present disclosure. As Figure 4 shown, the method includes the following steps:

[0147] Step 401, obtain multiple image frames captured by the user by adjusting the camera focal length.

[0148] In this embodiment, the user can keep the mobile phone stationary and adjust the focal length of the camera so that the subject to be photographed always remains in a fixed position in the frame during the image shooting process. This shooting method is equivalent to changing the focal length to make the background in the captured image constantly change.

[0149] In actual operation, after the user opens the image shooting application installed on the mobile phone, the video effects that can be achieved can be shown to the user. When the user selects the Hitchcock effect, operation controls for adjusting the focal length of the camera can be further shown in the shooting interface, and at the same time, a prompt message "Please try to keep the mobile phone stationary and shoot by adjusting the focal length of the camera" can be further displayed.

[0150] Step 402, obtain the focal length change speed of the camera when shooting multiple image frames.

[0151] In this embodiment, the mobile phone can monitor the focal length change speed of the camera in real time to obtain the focal length change speeds corresponding to the camera when shooting multiple image frames.

[0152] Step 403, determine the scaling ratio corresponding to each image frame according to the preset correspondence between the focal length change speed and the scaling ratio.

[0153] In this embodiment, the correspondence between the focal length change speed and the scaling ratio can be preset to be used for scaling the image frames after obtaining the image to ensure that the sizes of the subjects to be photographed in each image frame are kept consistent.

[0154] In actual operation, technicians can obtain the relationship between the focal length change speed and the scaling multiple of the subject to be photographed through experiments, and determine the correspondence between the focal length change speed and the scaling ratio according to this relationship.

[0155] For example, assume that it is experimentally measured that when shooting a certain image frame, the speed at which the focal length of the camera increases is 3 mm / s, and the size of the subject to be photographed in this image frame is 1.5 times larger than that in the previous image frame. Then, in order to keep the size of the subject to be photographed consistent with that in the previous image frame after scaling this image frame, the scaling ratio can be set to 2 / 3.

[0156] This embodiment is similar to the previous embodiment. In both cases, the previous image frame of each captured image frame is used as the respective image reference frame, and the focal length change speed of the camera is used as the focal length parameter of the camera. Since the size of the subject to be photographed in each frame of the image is consistent with that in the previous frame of the image after scaling, it is possible to ensure that the size of the subject to be photographed remains consistent in multiple target image frames after scaling.

[0157] Suppose the relationship between the focal length change speed and the zoom ratio preset by the technician is shown in Table 4 below:

[0158]

[0159]

[0160] Table 4

[0161] In the above table, the focal length change speed is positive, indicating that the focal length increases; the focal length change speed is negative, indicating that the focal length decreases.

[0162] Step 404, scale the corresponding image frame based on the determined zoom ratio.

[0163] Continuing with the above example, suppose 4 image frames E, F, G, and H are captured, and the focal length speed of the device when capturing these 4 image frames is shown in Table 5 below:

[0164] Image frame Focal length change speed (mm / s) E 0 F +1 G +2 H +3

[0165] Table 5

[0166] Based on this, Table 4 can be compared with Table 5 to further determine that the zoom ratio corresponding to image frame E is 1; the zoom ratio corresponding to image frame F is 4 / 5; the zoom ratio corresponding to image frame G is 3 / 4; the zoom ratio corresponding to image frame H is 2 / 3. In other words, based on Table 5, the following Table 6 can be further obtained:

[0167] Image frame Focal length change speed (mm / s) Scaling ratio E 0 1 F +1 4 / 5 G +2 3 / 4 H +3 2 / 3

[0168] Table 6

[0169] After obtaining the zoom ratios corresponding to multiple image frames respectively, the multiple image frames can be scaled respectively. Specifically, since the zoom ratio of image frame E is 1, image frame A is not scaled, and image frame E is directly used as the target image frame E'; image frame F is scaled to 4 / 5 of the original to obtain the target image frame F'; image frame G is scaled to 3 / 4 of the original to obtain the target image frame G'; image frame H is scaled to 2 / 3 of the original to obtain the target image frame H'.

[0170] Step 405, encode the multiple scaled target image frames into a video file according to the shooting order.

[0171] In this embodiment, the multiple target image frames can be directly encoded into a video file according to the order of the captured image frames.

[0172] Continuing with the above example, since the time sequence of shooting multiple image frames is "E→F→G→H", the target image frames E', F', G', and H' can be encoded into a video file in the order of "E’→F’→G’→H’".

[0173] It should be noted that the above example is only illustrative. It is only an example where the shooting state parameter is the focal length change speed and the image reference frame of each image frame is the previous image frame of its own image frame to introduce the technical solution of the present disclosure. In actual applications, any of the above-introduced methods can be adopted, and no further examples will be elaborated here.

[0174] As can be seen from the above technical solution, through the technical solution of the present disclosure, the user only needs to keep the mobile phone still and can achieve the Hitchcock effect by changing the focal length of the camera. Compared with the previous embodiment, although the technical solution of this embodiment requires the user to master certain photography knowledge, that is, the relationship between focal length change and imaging change, the shooter can accurately determine the required video special effects for shooting and the shooting method adopted (mainly referring to the direction of focal length adjustment during shooting: increasing or decreasing). However, this photography knowledge belongs to entry-level knowledge and is relatively easy to master. And in actual shooting, the user does not need to move forward and backward or push and pull, but only needs to keep the electronic device still to shoot a video picture with the Hitchcock effect, further simplifying the operation of obtaining the Hitchcock effect and reducing the energy consumed by the user to achieve the Hitchcock effect in the video picture.

[0175] Figure 5 It is a flowchart of a method for implementing the Hitchcock effect shown in an exemplary embodiment of the present disclosure. As Figure 5 shown, the method may include the following steps:

[0176] Step 501, detecting a trigger operation for the image shooting icon.

[0177] In this embodiment, an application program with an image shooting function can be pre-installed in the mobile phone. Then, when the user needs to shoot an image, the user only needs to trigger the corresponding image shooting icon (the icon corresponding to the above application program with an image shooting function) to shoot an image.

[0178] Step 502, starting the image shooting application program and displaying the corresponding program interface.

[0179] In this embodiment, after detecting the trigger operation of the user for the image shooting icon, the corresponding program interface can be displayed. Among them, two shooting options, "Video" and "Photo", can be displayed in this interface for the user to select the image form of this shooting.

[0180] After detecting that the user selects the "Video" option, this embodiment can further display various video special effect options for the user to select the video special effects to be added to the video frame obtained in this shooting. For example, the following can be displayed: fade in, fade out, from bright to dark, from dark to bright, Hitchcock effect, and other optional video special effect options.

[0181] Step 503: Detect a trigger operation for the Hitchcock effect option.

[0182] In this embodiment, the user can trigger the Hitchcock effect option to indicate that the Hitchcock effect is to be added to the video frame shot this time.

[0183] After the mobile phone detects the trigger operation of the user for the Hitchcock effect option, it can adjust the video shooting mode to the shooting mode for implementing the Hitchcock effect. On this basis, once the mobile phone detects the trigger operation for the start shooting option, it can perform corresponding post-processing on the captured image frames to add the Hitchcock effect to the captured video frame.

[0184] Step 504: Display a prompt message of "Please keep the subject fixed in the frame and move the phone forward and backward to shoot the video".

[0185] In this embodiment, after the mobile phone detects the trigger operation of the user for the Hitchcock effect option, it can further display the precautions for shooting.

[0186] In this embodiment, if the Hitchcock effect is implemented in the Figure 3 shown manner, then a prompt message of "Please keep the subject fixed in the frame and move the phone forward and backward to shoot the video" can be displayed to the user.

[0187] Of course, this embodiment is only introduced by taking "implementing the Hitchcock effect in the Figure 3 shown manner" as an example. It should be understood that the actually displayed prompt message should correspond to the implementation method adopted. For example, when implementing the Hitchcock effect in the Figure 4 shown manner, a prompt message of "Please keep the phone stationary and adjust the focal length to shoot the video" can be displayed to the user.

[0188] Furthermore, the above two prompt messages can also be displayed simultaneously. For example, a prompt message of "Please keep the subject fixed in the frame and move the phone forward and backward to shoot the video; or, keep the phone stationary and adjust the focal length to shoot the video" can be displayed. On this basis, after the mobile phone detects that the user changes the motion state of the phone or changes the focal length of the phone camera after triggering the start shooting option, it can then select the above Figure 3 or Figure 4Implement the Hitchcock effect in the manner shown.

[0189] Step 505, detect a trigger operation for the start shooting option.

[0190] In this embodiment, since the Figure 3 manner shown is adopted, therefore, after detecting the trigger operation of the user for the start shooting option, the movement speed of the mobile phone can be monitored in real time when each image frame in the shooting video is captured, so as to determine the scaling ratio corresponding to each image frame.

[0191] Step 506, call the camera to capture an image, and detect the movement speed of the mobile phone in real time through the gyroscope when each frame of the image is captured.

[0192] Step 507, determine the scaling ratio corresponding to each frame of the image according to the corresponding relationship between the movement speed and the scaling ratio.

[0193] Step 508, scale the corresponding image frame based on the determined scaling ratio.

[0194] Step 509, encode the multiple scaled image frames into a video file.

[0195] In the above steps, specifically how to determine the scaling ratio and how to encode the multiple target image frames obtained through scaling into a video file can both refer to the introduction of the Figure 3 embodiment shown, and will not be elaborated here.

[0196] It should be noted that in actual operation, the image can be scaled in real time during the video shooting process; or the unified scaling process can be performed after the entire video shooting is completed. How to operate specifically can be determined by those skilled in the art according to actual needs, and the present disclosure does not limit this.

[0197] For example, in the case of "scaling the obtained image frames in real time during the video shooting process", every time the mobile phone captures an image frame, the movement speed of the mobile phone detected by the gyroscope at this time can be obtained, and the scaling ratio corresponding to this image frame can be determined according to the preset corresponding relationship between the movement speed and the scaling ratio, so as to scale this image frame.

[0198] In this case, the scaled image frame can be directly used as a preview screen for display. It can be imagined that during the video shooting process, the user can see that the image changes in the preview screen already have the Hitchcock effect. In other words, the video with the Hitchcock effect can be previewed in real time during the shooting process. In addition, in this case, when the video shooting is completed, the scaling process for multiple image frames is also completed, which improves the efficiency of post-processing and reduces the time required to obtain the Hitchcock effect.

[0199] In the case of "performing unified scaling processing after the entire video is shot", the mobile phone can perform unified scaling processing on the obtained multiple image frames after the user triggers the end shooting option (in actual operation, the end shooting option can be the start shooting option triggered again). In this case, the mobile phone does not need to perform complex post-processing during shooting, and has relatively low requirements for the performance of the processor. And in this case, more processing resources can be called for scaling processing, increasing the accuracy of post-processing.

[0200] In addition, although compared with the previous case, since no scaling processing is performed during shooting, it takes longer to obtain a video with a Hitchcock effect. However, for the same reason, the preview screen during shooting is the original screen obtained by actual shooting, which is convenient for the user to compare the original screen with the video screen with the Hitchcock effect obtained by post-processing.

[0201] As can be seen from the above technical solutions, the present disclosure can add a Hitchcock effect option to the image shooting application, so that when the user needs to shoot a video screen with a Hitchcock effect, a Hitchcock effect can be added to the shot video screen through a simple selection operation.

[0202] Among them, after the user selects the Hitchcock effect option, the mobile phone can display corresponding shooting precautions in the interface to prompt the user to shoot in a standardized manner, avoiding the situation where the Hitchcock effect cannot be added to the video screen due to non-standard shooting methods.

[0203] In addition, in this embodiment, scaling operations can be performed on the obtained image frames in real time during video shooting, so as to preview the imaging effect of the video screen with the Hitchcock effect in real time during shooting and improve the efficiency of adding the Hitchcock effect to the video screen; or after the video shooting is completed, scaling operations can be performed on the obtained multiple image frames in a unified manner, so as to reduce the requirements for processing while improving the accuracy of post-processing.

[0204] Figure 6 is a block diagram of an image processing device shown in an exemplary embodiment of the present disclosure. Referring to Figure 6 , the device includes an acquisition unit 601, a determination unit 602, and a scaling unit 603.

[0205] The acquisition unit 601 acquires a plurality of image frames obtained by shooting a subject and a plurality of shooting state parameters respectively used when shooting the plurality of image frames, and the subject is located at the same position in each image frame;

[0206] Determination unit 602 determines the scaling ratios corresponding to the multiple shooting state parameters respectively according to the corresponding relationship between the preset shooting state parameters and the scaling ratios; wherein, after scaling the image frames obtained by shooting under the corresponding shooting state parameters by the scaling ratio in any corresponding relationship, the size of the subject in the scaled target image frame is the same as the size of the subject in the image reference frame;

[0207] Scaling unit 603 scales the corresponding image frames according to the determined scaling ratios to obtain multiple target image frames, and encodes the obtained multiple target image frames into a video file.

[0208] Optionally,

[0209] The acquisition unit 601 is further configured to: acquire multiple image frames and the device motion parameters respectively used when shooting the multiple image frames;

[0210] The determination unit 602 is further configured to: determine the scaling ratios corresponding to the respective device motion parameters respectively according to the corresponding relationship between the preset device motion parameters and the scaling ratios.

[0211] Optionally,

[0212] The acquisition unit 601 is further configured to: acquire the motion information respectively detected by the motion sensor when shooting the multiple image frames; perform data analysis on the acquired motion information to determine the device motion parameters respectively used when shooting the multiple image frames.

[0213] Optionally,

[0214] The acquisition unit 601 is further configured to: acquire the first ratio of the subject in any image frame and the second ratio of the subject in the adjacent image frame of the any image frame; determine the device motion parameter used when shooting the any image frame according to the change information of the first ratio and the second ratio.

[0215] Optionally,

[0216] The acquisition unit 601 is further configured to: acquire multiple image frames and the focal length parameters respectively used when shooting the multiple image frames;

[0217] The determination unit 602 is further configured to: determine the scaling ratios corresponding to the respective focal length parameters respectively according to the corresponding relationship between the preset focal length parameters and the scaling ratios.

[0218] Optionally,

[0219] The image reference frame is any one of the multiple image frames; alternatively, the image reference frame is the earliest or the latest captured image frame among the multiple image frames; alternatively, each of the multiple image frames uses the previous image frame in a preset arrangement order as its own image reference frame, or each of the multiple image frames uses the next image frame in a preset arrangement order as its own image reference frame.

[0220] Optionally,

[0221] The scaling unit 603 is further configured to: determine the time sequence of capturing the multiple image frames; encode the obtained multiple target image frames into a video file according to the time sequence.

[0222] Optionally,

[0223] The scaling unit 603 is further configured to: determine the scaling ratios respectively adopted by the multiple target image frames, and sort the multiple target image frames according to a predefined scaling ratio sorting rule; encode the multiple target image frames into a video file according to the sorting result.

[0224] Optionally, the same position of each image frame is the central position of each image frame.

[0225] For the apparatus embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The apparatus embodiment described above is only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present disclosure. A person of ordinary skill in the art can understand and implement it without creative work.

[0226] Correspondingly, the present disclosure also provides an image processing apparatus, including: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the image processing method as described in any one of the above embodiments. For example, the method may include: obtaining a plurality of image frames captured for a subject and a plurality of shooting state parameters respectively adopted when shooting the plurality of image frames, where the subject is located at the same position in each image frame; determining respectively the scaling ratios corresponding to the plurality of shooting state parameters according to a preset correspondence between shooting state parameters and scaling ratios; wherein, after scaling the image frame captured under a corresponding shooting state parameter by the scaling ratio in any one of the correspondences, the size of the subject in the scaled target image frame is the same as the size of the subject in the image reference frame; scaling the corresponding image frames according to the determined scaling ratios to obtain a plurality of target image frames, and encoding the obtained plurality of target image frames into a video file.

[0227] Correspondingly, the present disclosure also provides an electronic device, the electronic device includes a memory, and one or more programs, wherein one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs include instructions for implementing the image processing method as described in any one of the above embodiments. For example, the method may include: obtaining a plurality of image frames captured for a subject and a plurality of shooting state parameters respectively adopted when shooting the plurality of image frames, where the subject is located at the same position in each image frame; determining respectively the scaling ratios corresponding to the plurality of shooting state parameters according to a preset correspondence between shooting state parameters and scaling ratios; wherein, after scaling the image frame captured under a corresponding shooting state parameter by the scaling ratio in any one of the correspondences, the size of the subject in the scaled target image frame is the same as the size of the subject in the image reference frame; scaling the corresponding image frames according to the determined scaling ratios to obtain a plurality of target image frames, and encoding the obtained plurality of target image frames into a video file.

[0228] Figure 7 It is a block diagram of an apparatus 700 for implementing a process scheduling method shown according to an exemplary embodiment. For example, the apparatus 700 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0229] Refer to Figure 7, Device 700 may include one or more of the following components: a processing component 702, a memory 704, a power component 706, a multimedia component 708, an audio component 710, an input / output (I / O) interface 712, a sensor component 714, and a communication component 716.

[0230] The processing component 702 generally controls the overall operation of the device 700, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the above-described methods. In addition, the processing component 702 may include one or more modules to facilitate the interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0231] The memory 704 is configured to store various types of data to support the operation of the device 700. Examples of such data include instructions for any application or method operating on the device 700, contact data, phone book data, messages, pictures, videos, etc. The memory 704 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0232] The power component 706 provides power to the various components of the device 700. The power component 706 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the device 700.

[0233] The multimedia component 708 includes a screen that provides an output interface between the device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of the touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0234] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC) that is configured to receive external audio signals when the device 700 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 704 or transmitted via the communication component 716. In some embodiments, the audio component 710 further includes a speaker for outputting audio signals.

[0235] The I / O interface 712 provides an interface between the processing component 702 and a peripheral interface module, which may be a keyboard, a click wheel, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a power button, and a lock button.

[0236] The sensor component 714 includes one or more sensors for providing an assessment of the state of the device 700 in various aspects. For example, the sensor component 714 can detect the on / off state of the device 700, the relative positioning of components, such as the display and keypad of the device 700. The sensor component 714 can also detect a change in the position of the device 700 or a component of the device 700, the presence or absence of user contact with the device 700, the orientation or acceleration / deceleration of the device 700, and the temperature change of the device 700. The sensor component 714 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 714 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 714 can further include an acceleration sensor, a gyro sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0237] The communication component 716 is configured to facilitate communication between the device 700 and other devices in a wired or wireless manner. The device 700 can access a wireless network based on communication standards, such as WiFi, 2G or 3G, 4G LTE, 5G NR (New Radio), or a combination thereof. In an exemplary embodiment, the communication component 716 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0238] In an exemplary embodiment, the apparatus 700 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0239] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as a memory 704 including instructions, is also provided. The above instructions can be executed by a processor 720 of the apparatus 700 to complete the above method. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0240] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the disclosure herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not disclosed herein. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0241] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

[0242] The foregoing is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present disclosure shall be included within the scope of protection of the present disclosure.

Claims

1. An image processing method, characterized in that, it includes: Obtaining a plurality of image frames captured for a subject and a plurality of shooting state parameters respectively used when shooting the plurality of image frames, including: obtaining a plurality of image frames captured for the subject and the focal length parameters respectively used when shooting the plurality of image frames; the subject is located at the same position in each image frame; According to the preset correspondence between the shooting state parameters and the scaling ratio, respectively determining the scaling ratios corresponding to the plurality of shooting state parameters, including: according to the preset correspondence between the focal length parameters and the scaling ratio, respectively determining the scaling ratios corresponding to each focal length parameter; wherein, after scaling the image frame captured under the corresponding shooting state parameter by the scaling ratio in any correspondence, the size of the subject in the scaled target image frame is the same as the size of the subject in the image reference frame; Scaling the corresponding image frames according to the determined scaling ratios to obtain a plurality of target image frames, and encoding the obtained plurality of target image frames into a video file.

2. The method according to claim 1, characterized in that, The obtaining of the plurality of image frames and the plurality of shooting state parameters respectively used when shooting the plurality of image frames includes: obtaining a plurality of image frames and the device motion parameters respectively used when shooting the plurality of image frames; The respectively determining the scaling ratios corresponding to the plurality of shooting state parameters according to the preset correspondence between the shooting state parameters and the scaling ratio includes: respectively determining the scaling ratios corresponding to each device motion parameter according to the preset correspondence between the device motion parameters and the scaling ratio.

3. The method according to claim 2, characterized in that, The obtaining of the plurality of device motion parameters respectively used when shooting the plurality of image frames includes: Obtaining the motion information respectively detected by the motion sensor when shooting the plurality of image frames; Performing data analysis on the obtained motion information to determine the device motion parameters respectively used when shooting the plurality of image frames.

4. The method according to claim 2, characterized in that, Obtaining the device motion parameter used when shooting any one image frame includes: Obtaining the first ratio of the subject in any one image frame and the second ratio of the subject in the adjacent image frame of the any one image frame; Determining the device motion parameter used when shooting the any one image frame according to the change information of the first ratio and the second ratio.

5. The method according to claim 1, characterized in that, The image reference frame is any one of the plurality of image frames; or, the image reference frame is the earliest or the last image frame captured among the plurality of image frames; or, each of the plurality of image frames uses the previous image frame in the preset arrangement order as its own image reference frame, or each of the plurality of image frames uses the next image frame in the preset arrangement order as its own image reference frame.

6. The method according to claim 1, characterized in that, The encoding of the obtained plurality of target image frames into a video file includes: Determine the time sequence for capturing the multiple image frames; Encode the obtained multiple target image frames into a video file according to the time sequence.

7. The method according to claim 1, wherein, the encoding of the obtained multiple target image frames into a video file includes: Determine the scaling ratios respectively adopted by the multiple target image frames, and sort the multiple target image frames according to a predefined scaling ratio sorting rule; Encode the multiple target image frames into a video file according to the sorting result.

8. The method according to claim 1, wherein, the same position of each image frame is the center position of each image frame.

9. An image processing apparatus, wherein, comprising: An acquisition unit that acquires multiple image frames obtained by photographing a subject and multiple shooting state parameters respectively adopted when shooting the multiple image frames, including: acquiring multiple image frames obtained by photographing a subject and focal length parameters respectively adopted when shooting the multiple image frames; the subject is located at the same position in each image frame; A determination unit that respectively determines the scaling ratios corresponding to the multiple shooting state parameters according to a preset correspondence between shooting state parameters and scaling ratios, including: respectively determining the scaling ratios corresponding to each focal length parameter according to a preset correspondence between focal length parameters and scaling ratios; wherein, after scaling the image frame obtained under the corresponding shooting state parameter by the scaling ratio in any one of the correspondences, the size of the subject in the scaled target image frame is the same as the size of the subject in the image reference frame; A scaling unit that scales the corresponding image frames according to the determined scaling ratios to obtain multiple target image frames, and encodes the obtained multiple target image frames into a video file.

10. An electronic device, wherein, comprising: A processor; A memory for storing instructions executable by the processor; wherein, the processor realizes the method according to any one of claims 1-8 by running the executable instructions.

11. A computer-readable storage medium, on which computer instructions are stored, wherein, when the instructions are executed by a processor, the steps of the method according to any one of claims 1-8 are realized.

Images