Image processing method and device, electronic equipment and medium

Abstract

The invention relates to an image processing method and device, electronic equipment and a medium. The image processing method comprises the steps that first pose parameters of a target object in multiple frames of images are acquired, the first pose parameters are expressed in a quaternion form, the first pose parameters are used for representing the rotation state of the target object, target pose parameters corresponding to the first pose parameters are determined based on the multiple first pose parameters, and multiple target pose parameters are obtained; the target pose parameters are used for representing the first pose parameters after secondary filtering, and processing the multiple frames of images based on the multiple target pose parameters. According to the method, the first pose parameter is filtered twice, so that the accuracy of the target pose parameter can be further improved, the stable effect of the processed image is better, and the display effect of the image and the use experience of a user are improved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device and medium. Background Technology

[0002] Currently, in the field of image stabilization, such as EIS (Electronic Image Stabilization), electronic devices analyze multiple frames of continuously captured images and perform corresponding digital processing to obtain stable images.

[0003] For example, electronic devices perform a moving average operation on the parameters of the quaternions corresponding to each frame of the image, and then process the image based on the quaternions obtained from the moving average. However, the moving average operation has a certain degree of error, and the accuracy of the quaternions obtained from the moving average operation is low. The image processed based on the quaternions obtained from the moving average cannot achieve the expected stable effect, resulting in a poor user experience. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides an image processing method, apparatus, electronic device, and medium.

[0005] According to a first aspect of the present disclosure, an image processing method is provided, comprising:

[0006] Obtain the first pose parameter of the target object in multiple frames of images; the first pose parameter is represented in quaternion form and is used to characterize the rotation state of the target object;

[0007] Based on multiple first pose parameters, a target pose parameter corresponding to each first pose parameter is determined, resulting in multiple target pose parameters; the target pose parameter is used to characterize the first pose parameter after secondary filtering.

[0008] The multi-frame images are processed based on multiple target pose parameters.

[0009] In some embodiments, determining the target pose parameter corresponding to each of the plurality of first pose parameters includes:

[0010] Based on multiple first pose parameters, a second pose parameter corresponding to each first pose parameter is determined; the second pose parameter is used to characterize the first pose parameter after one filtering.

[0011] Based on multiple second pose parameters, multiple target pose parameters are determined.

[0012] In some embodiments, determining a second pose parameter corresponding to each of the plurality of first pose parameters includes:

[0013] Based on a first preset function and multiple first pose parameters, a second pose parameter corresponding to each first pose parameter is determined; the first preset function is used to indicate the functional relationship between the second pose parameter of the target object in the current frame image and the second pose parameter of the target object in the previous frame image, wherein the current frame image is any frame image among the multiple frames, and the previous frame image is the frame image preceding the current frame image.

[0014] In some embodiments, determining the second pose parameter corresponding to each of the first pose parameters based on a first preset function and a plurality of first pose parameters includes:

[0015] For a first target image, obtain the second pose parameters of the target object in the second target image, wherein the first target image is any frame image in the multi-frame images, and the second target image is the previous frame image of the first target image in the multi-frame images;

[0016] The second pose parameter of the target object in the first target image is determined based on the first pose parameter of the target object in the first target image, the second pose parameter of the target object in the second target image, and the first preset function.

[0017] In some embodiments, when the second target image is the first frame image in the multi-frame image, the second pose parameter of the target object in the second target image is the first pose parameter of the target object in the second target image.

[0018] In some embodiments, determining a plurality of target pose parameters based on a plurality of second pose parameters includes:

[0019] Based on multiple second pose parameters and a second preset function, multiple target pose parameters are determined. The second preset function is used to indicate the functional relationship between the target pose parameters of the target object in the next frame image and the target pose parameters of the target object in the current frame image. The next frame image is the next frame image after the current frame image.

[0020] In some embodiments, determining the plurality of target pose parameters based on a plurality of second pose parameters and a second preset function includes:

[0021] For the first target image, the second pose parameters of the target object in the third target image are obtained, wherein the first target image is any frame image in the multi-frame images, and the third target image is the next frame image of the target image in the multi-frame images;

[0022] Based on the second pose parameters of the target object in the third target image and the second preset function, the target pose parameters of the target object in the first target image are determined.

[0023] In some embodiments, when the third target image is the last frame of the multi-frame images, the target pose parameter of the target object in the third target image is the second pose parameter of the target object in the third target image.

[0024] In some embodiments, the method further includes:

[0025] Based on multiple first pose parameters and a preset model, a first preset function and a second preset function are determined; the preset model is used to indicate the functional relationship between the first pose parameters and the target pose parameters, and the preset model is a normal distribution model.

[0026] In some embodiments, processing the multi-frame images based on multiple target pose parameters includes:

[0027] Determine the rotation matrix corresponding to each of the target pose parameters;

[0028] Based on the rotation matrix corresponding to each target pose parameter, the image corresponding to each target pose parameter is processed.

[0029] According to a second aspect of the present disclosure, an image processing apparatus is provided, comprising:

[0030] The acquisition module is configured to acquire the first pose parameter of the target object in multiple frames of images; the first pose parameter is represented in quaternion form and is used to characterize the rotation state of the target object;

[0031] The determination module is configured to determine a target pose parameter corresponding to each of the first pose parameters based on a plurality of first pose parameters, thereby obtaining a plurality of target pose parameters; the target pose parameters are used to characterize the first pose parameter after secondary filtering;

[0032] The processing module is configured to process the multi-frame images based on multiple target pose parameters.

[0033] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0034] processor;

[0035] Memory used to store processor-executable instructions;

[0036] The processor is configured to perform the image processing method as described in the first aspect of this disclosure.

[0037] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the image processing method as described in the first aspect of the present disclosure.

[0038] The method described in this disclosure has the following advantages: by performing two filters on the first pose parameter, this disclosure can further improve the accuracy of the target pose parameter, thereby ensuring better stability of the processed image, and thus improving the image display effect and user experience.

[0039] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0041] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment.

[0042] Figure 2 This is a flowchart illustrating an image processing method according to an exemplary embodiment.

[0043] Figure 3 This is a schematic diagram of a motion path according to an exemplary embodiment.

[0044] Figure 4 This is a flowchart illustrating an image processing method according to an exemplary embodiment.

[0045] Figure 5 This is a schematic diagram of a preset model shown according to an exemplary embodiment.

[0046] Figure 6 This is a flowchart illustrating an image processing method according to an exemplary embodiment.

[0047] Figure 7 This is a block diagram of an image processing apparatus according to an exemplary embodiment.

[0048] Figure 8This is a block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0049] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0050] In pose representation, quaternions are commonly used to represent the rotation of an object or coordinate system. Quaternions can be used to implement rotational transformations of vectors, thereby enabling pose control of objects, camera angle adjustment, and other similar functions. Currently, in image stabilization fields, such as EIS, electronic devices analyze multiple consecutively captured images and perform corresponding digital processing to achieve image stabilization. Specifically, the electronic device performs a moving average operation on the parameters of the quaternions corresponding to each frame and processes the image based on the quaternions obtained from the moving average. However, because quaternions cannot be directly added or subtracted, the moving average operation introduces some error. Therefore, the accuracy of the quaternions obtained from the moving average is relatively low, and the image processed based on the moving average quaternions does not achieve the expected stabilization effect, resulting in a poor user experience.

[0051] To address the aforementioned problems, this disclosure provides an image processing method. This method obtains the first pose parameter of a target object from multiple frames of images. The first pose parameter is represented as a quaternion and characterizes the rotation state of the target object. Based on multiple first pose parameters, a target pose parameter corresponding to each first pose parameter is determined, resulting in multiple target pose parameters. These target pose parameters characterize the first pose parameters after secondary filtering. Subsequently, the multiple frames of images are processed based on these multiple target pose parameters. This embodiment of the disclosure can obtain more accurate target pose parameters through two filtering operations, and based on these more accurate target pose parameters, an image with better processing results is obtained, thereby improving image stabilization and user experience.

[0052] The image processing method provided in this disclosure is executed by an electronic device, which may specifically be a mobile phone, tablet computer, laptop, intelligent robot, smart wearable device, or other intelligent device. Furthermore, the electronic device is equipped with various hardware resources and energy storage devices that provide power for the operation of these hardware resources. It should be noted that the filtering algorithm shown in this disclosure can be applied to path planning and image processing. Since this disclosure involves image processing, it can be applied to image stabilization scenarios for electronic devices and can process shaky images into stable images, thereby achieving the technical effect of image stabilization.

[0053] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment, executed by an electronic device, see [link to flowchart]. Figure 1 The method includes the following steps:

[0054] Step S101: Obtain the first pose parameter of the target object in the multi-frame images.

[0055] The first pose parameter is represented as a quaternion, which consists of one real part and three imaginary parts. Quaternions can be used to represent rotations about any axis. The first pose parameter is used to characterize the rotational state of the target object, such as its position and orientation in three-dimensional space.

[0056] In some embodiments, the first pose parameter of a target object in multiple frames of images can be acquired by a sensor installed in an electronic device. That is, the first pose parameter can be monitored or observed by a device in the electronic device. In some embodiments, the first pose parameter can be represented as... Among them, X i p represents the first pose parameter of the i-th frame image. wi Let represent the real part of the quaternion corresponding to the first pose parameter of the i-th frame image. These represent the components of the three imaginary parts of the quaternion corresponding to the first pose parameter of the i-th frame image, where i represents the i-th frame image and the maximum value of i is N, meaning that a total of N frames are acquired in this embodiment.

[0057] It should be noted that the target object can be any object in the image, such as a person or object in the image; or, the target object can be a pixel in the image. In some embodiments, after sorting multiple frames of images in chronological order, multiple first pose parameters arranged in chronological order can be obtained. Based on the multiple first pose parameters arranged in chronological order, the motion path of the target object in the multiple frames of images can be determined.

[0058] Step S102: Based on multiple first pose parameters, determine the target pose parameters corresponding to each first pose parameter to obtain multiple target pose parameters; the target pose parameters are used to characterize the first pose parameters after secondary filtering.

[0059] In some embodiments, the target pose parameters can be expressed as: in, Let q represent the target pose parameters of the i-th frame image. wi Let represent the real part of the quaternion corresponding to the target pose parameters of the i-th frame image. These represent the components of the three imaginary parts of the quaternions corresponding to the target pose parameters of the i-th frame image.

[0060] Furthermore, it should be noted that the secondary filtering in this embodiment includes a first filtering and a second filtering, and neither of these two filtering operations involves addition or subtraction of the quaternions corresponding to the pose parameters. In one example, the first and second filtering operations may involve multiplication or division of the quaternions corresponding to the pose parameters. Since no addition or subtraction of quaternions is involved, the accuracy of the target pose parameters determined after the two filtering operations is high, and the stability of the image processed based on the target pose parameters is also better.

[0061] Step S103: Process multiple frames of images based on multiple target pose parameters.

[0062] Each frame of the image contains a target image corresponding to a first pose parameter and a target pose parameter. In some embodiments, processing multiple frames of images includes: processing the image corresponding to each target pose parameter. For example, in this embodiment of the disclosure, images A, B, and C are provided, wherein the target pose parameters corresponding to images A, B, and C are target pose parameter a, target pose parameter b, and target pose parameter c, respectively. Accordingly, image A can be processed based on target pose parameter a, image B can be processed based on target pose parameter b, and image C can be processed based on target pose parameter c, thereby achieving targeted image processing and obtaining images with better processing results.

[0063] In some embodiments, processing the image corresponding to each target pose parameter includes: determining a rotation matrix corresponding to each target pose parameter, and processing the image corresponding to each target pose parameter based on the rotation matrix. The rotation matrix can be used to rotate and transform the pose parameters of the target object in the image, and can be calculated based on the imaginary part of the quaternion corresponding to the target pose parameter. The specific calculation process will not be elaborated here.

[0064] In some embodiments, after determining multiple target pose parameters, the motion path of the target object in multiple frames of images can be re-determined using multiple target pose parameters arranged in chronological order, thereby determining the jitter path of the electronic device, and thus processing the images acquired during image capture or video capture of the electronic device with jitter based on the jitter path.

[0065] In related technologies, a moving average operation is used to filter quaternions, which involves addition and subtraction operations between quaternions. Since quaternions cannot be directly added or subtracted, the accuracy of the quaternions after moving average is low. Using the quaternions after moving average for image processing also leads to poor image processing results and a poor user experience. Based on the above embodiments, it can be determined that the two filtering operations on the first pose parameter in this embodiment do not involve addition and subtraction operations between quaternions, but rather multiplication and division operations. Therefore, this embodiment can obtain more accurate target pose parameters through two filtering operations, and based on these more accurate target pose parameters, obtain an image with better processing results, thereby improving image stabilization and user experience.

[0066] In some embodiments, determining the target pose parameter corresponding to each first pose parameter based on multiple first pose parameters includes: determining the second pose parameter corresponding to each first pose parameter based on multiple first pose parameters; the second pose parameter is used to characterize the first pose parameter after one filtering step, and multiple target pose parameters are determined based on the multiple second pose parameters. That is, for each frame of image, the first pose parameter is first filtered to obtain the second pose parameter corresponding to the first pose parameter, and then the second pose parameter is filtered a second time to obtain the target pose parameter corresponding to the second pose parameter. The following describes... Figure 2 The illustrated embodiment explains the process of determining multiple target pose parameters.

[0067] Figure 2 This is a flowchart illustrating an image processing method according to an exemplary embodiment, executed by an electronic device, see [link to flowchart]. Figure 2 The method includes the following steps:

[0068] Step S201: Based on multiple first pose parameters, determine the second pose parameter corresponding to each first pose parameter; the second pose parameter is used to characterize the first pose parameter after one filtering.

[0069] In some embodiments, the target pose parameters can be expressed as: Among them, Z i This represents the second pose parameter of the i-th frame image. Let represent the real part of the quaternion corresponding to the second pose parameter of the i-th frame image. These represent the components of the three imaginary parts of the quaternion corresponding to the second pose parameter of the i-th frame image.

[0070] In some embodiments, a second pose parameter corresponding to each first pose parameter can be determined based on a first preset function and multiple first pose parameters. The first preset function indicates the functional relationship between the second pose parameter of the target object in the current frame image and the second pose parameter of the target object in the previous frame image. The current frame image is any frame image among multiple frames, and the previous frame image is the frame image preceding the current frame image. Furthermore, the parameters in the first preset function include multiple first pose parameters.

[0071] In one example, the first preset function may further include a first preset parameter. The functional relationship between the second pose parameter of the target object in the current frame and the second pose parameter of the target object in the previous frame can be a functional relationship between the second pose parameter of the target object in the current frame and the second pose parameter of the target object in the previous frame, determined based on the first preset parameter. For example, the first preset function can be multiplying the second pose parameter of the target object in the previous frame by the first preset parameter to obtain the second pose parameter of the target object in the next frame; or, the first preset function can be dividing the second pose parameter of the target object in the previous frame by the first preset parameter to obtain the second pose parameter of the target object in the next frame. The type and value of the first preset parameter can be set based on actual needs.

[0072] In some embodiments, since the second pose parameter of the target object in the current frame image is influenced by the second pose parameter of the target object in the previous frame image, the forward calculation process can be defined by determining the second pose parameter corresponding to each first pose parameter based on the first preset function and multiple first pose parameters. The forward calculation process will be discussed below, with the first target image defined as the current frame image and the second target image defined as the previous frame image.

[0073] For a first target image, the second pose parameters of the target object in the second target image are obtained. Based on the first pose parameters of the target object in the first target image, the second pose parameters of the target object in the second target image, and a first preset function, the second pose parameters of the target object in the first target image are determined. Here, the first target image is any frame in a multi-frame image set, and the second target image is the frame preceding the first target image in the multi-frame image set. The first preset function indicates the parameter relationship between the second pose parameters of the target object in the first target image and the second pose parameters of the target object in the second target image. The first preset function includes the first pose parameters of the target object in the first target image, or parameters determined based on the first pose parameters of the target object in the first target image. By repeatedly executing the above process, the second pose parameters of the target objects in all images can be determined.

[0074] It should be noted that, in order to ensure the successful execution of the first filtering operation, in some embodiments, the second pose parameter of the target object in the first frame of the multi-frame image can be determined based on the first pose parameter of the target object in the first frame of the multi-frame image. Then, based on the second pose parameter of the target object in the first frame image and the first preset function, the second pose parameter of the target object in the second frame of the multi-frame image can be determined. After that, based on the determined second pose parameter of the target object in the second frame of the multi-frame image and the first preset function, the second pose parameter of the target object in the third frame of the multi-frame image can be determined. This process is repeated until the second pose parameter of the target object in all images is determined.

[0075] In one example, when the second target image is the first frame of a multi-frame image, the second pose parameter of the target object in the second target image is the first pose parameter of the target object in the second target image.

[0076] To facilitate understanding, a specific example is given below to illustrate the forward computation process:

[0077] The multi-frame images include images A, B, and C, which are ordered chronologically. The first pose parameters of images A, B, and C are a1, b1, and c1, respectively. When determining the second pose parameter b2 of the first target image B (the second frame in the multi-frame image set), the first pose parameter a1 is first determined as the second pose parameter of the second target image A (here named second pose parameter a2; the first pose parameter a1 and the second pose parameter a2 are the same). Based on the second pose parameter a2 and a first preset function, the second pose parameter b2 is determined. Similarly, after determining the second pose parameter b2, the second pose parameter c2 of the first target image C is determined based on the second pose parameter b2 and the first preset function. In this way, the second pose parameter corresponding to each first pose parameter can be successfully determined.

[0078] In some embodiments, the first preset function can be expressed as the following formula:

[0079] P(Z i ,X 1,2,…,i )=N(Z i F1μ i-1 +K i (X i -F1μ i-1 ),(1-K i V i-1 (1)

[0080]

[0081] Among them, P(Z) i ,X 1,2,…,i Z represents the first preset function, N() represents the normal distribution model, and Z i F1 represents the second pose parameter of the target object in the i-th frame image, and μ represents the first collision parameter. i-1 Z represents i-1 The expected value, K i X represents the first interpolation ratio. i V represents the first pose parameter of the target object in the i-th frame image. i-1 Let Q represent the first variance of the (i-1)th frame, Q represent the second variance, and R represent the third variance.

[0082] In one example, Z1 = X1, V1 = 1, Q = 1, R = 1. Expanding the above formula (1), the forward calculation process after expansion includes:

[0083] ΔZ i =SLERP(ΔZ) i-1 ,ΔX i ,Ki (3)

[0084]

[0085] V i = (1-K) i V i-1 (6)

[0086]

[0087] Z i =Z i-1 *ΔZ i (8)

[0088] Where, ΔZ i SLERP() represents the angular difference between the second pose parameter of the target object in the i-th frame and the second pose parameter of the target object in the previous frame (i.e., the (i-1)-th frame). ΔZ represents the interpolation algorithm. i-1 Let ΔX represent the angular difference between the second pose parameter of the target object in the (i-1)th frame and the second pose parameter of the target object in the previous frame of the (i-2)th frame. i α1(ΔZ) represents the angular difference between the first pose parameter of the target object in the i-th frame and the first pose parameter of the target object in the (i-1)-th frame. i-1 ) represents the first collision edge parameter, and F = α1(ΔZ) i-1 V i Let represent the first variance of the i-th frame image.

[0089] Using formulas (3)-(8) above, the second pose parameters of the target object in each frame of the image can be determined. Specifically, K in formula (7) can be determined using formulas (4) and (6). i The value of ΔZ is determined by formulas (4), (5) and (7). i Therefore, based on ΔZ i and Z i-1 Determine Z i .

[0090] Step S202: Determine multiple target pose parameters based on multiple second pose parameters.

[0091] Through step S201 described above, the second pose parameter corresponding to each first pose parameter can be determined. In some embodiments, multiple target pose parameters can be determined based on multiple second pose parameters and a second preset function. The second preset function indicates the functional relationship between the target pose parameters of the target object in the next frame image and the target pose parameters of the target object in the current frame image; the next frame image is the frame following the current frame image.

[0092] In one example, the second preset function may include a second preset parameter. The functional relationship between the target pose parameters of the target object in the next frame and the target pose parameters of the target object in the current frame can be a functional relationship between the target pose parameters of the target object in the next frame and the target pose parameters of the target object in the current frame, determined based on the second preset parameter. For example, the second preset function can be the target pose parameters of the target object in the next frame multiplied by the second preset parameter to obtain the target pose parameters of the target object in the current frame; or, the second preset function can be the target pose parameters of the target object in the previous frame divided by the second preset parameter to obtain the target pose parameters of the target object in the current frame. The type and value of the second preset parameter can be set based on actual needs.

[0093] In some embodiments, since the target pose parameters of the target object in the current frame image are affected by the target pose parameters of the target object in the next frame image, the process of determining the target pose parameter corresponding to each second pose parameter based on the second preset function and multiple second pose parameters can be defined as the backward calculation process. The backward calculation process will be discussed below, with the first target image defined as the current frame image and the third target image defined as the next frame image of the current frame image:

[0094] For the first target image, the second pose parameters of the target object in the third target image are obtained. Based on the second pose parameters of the target object in the third target image and a second preset function, the target pose parameters of the target object in the first target image are determined. Here, the first target image is any frame in a multi-frame image set, the third target image is the next frame in the multi-frame image set, and the second preset function indicates the functional relationship between the target pose parameters of the target object in the third target image and the target pose parameters of the target object in the first target image.

[0095] It should be noted that, to ensure the successful execution of the second filtering operation, in some embodiments, the target pose parameters of the target object in the last frame of the multi-frame image can be determined based on the second pose parameters of the target object in the last frame. Then, based on the target pose parameters of the target object in the last frame and a second preset function, the target pose parameters of the target object in the penultimate frame are determined. Subsequently, based on the determined target pose parameters of the target object in the penultimate frame and the second preset function, the target pose parameters of the target object in the third-to-last frame are determined. This process is repeated until the target pose parameters of the target objects in all images are determined. In one example, when the third target image is the last frame of the multi-frame image, the second pose parameter of the target object in the third target image is the first pose parameter of the target object in the second target image.

[0096] To facilitate understanding, a specific example is given below to illustrate the backward computation process:

[0097] For example, a multi-frame image includes image A, image B, and image C, and images A, B, and C are ordered chronologically. The second pose parameters of images A, B, and C are second pose parameter a2, second pose parameter b2, and second pose parameter c2, respectively. When determining the target pose parameter b3 of the first target image B (the penultimate frame in the multi-frame image), the second pose parameter c2 is first determined as the target pose parameter of the third target image C (here named target pose parameter c3, the same as the second pose parameter c2 and target pose parameter c3), and the target pose parameter b3 is determined based on the target pose parameter c3 and the second preset function. Similarly, after determining the target pose parameter b3, the target pose parameter a3 of the first target image A is determined based on the target pose parameter b3 and the second preset function. In this way, the target pose parameter corresponding to each second pose parameter can be successfully determined.

[0098] In some embodiments, the second preset function can be expressed as the following formula:

[0099]

[0100] in, L represents the target pose parameters of the target object in the i-th frame image. i Indicates the second interpolation ratio. F2 represents the second variance of the (i+1)th frame image, F2 represents the second collision parameter, and ∝ represents the proportional sign.

[0101] In one example, define Expanding the above formula (9), the backward calculation process after expansion includes:

[0102]

[0103] in, Δμ represents the angular difference between the target pose parameters of the target object in the (i+1)th frame and the target pose parameters of the target object in the i-th frame. i+1 ΔZ represents the angular difference between the target pose parameters of the target object in the (i+2)th frame and the target pose parameters of the target object in the (i+1)th frame. i ' represents the angular difference between the second pose parameter of the target object in the (i+1)th frame and the second pose parameter of the target object in the ith frame, 1-α²(Δμ) i+1 ) represents the second collision edge parameter, and F2 = 1 - α2(Δμ) i+1 ).

[0104] Using formulas (11)-(15) above, the target pose parameters of the target object in each frame of the image can be determined. Specifically, L in formula (14) can be determined using formula (12). i The value of is determined by formulas (12), (13), and (14). Furthermore based on and Determine Z i .

[0105] In some embodiments, a first motion path corresponding to a plurality of first pose parameters, a second motion path corresponding to a plurality of second pose parameters, and a third motion path corresponding to a plurality of third pose parameters can be determined respectively. For example... Figure 3 The diagram shows the motion path. Figure 3 Solid lines represent the first motion path, dashed lines represent the second motion path, dashed lines represent the third motion path, and dotted lines represent the fourth motion path. The third and fourth motion paths represent the third motion paths corresponding to different variances. Figure 3 The schematic diagram of the motion paths shown indicates that the second, third, and fourth motion paths are smoother than the first motion path, and the third and fourth motion paths are smoother than the second motion path. It should be noted that paths with smaller variance are smoother, and the filtering state may lag more. Figure 3 The schematic diagram of the motion paths shown indicates that the third motion path lags behind the fourth motion path. Therefore, the variance corresponding to the third motion path is less than the variance corresponding to the fourth motion path.

[0106] This disclosure provides a specific method for determining the second pose parameter and the target pose parameter. The second pose parameter can be calculated first through a forward calculation process shown by a first preset function, and then updated and optimized through a backward calculation process shown by the second preset function to obtain the target pose parameter. This further improves the accuracy of the pose parameter, as well as the image stabilization effect and user experience. In addition, this disclosure can effectively reduce the noise of the signal corresponding to the first pose parameter, making the final motion path of the electronic device smoother and more stable.

[0107] In some embodiments, before determining the second pose parameters and the target pose parameters, a first preset function and a second preset function can be determined based on multiple first pose parameters and a preset model. The following describes... Figure 4 The illustrated embodiment explains the process of determining multiple target pose parameters.

[0108] Figure 4This is a flowchart illustrating an image processing method according to an exemplary embodiment, executed by an electronic device, see [link to flowchart]. Figure 4 The method includes the following steps:

[0109] Step S401: Determine the preset probability model based on multiple first pose parameters and preset models.

[0110] The preset probability model is used to characterize the maximum probability of the target pose parameter corresponding to the first pose parameter. The preset model indicates the functional relationship between the first pose parameter and the target pose parameter; the preset model is a normal distribution model. For example, ... Figure 5 The diagram shown is a schematic of the preset model. The preset model can be obtained by adding the target pose parameters and the error parameters to obtain the first pose parameter. The error parameters follow a normal distribution.

[0111] In some embodiments, the preset probability model can be expressed as follows:

[0112]

[0113] in, This indicates a pre-defined probability model.

[0114] Step S402: Based on the preset probability model and multiple first pose parameters, determine the first preset function and the second preset function.

[0115] Decomposing the above-mentioned pre-defined probability model yields the following formula:

[0116] P(Z i |X 1,2,…,N )=∫P(Z i |Z i+1 ,X 1,2,…,N )P(Z i+1 |X 1,2,…,N )dZ i+1 (18)

[0117]

[0118] P(Z i ,X 1,2,…,i )=∫P(Z i Z i-1 ,X 1,2,…,i )dZ i-1 =∫P(Z) i-1 ,X 1,2,…,i-1 )P(Z i |Z i-1 )P(X i |Z i )dZ i-1 (twenty one)

[0119] To save space, some parameters in the above formula will be explained. The definitions of other parameters can be found in the previously explained definitions and will not be repeated here. Among them, P(Z i |X 1,2,…,N P(Z) represents the probability of the second pose parameter of the target object in the i-th frame image given the first pose parameter of the target object in all images; i |Z i+1 ,X 1,2,…,N P(Z) represents the probability of the second pose parameter of the target object in the i-th frame, given the probabilities of the first pose parameter and the second pose parameter of the target object in all images; i Z i+1 ,X 1,2,…,N ) represents the probability that the second pose parameter of the target object in the i-th frame is in the hidden state when given the second pose parameter of the target object in the (i+1)-th frame and the first pose parameter of the target object in all images (here, it can refer to the probability that the second pose parameter of the target object in the i-th frame is a preset value).

[0120] In some embodiments, a first preset function and a second preset function can be obtained based on multiple formulas derived from decomposing a preset probability model and a known probability model. The formula expression for the known probability model is as follows:

[0121] P(Z i ,X 1,2,…,i )=N(Z i |μ i V i )(twenty two)

[0122] Where, μ i Z represents i The expected value, V i This represents the first variance.

[0123] P(Z i |Z i-1 )=N(Z i |FZ i-1 ,Q)(23)

[0124] Among them, FZ i-1 For ΔZ i-1 Q represents the difference between the third and third parties.

[0125] P(X i |Z i )=N(X i |Z i ,R)(24)

[0126] Where R represents the fourth variance.

[0127] In one example, substituting formulas (22)-(24) into formula (21) yields the first preset function. In another example, substituting formula (23) and the first preset function into formula (20) yields the second preset function.

[0128] The embodiments of this disclosure can determine a first preset function and a second preset function through a preset probability model, thereby determining the second pose parameter corresponding to each first pose parameter, and determining the target pose parameter corresponding to each second pose parameter, thereby realizing image processing and obtaining an image with better stability.

[0129] like Figure 6 As shown below, a specific implementation process is given:

[0130] S601. Based on multiple first pose parameters and preset models, determine the first preset function and the second preset function.

[0131] S602. For the first target image, obtain the second pose parameters of the target object in the second target image.

[0132] S603. Based on the first pose parameter of the target object in the first target image, the second pose parameter of the target object in the second target image, and the first preset function, determine the second pose parameter of the target object in the first target image.

[0133] S604. Determine multiple second pose parameters for all first target images.

[0134] S605. For the first target image, obtain the second pose parameters of the target object in the third target image.

[0135] S606. Based on the second pose parameters and the second preset function of the target object in the third target image, determine the target pose parameters of the target object in the first target image.

[0136] S607. Determine multiple target pose parameters for all first target images.

[0137] S608. Determine the rotation matrix corresponding to each target pose parameter.

[0138] S609. Based on the rotation matrix corresponding to each target pose parameter, process the image corresponding to each target pose parameter.

[0139] Figure 7 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment, configured in an electronic device, see [link to relevant documentation]. Figure 7 The device includes:

[0140] The acquisition module 701 is configured to acquire the first pose parameter of the target object in multiple frames of images; the first pose parameter is represented in quaternion form and is used to characterize the rotation state of the target object.

[0141] The determination module 702 is configured to determine the target pose parameter corresponding to each first pose parameter based on multiple first pose parameters, thereby obtaining multiple target pose parameters; the target pose parameters are used to characterize the first pose parameters after secondary filtering.

[0142] The processing module 703 is configured to process multiple frames of images based on multiple target pose parameters.

[0143] In some embodiments, the determining module 702 is configured to:

[0144] Based on multiple first pose parameters, determine the second pose parameter corresponding to each first pose parameter; the second pose parameter is used to characterize the first pose parameter after one filtering.

[0145] Multiple target pose parameters are determined based on multiple second pose parameters.

[0146] In some embodiments, the determining module 702 is configured to:

[0147] Based on the first preset function and multiple first pose parameters, the second pose parameter corresponding to each first pose parameter is determined; the first preset function is used to indicate the functional relationship between the second pose parameter of the target object in the current frame image and the second pose parameter of the target object in the previous frame image, the current frame image is any frame image in the multiple frames image, and the previous frame image is the frame image before the current frame image.

[0148] In some embodiments, the determining module 702 is configured to:

[0149] For the first target image, obtain the second pose parameters of the target object in the second target image. The first target image is any frame image in the multi-frame image, and the second target image is the previous frame image of the first target image in the multi-frame image.

[0150] The second pose parameter of the target object in the first target image is determined based on the first pose parameter of the target object in the first target image, the second pose parameter of the target object in the second target image, and the first preset function.

[0151] In some embodiments, when the second target image is the first frame of a multi-frame image, the second pose parameter of the target object in the second target image is the first pose parameter of the target object in the second target image.

[0152] In some embodiments, the determining module 702 is configured to:

[0153] Based on multiple second pose parameters and a second preset function, multiple target pose parameters are determined. The second preset function is used to indicate the functional relationship between the target pose parameters of the target object in the next frame image and the target pose parameters of the target object in the current frame image. The next frame image is the next frame image after the current frame image.

[0154] In some embodiments, the determining module 702 is configured to:

[0155] For the first target image, obtain the second pose parameters of the target object in the third target image. The first target image is any frame image in the multi-frame image, and the third target image is the next frame image of the target image in the multi-frame image.

[0156] Based on the second pose parameters of the target object in the third target image and the second preset function, the target pose parameters of the target object in the first target image are determined.

[0157] In some embodiments, when the third target image is the last frame of a multi-frame image, the target pose parameter of the target object in the third target image is the second pose parameter of the target object in the third target image.

[0158] In some embodiments, the determining module 702 is configured to:

[0159] Based on multiple first pose parameters and preset models, a first preset function and a second preset function are determined; the preset model is used to indicate the functional relationship between the first pose parameters and the target pose parameters, and the preset model is a normal distribution model.

[0160] In some embodiments, the processing module 703 is configured to:

[0161] Determine the rotation matrix corresponding to each target pose parameter;

[0162] Based on the rotation matrix corresponding to each target pose parameter, the image corresponding to each target pose parameter is processed.

[0163] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0164] This disclosure also provides an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the image processing method described above.

[0165] Figure 8 This is a block diagram of an electronic device 800 according to an exemplary embodiment.

[0166] Reference Figure 8 The electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0167] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0168] Memory 804 is configured to store various types of data to support the operation of electronic device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0169] Power supply component 806 provides power to various components of electronic device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0170] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0171] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0172] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0173] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 can detect the on / off state of electronic device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0174] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0175] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0176] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0177] A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform an image processing method provided by an exemplary embodiment of the present disclosure.

[0178] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0179] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized by, The method comprises: obtaining a first pose parameter of a target object in multiple images; the first pose parameter is expressed in quaternion form, and the first pose parameter is used to represent a rotation state of the target object; based on multiple first pose parameters, determining a target pose parameter corresponding to each first pose parameter, obtaining multiple target pose parameters; the target pose parameter is used to represent a first pose parameter after secondary filtering; based on multiple target pose parameters, processing the multiple images.

2. The image processing method of claim 1, wherein, The method comprises: based on multiple first pose parameters, determining a second pose parameter corresponding to each first pose parameter; the second pose parameter is used to represent a first pose parameter after primary filtering; based on multiple second pose parameters, determining multiple target pose parameters.

3. The image processing method of claim 2, wherein, The method comprises: based on a first preset function and multiple first pose parameters, determining a second pose parameter corresponding to each first pose parameter; the first preset function is used to indicate a functional relationship between a second pose parameter of the target object in a current frame image and a second pose parameter of the target object in a previous frame image, the current frame image is any frame image in the multiple images, and the previous frame image is a frame image before the current frame image.

4. The image processing method of claim 3, wherein, The method comprises: for a first target image, obtaining a second pose parameter of the target object in a second target image, the first target image being any frame image in the multiple images, and the second target image being a frame image before the first target image in the multiple images; based on a first pose parameter of the target object in the first target image, a second pose parameter of the target object in the second target image, and the first preset function, determining a second pose parameter of the target object in the first target image.

5. The image processing method of claim 4, wherein, When the second target image is a first frame image in the multiple images, the second pose parameter of the target object in the second target image is a first pose parameter of the target object in the second target image.

6. The image processing method of claim 2, wherein, The method comprises: based on multiple second pose parameters and a second preset function, determining multiple target pose parameters, the second preset function being used to indicate a functional relationship between a target pose parameter of the target object in a next frame image and a target pose parameter of the target object in a current frame image, the next frame image being a frame image after the current frame image.

7. The image processing method of claim 6, wherein, The method comprises: For a first target image, a second pose parameter of the target object in a third target image is acquired, the first target image is any frame image in the plurality of images, and the third target image is a next frame image of the target image in the plurality of images; Based on the second preset function and the second pose parameter of the target object in the third target image, a target pose parameter of the target object in the first target image is determined.

8. The image processing method of claim 7, wherein, When the third target image is the last frame image in the plurality of images, the target pose parameter of the target object in the third target image is the second pose parameter of the target object in the third target image.

9. The image processing method of claim 1, wherein, The method further comprises: Based on the plurality of first pose parameters and a preset model, a first preset function and a second preset function are determined; the preset model is used to indicate a functional relationship between the first pose parameters and the target pose parameters, and the preset model is a normal distribution model.

10. The image processing method of claim 1, wherein, The processing of the plurality of images based on the plurality of target pose parameters comprises: A rotation matrix corresponding to each target pose parameter is determined; Based on the rotation matrix corresponding to each target pose parameter, an image corresponding to each target pose parameter is processed.

11. An image processing apparatus characterized by comprising: It comprises: An acquisition module configured to acquire a first pose parameter of a target object in a plurality of images; The first pose parameter is expressed in quaternion form, and the first pose parameter is used to represent a rotation state of the target object; A determination module configured to determine a target pose parameter corresponding to each first pose parameter based on a plurality of first pose parameters, thereby obtaining a plurality of target pose parameters; The target pose parameter is used to represent a second-filtered first pose parameter; A processing module configured to process the plurality of images based on a plurality of target pose parameters.

12. An electronic device, comprising: It comprises: A processor; A memory for storing processor-executable instructions; The processor is configured to execute the image processing method according to any one of claims 1-10.

13. A non-transitory computer-readable storage medium, comprising: When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can execute the image processing method according to any one of claims 1-10.

Images