Photographing method and device, electronic equipment and storage medium

Abstract

The invention relates to a photographing method and device, electronic equipment and a storage medium. The photographing method comprises the following steps: in response to starting of a camera application, identifying a target main body in a preview image, and obtaining a motion vector of the target main body between adjacent frames in a preview image frame queue to obtain an image coordinate system motion vector; converting the motion vector of the image coordinate system into a motion vector of a Hall coordinate system based on a vector conversion relation between the image coordinate system and the Hall coordinate system; based on the Hall coordinate system motion vector, determining a Hall compensation value for carrying out motion compensation on the target main body; and in response to a received photographing instruction, based on the Hall compensation value, motion compensation is performed on a target subject in a target image frame to obtain a photographed image, the target image frame is first N preview image frames closest to the moment when the photographing instruction is received, and N is a positive integer. According to the invention, the photographed image is sharpened.

Description

Technical Field

[0001] This disclosure relates to the field of photographic imaging technology, and in particular to a photographic method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the continuous development of computer technology, people have increasingly higher requirements for the clarity of photographic images.

[0003] In related technologies, image stability has become a crucial factor affecting image quality due to factors such as hand shakiness. Optical image stabilization (OIS) technology typically relies on Hall effect sensors to detect camera movement, but these sensors are expensive and difficult to meet user needs. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a photographing method, apparatus, electronic device, and storage medium.

[0005] According to a first aspect of the present disclosure, a method for taking a picture is provided, comprising: in response to the launch of a camera application, identifying a target subject in a preview image and obtaining motion vectors of the target subject between adjacent frames in a preview image frame queue to obtain an image coordinate system motion vector; converting the image coordinate system motion vector into a Hall coordinate system motion vector based on a vector transformation relationship between the image coordinate system and the Hall coordinate system; determining a Hall compensation value for motion compensation of the target subject based on the Hall coordinate system motion vector; and in response to receiving a picture-taking command, performing motion compensation on the target subject in a target image frame based on the Hall compensation value to obtain a picture image, wherein the target image frame is the Nth preview image frame closest to the moment the picture-taking command is received, and N is a positive integer.

[0006] In one embodiment, converting the image coordinate system motion vector into a Hall coordinate system motion vector based on the transformation relationship between the image coordinate system and the Hall coordinate system includes: determining the sign value of the Hall value mapped from the image coordinate system to the Hall coordinate system, and using the first product between the sign value and the image coordinate system as the initial motion vector of the Hall coordinate system; adjusting the initial motion vector of the Hall coordinate system based on an adjustment parameter, wherein the adjustment parameter is the difference between the absolute value of the sign value and a constant term; and using the reverse motion vector of the adjusted initial motion vector of the Hall coordinate system as the motion vector of the Hall coordinate system.

[0007] In one embodiment, determining the Hall compensation value for motion compensation of the target subject based on the Hall coordinate system motion vector includes: determining the motion speed of the target subject in adjacent frames according to the calibration value, the frame interval between adjacent frames, and the Hall coordinate system motion vector, wherein the calibration value is the Hall quantity corresponding to each pixel size; and determining the ratio between the motion speed and the preset gain value as the Hall compensation value for motion compensation of the target subject.

[0008] In one embodiment, determining the motion velocity of the target subject in adjacent frames based on the calibration value, the frame interval of adjacent frames, and the Hall coordinate system motion vector includes: converting the Hall coordinate system motion vector into a Hall coordinate system motion vector in a corresponding Hall coordinate system motion vector measurement unit to obtain a unit-converted Hall coordinate system motion vector; determining a second product between the calibration value and the frame interval of adjacent frames; and determining the ratio between the unit-converted Hall coordinate system motion vector and the second product as the motion velocity of the target subject in adjacent frames.

[0009] In one embodiment, obtaining the motion vector of the target subject between adjacent frames in the preview image frame queue to obtain the motion vector of the image coordinate system includes: determining the region of interest (ROI) of the target subject in the preview image; determining the dense optical flow of each pixel in the ROI between adjacent frames in the preview image frame queue; and determining the motion vector of the image coordinate system based on the dense optical flow of each pixel in the ROI.

[0010] According to a second aspect of the present disclosure, a photographing device is provided, comprising: an acquisition unit, configured to, in response to the launch of a camera application, identify a target subject in a preview image and acquire motion vectors of the target subject between adjacent frames in a preview image frame queue, thereby obtaining an image coordinate system motion vector; a determination unit, configured to, based on a vector transformation relationship between the image coordinate system and the Hall coordinate system, convert the image coordinate system motion vector into a Hall coordinate system motion vector; and, based on the Hall coordinate system motion vector, determine a Hall compensation value for motion compensation of the target subject; and a processing unit, configured to, in response to receiving a photographing command, perform motion compensation on the target subject in a target image frame based on the Hall compensation value, thereby obtaining a photographed image, wherein the target image frame is the Nth preview image frame closest to the moment the photographing command is received, and N is a positive integer.

[0011] In one embodiment, the determining unit converts the image coordinate system motion vector into a Hall coordinate system motion vector based on the transformation relationship between the image coordinate system and the Hall coordinate system as follows: determining the sign value of the Hall value mapped from the image coordinate system to the Hall coordinate system, and using the first product between the sign value and the image coordinate system as the initial motion vector of the Hall coordinate system; adjusting the initial motion vector of the Hall coordinate system based on an adjustment parameter, wherein the adjustment parameter is the difference between the absolute value of the sign value and a constant term; and using the reverse motion vector of the adjusted initial motion vector of the Hall coordinate system as the motion vector of the Hall coordinate system.

[0012] In one embodiment, the determining unit determines the Hall compensation value for motion compensation of the target subject based on the Hall coordinate system motion vector in the following manner: determining the motion speed of the target subject in adjacent frames according to the calibration value, the frame interval of adjacent frames, and the Hall coordinate system motion vector, wherein the calibration value is the Hall quantity corresponding to each pixel size; and determining the ratio between the motion speed and the preset gain value as the Hall compensation value for motion compensation of the target subject.

[0013] In one embodiment, the determining unit determines the motion speed of the target subject in adjacent frames based on the calibration value, the frame interval of adjacent frames, and the Hall coordinate system motion vector as follows: converting the Hall coordinate system motion vector into a Hall coordinate system motion vector of the corresponding Hall coordinate system motion vector measurement unit to obtain a unit-converted Hall coordinate system motion vector; determining the second product between the calibration value and the frame interval of adjacent frames; and determining the ratio between the unit-converted Hall coordinate system motion vector and the second product as the motion speed of the target subject in adjacent frames.

[0014] In one embodiment, the acquisition unit acquires the motion vector of the target subject between adjacent frames in the preview image frame queue in the following manner to obtain the motion vector of the image coordinate system: determining the region of interest (ROI) of the target subject in the preview image; determining the dense optical flow of each pixel in the ROI between adjacent frames in the preview image frame queue; and determining the motion vector of the image coordinate system based on the dense optical flow of each pixel in the ROI.

[0015] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to perform the photographing method of the first aspect or any embodiment of the first aspect.

[0016] According to a fourth aspect of the present disclosure, a storage medium is provided, the storage medium storing instructions that, when executed by a processor, enable the execution of the photographing method of the first aspect or any embodiment of the first aspect.

[0017] The technical solution provided in this disclosure can include the following beneficial effects: During the shooting process, the image coordinate system motion vector of the target subject is determined based on the preview image, and the image coordinate system motion vector is converted into a Hall coordinate system motion vector, thereby realizing motion compensation based on the Hall coordinate system motion vector. This motion compensation method uses a relational transformation method for motion compensation, which can improve the compensation accuracy, achieve a good anti-shake effect, and thus display the image clearly.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0020] Figure 1 This is a flowchart illustrating a photographing method according to an exemplary embodiment.

[0021] Figure 2 This is a flowchart illustrating a method for determining the motion vector of a target subject in an image coordinate system, according to an exemplary embodiment.

[0022] Figure 3 This is a flowchart illustrating a method for determining the motion vector of a target subject in an image coordinate system, according to an exemplary embodiment.

[0023] Figure 4 This is a flowchart illustrating a method for determining optical flow information of a target subject according to an exemplary embodiment.

[0024] Figure 5 This is a flowchart illustrating a method for determining the motion vector of a target in a Hall coordinate system according to an exemplary embodiment.

[0025] Figure 6 This is a flowchart illustrating the determination of Hall compensation according to an exemplary embodiment.

[0026] Figure 7 This is a flowchart illustrating the determination of motion speed according to an exemplary embodiment.

[0027] Figure 8 This is a schematic diagram illustrating a photographic image according to an exemplary embodiment.

[0028] Figure 9 This is a schematic diagram illustrating a photographic imaging comparison according to an exemplary embodiment.

[0029] Figure 10 This is a block diagram of a photographing device according to an exemplary embodiment.

[0030] Figure 11 This is a block diagram illustrating a photographing device according to an exemplary embodiment.

[0031] Figure 12 This is a block diagram illustrating a photographing device according to an exemplary embodiment. Detailed Implementation

[0032] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure.

[0033] The photographing method provided in this disclosure is applicable to scenarios requiring snapshots or panning shots. The photographing method described in this disclosure is primarily applied to scenarios involving photographing moving objects.

[0034] Optical image stabilization (OIS) is a technology used to reduce image blur caused by camera movement. When a photographer holds the camera hand-held, even slight hand tremors can cause blurry photos, especially at slower shutter speeds or when using a telephoto lens. OIS compensates for these minute movements by detecting them and adjusting the lens position accordingly, thus maintaining image sharpness.

[0035] In related technologies, when taking pictures of a moving target using a stationary camera, the movement of the camera is usually detected by a Hall effect sensor. However, Hall effect sensors are expensive and may not be available in some cases.

[0036] In view of this, this disclosure provides a photographing method, which is mainly used to improve the image quality of moving subjects in photographing scenarios that require capturing and panning. By actively tracking the target subject and accurately estimating its motion, the OIS hardware is driven to perform real-time subject tracking, thereby ensuring the clarity of the target subject in the captured photo. At the same time, the background other than the target subject is inevitably blurred, thus achieving an effect similar to panning.

[0037] Figure 1 This is a flowchart illustrating a photographing method according to an exemplary embodiment, such as... Figure 1 As shown, it includes the following steps.

[0038] In step S11, in response to the camera application being launched, the target subject in the preview image is identified, and the motion vector of the target subject between adjacent frames in the preview image frame queue is obtained to obtain the motion vector of the image coordinate system.

[0039] In this embodiment of the disclosure, in response to the startup of the camera application, the target subject in the preview image is identified, and the motion vector of the target subject in the image coordinate system in the adjacent frames in the preview image frame queue is determined.

[0040] The preview image frame queue can be part of a video stream or multiple still images obtained through rapid burst shooting. The target subject can be understood as the subject that needs to be sharpened during the shooting process. For example, the target subject could be a subject that needs sharpening due to hand tremors or its own movement. Motion vectors can be understood as horizontal motion vectors, vertical motion vectors, or both horizontal and vertical motion vectors.

[0041] In step S12, based on the vector transformation relationship between the image coordinate system and the Hall coordinate system, the motion vector of the image coordinate system is converted into the motion vector of the Hall coordinate system.

[0042] In this embodiment of the disclosure, the motion vector in the image coordinate system is transformed into the target motion vector in the Hall coordinate system.

[0043] In step S13, the Hall compensation value for motion compensation of the target body is determined based on the motion vector of the Hall coordinate system.

[0044] In this embodiment of the disclosure, the Hall compensation value at the target time is determined based on the target motion vector in the obtained Hall coordinate system.

[0045] In step S14, in response to receiving the photo-taking command, motion compensation is performed on the target subject in the target image frame based on the Hall compensation value to obtain the photo image.

[0046] The target image frame is the Nth preview image frame closest to the moment the photo capture command was received, where N is a positive integer.

[0047] According to exemplary embodiments of this disclosure, by calculating the motion vector of the target subject in the image coordinate system and performing motion compensation, image blurring caused by shaking and other factors is reduced, thereby improving image quality. By identifying the target subject in the preview image and calculating its motion vector, the movement of the target subject can be accurately captured. Based on the vector transformation relationship between the image coordinate system and the Hall coordinate system, the motion vector is converted into a Hall coordinate system motion vector, and the Hall compensation value is further determined, achieving precise motion compensation for the target subject. This effectively reduces image blurring caused by hand shaking and other factors, significantly improving image quality. The identification of the target subject and the calculation of the motion vector begin when the camera application starts, ensuring that motion compensation is ready when the shooting command is issued, improving real-time response capabilities.

[0048] In this embodiment of the disclosure, when the target subject is displaced in both the horizontal and vertical directions, the Hall compensation value required for the target subject at the moment of taking a picture is determined, thereby enabling the OIS hardware to compensate according to the image motion and achieve optical image stabilization.

[0049] In exemplary embodiments of this disclosure, the following methods may be employed: Figure 2 The motion vectors of the target subject between adjacent frames in the preview image frame queue are obtained in the manner shown, thus obtaining the motion vectors of the image coordinate system. Figure 2 This is a flowchart illustrating a method for determining the motion vector of a target subject in an image coordinate system according to an exemplary embodiment, including the following steps.

[0050] In step S21, the region of interest (ROI) of the target subject in the preview image is determined.

[0051] In this embodiment of the disclosure, for example, a target detection algorithm can be used to determine the optical flow information of the ROI where the target subject is located.

[0052] In step S22, the dense optical flow of each pixel in the ROI between adjacent frames in the preview image frame queue is determined.

[0053] In step S23, the motion vector of the image coordinate system is determined based on the dense optical flow of each pixel in the ROI.

[0054] In this embodiment of the disclosure, for example, the following can be used: Figure 3 The method shown obtains the preview image frame queue before the target time and determines the motion vector of the target subject in the image coordinate system in the preview image frame queue. Figure 3 This is a flowchart illustrating a method for determining the motion vector of a target subject in an image coordinate system according to an exemplary embodiment, including the following steps.

[0055] In step S31, optical flow information of the area where the target subject is located is determined based on the preview image frame queue.

[0056] In this embodiment of the disclosure, optical flow information of the area where the target subject is located can be determined based on the preview image frame queue, for example, by using a target detection algorithm.

[0057] Among them, optical flow information represents the motion of each pixel in the target attention region in consecutive frames.

[0058] In step S32, the mean value of the optical flow information of the region where the target subject is located is determined as the motion vector of the target subject in the image coordinate system in the preview image frame queue.

[0059] According to exemplary embodiments of this disclosure, the target detection algorithm can accurately locate the target subject, ensuring that subsequent optical flow calculations are performed within the region where the target subject is located, thus improving calculation accuracy. Since optical flow information characterizes the motion of each pixel in the region where the target subject is located, it can capture the local motion of the target subject, avoiding interference from global motion. Furthermore, the optical flow algorithm can quickly calculate the optical flow vector between consecutive frames, making it suitable for real-time applications. By calculating the average optical flow, the noise impact of individual pixels can be reduced, improving the stability of the motion vector. Since the average optical flow characterizes the overall motion of the region where the target subject is located, it can more accurately reflect the motion of the target subject. Using the mean of the optical flow information as the motion vector simplifies subsequent calculation steps and improves processing efficiency.

[0060] In one exemplary embodiment, a tracking algorithm can be used to track information such as the position and motion of a target subject in consecutive frames. Since different tracking algorithms have different characteristics and applicable scenarios, this disclosure allows for flexible selection of a tracking algorithm suitable for the current scenario, or it can pre-define a tracking algorithm.

[0061] In this embodiment, the motion vector of each pixel at each pixel location in the target subject within the preview image frame queue can be tracked using different tracking algorithms. For example, a Kalman filter, suitable for linear systems and capable of effectively handling noise and uncertainty, can be used. Its working principle is as follows: State prediction: Predict the state of the current frame based on the state of the previous frame; Measurement update: Update the predicted state based on the actual observations of the current frame to obtain the final estimate. The execution steps can be: Initialization: Set initial state vectors (e.g., position and velocity) and covariance matrix; Prediction: Predict the state of the next frame using the state transition matrix; Update: Update the predicted state based on observations (e.g., the target's position); Repetition: Repeat the prediction and update steps in each frame. Alternatively, a simple and easy-to-implement non-parametric feature space analysis algorithm (Mean-Shift) can be used. Its implementation principle is as follows: Density estimation: Find the density peak of the target region through kernel density estimation (KDE); Iterative update: Continuously move the window, gradually shifting its center towards the density peak until convergence. The steps are as follows: Initialization: Select an initial window to cover the target area; Density estimation: Calculate the density of each pixel within the window; Window movement: Move the center of the window to the position with the highest density; Repetition: Repeat density estimation and window movement in each frame until the center of the window no longer moves or the maximum number of iterations is reached, etc. The above tracking method is an exemplary method involved in this disclosure, and this disclosure does not limit the method of tracking the target subject.

[0062] In this embodiment of the disclosure, by tracking the target subject, the position and movement of the target subject in consecutive frames can be effectively tracked, thereby improving the accuracy and stability of the tracking.

[0063] In one exemplary embodiment, parameters of the target tracking bounding box are obtained. These parameters typically include the pixel position (top-left corner coordinates) of the target tracking bounding box containing the target object, its width, height, and a flag indicating whether the target object was successfully detected.

[0064] In this embodiment of the disclosure, the specific parameters of the target tracking box can be obtained, for example, in the following manner. First, the initial position of the target in the image is determined using a target detection algorithm. After target detection is completed, the target tracking box can be initialized. For example, the following steps can be used: Obtain detection results: Obtain the bounding box information of the target from the target detection algorithm, including the top-left corner coordinates (x, y), width w, and height h. Set initial parameters: Top-left corner coordinates: (x, y); Width: w; Height: h; Flag: Set a flag to indicate whether the target body was successfully detected. Typically, if the target is detected, the flag is True; otherwise, it is False. After initializing the target tracking box, use the target tracking algorithm to continuously track the target in subsequent frames. In each frame, update the parameters of the target tracking box based on the results of the tracking algorithm. The following steps can be used: Obtain the new position of the target body in the current frame. Update the original top-left corner coordinates to the new top-left corner coordinates (x′, y′). Update to the new width w′ (if the tracking algorithm supports width variation). Update to the new height h′ (if the tracking algorithm supports height variation). Flag: Update the flag based on the tracking results. If tracking fails, the flag is set to False.

[0065] In this embodiment of the disclosure, the following methods can be used: Figure 4 The method shown determines the optical flow information of the area where the target subject is located based on the preview image frame queue. Figure 4 This is a flowchart illustrating a method for determining optical flow information of a target subject according to an exemplary embodiment, including the following steps.

[0066] In step S41, global optical flow information is determined based on the preview image frame queue.

[0067] In this embodiment of the disclosure, for example, an optical flow algorithm is used to calculate the optical flow vector in the entire image frame based on the preview image frame queue.

[0068] Among them, global optical flow information represents the motion of all pixels in the preview image frame queue in consecutive frames.

[0069] In step S42, the pixel position of the target subject in the preview image frame queue is identified, and the region where the target subject is located is determined based on the pixel position.

[0070] The pixel position of the target subject in the preview image frame queue can be understood as the location of the target tracking box containing the target subject.

[0071] In step S43, optical flow information of the target subject's region is selected from the global optical flow information based on the region where the target subject is located.

[0072] In step S44, the boundary optical flow points of the optical flow information of the target subject area are removed using a preset format to determine the optical flow information of the target subject area.

[0073] In this embodiment of the disclosure, optical flow points at the edges of the region where the target subject is located are removed using a preset format. For example, optical flow points of 1-2 pixels at the edges can be removed to reduce the impact of boundary effects. After removing the boundary optical flow points, the final optical flow information of the region where the target subject is located is determined.

[0074] According to exemplary embodiments of this disclosure, by calculating global optical flow information, the motion of all pixels in an image can be captured, ensuring no omissions. Removing boundary optical flow points can reduce the impact of boundary effects and improve the reliability of optical flow information. Boundary optical flow points are often significantly affected by environmental noise; removing these points can improve the stability of optical flow information.

[0075] In this embodiment, the target tracking bounding box containing the detected target subject and the entire image are used as input. Dense optical flow is calculated on the preceding and following frames in the preview image frame queue to obtain global optical flow information. Based on the global optical flow information, the motion vector of each pixel in the image sequence is estimated. The motion of the target subject between different frames is determined, thereby achieving more accurate tracking and motion compensation.

[0076] In an exemplary embodiment, global optical flow information can be calculated as follows: First, a target object is detected using a target detection algorithm, and its target tracking box in the current frame is obtained. This target tracking box includes the top-left corner coordinates (x, y), width w, and height h. The detected target tracking box and the image of the current frame are used as input to calculate the optical flow. For example, the current frame It and the previous frame It-1 are used to calculate the optical flow. The steps for calculating the optical flow can be as follows: First, the image is preprocessed, converting the current frame It and the previous frame It-1 into grayscale images to reduce computational complexity. The image can also be downsampled to speed up the calculation. The preprocessed image is then used to calculate the dense optical flow between each pair of consecutive frames using an optical flow algorithm. Optical flow information within the target tracking box region is extracted from the calculated global dense optical flow. Global optical flow information is obtained, which contains the motion vector of each pixel in the entire image.

[0077] Dense optical flow (DOF) refers to calculating the motion vector of each pixel in an image between two consecutive frames. Unlike sparse optical flow (SOF), dense optical flow provides motion information for each pixel, thus reflecting the motion of objects in the image more comprehensively. This disclosure of optical flow algorithms is not restrictive.

[0078] In this embodiment, the Region of Interest (ROI) is extracted from the dense optical flow and further processed to more accurately estimate the motion of the target subject. For example, the motion vector of each pixel can be obtained by calculating the dense optical flow between consecutive frames. The dense optical flow provides motion information for each pixel, including horizontal and vertical motion components. The ROI containing the target subject is extracted from the calculated global optical flow. To reduce the influence of boundary noise, a portion of the dense points at the edge of the ROI is typically removed. For example, the outermost 5% of the pixels of the ROI are removed. To further simplify the calculation, the ROI with the boundary removed can be divided into a 5x5 grid. The optical flow information of each grid cell can be obtained by calculating the average of the optical flow vectors of all pixels within that cell.

[0079] The region of interest can also be understood as the area containing the target subject within the target tracking frame as disclosed in this disclosure.

[0080] In this embodiment of the disclosure, based on the transformation relationship between the image coordinate system and the Hall coordinate system, the motion vector of the image coordinate system is converted into a motion vector of the Hall coordinate system. This is typically achieved using methods such as... Figure 5 As shown in the diagram. Figure 5 This is a flowchart illustrating a method for determining a target motion vector in a Hall coordinate system according to an exemplary embodiment, including the following steps.

[0081] In step S51, the sign value of the Hall value mapped from the image coordinate system to the Hall coordinate system is determined, and the first product between the sign value and the image coordinate system is used as the initial motion vector of the Hall coordinate system.

[0082] In step S52, the initial motion vector of the Hall coordinate system is adjusted based on the adjustment parameter, which is the difference between the absolute value of the sign value and the constant term.

[0083] In this embodiment of the disclosure, the initial motion vector of the Hall coordinate system is described, for example, in the following manner.

[0084] mv'' x =mv′ x [|O HallXonCam |-1]*(-1)

[0085] mv'' y =mv′ y [|O HallYonCam |-1]*(-1)

[0086] Among them, mv′ x Let mv' be the horizontal motion vector in the Hall coordinate system. y Let mv' be the vertical motion vector in the Hall coordinate system. The resulting mv'... xand mv′ y Perform a reverse calculation to adjust the motion vector of the Hall coordinate system to conform to the installation method of the Hall sensor.

[0087] |O HallXonCam | Indicates the horizontal mounting method of the Hall sensor. |O HallYonCam | Indicates the vertical mounting method of the Hall sensor. |O HallXonCam |-1 and |O HallyonCam -1 is used to adjust the magnitude of the motion. (-1) is used to reverse the operation, ensuring that the direction of the motion vector is correct.

[0088] In step S53, the reverse motion vector of the adjusted initial motion vector of the Hall coordinate system is used as the motion vector of the Hall coordinate system.

[0089] According to an exemplary embodiment of this disclosure, based on the transformation relationship, a motion vector can be transformed from the image coordinate system to the Hall coordinate system through sign adjustment and inverse operation, ensuring the accuracy of the direction and magnitude of the motion vector in the Hall coordinate system.

[0090] In this embodiment of the disclosure, the motion vector is reverse-calculated according to the specific installation method of the Hall sensor to determine the motion vector of the Hall coordinate system.

[0091] According to an exemplary embodiment of this disclosure, based on the transformation relationship, a motion vector can be transformed from the image coordinate system to the Hall coordinate system through sign adjustment and inverse operation, ensuring the accuracy of the direction and magnitude of the motion vector in the Hall coordinate system.

[0092] An exemplary embodiment is further illustrated by the transformation relationship between the horizontal and vertical directions of the image coordinate system and the Hall coordinate system.

[0093] In this embodiment of the disclosure, there is a fixed transformation relationship between the Hall coordinate system and the image pixel coordinates.

[0094] In the OIS system, the Hall sensor is used to detect minute movements of the lens. The value output by the sensor is called the Hall code, usually represented as orgHall = {orgHallX, orgHallY}. Pixel coordinates in the image are represented as (C... x C y (), used to describe the location in the image.

[0095] Assume there is a fixed transformation relationship between hall and pixel coordinates, denoted as:

[0096] O h2c ={O HallXonCam O HallYonCam},Ox =±1, ±2

[0097] Among them, O HallXonCam and O HallYonCam This represents the mapping relationship of hall codes in the image coordinate system, O x =±1, ±2 represent the sign and scaling factor of the hall code.

[0098] For the input orgHall = {orgHallX, orgHallY}, it needs to be converted into image pixel coordinates (C x C y This is done to cancel out the motion in the image. The conversion formula is as follows:

[0099] C x =orgHallX[|O HallXonCam |-1]*sign(O HallXonCam )

[0100] C y =orgHallY[|O HallYonCam |-1]*sign(O HallYonCam )

[0101] Where orgHallX and orgHallY are the hall codes on the X and Y axes, respectively.

[0102] |O HallXonCam | and | O HallYoncam |: These are O HallXonCam and O HallYonCam The absolute value of.

[0103] sign(O HallXonCam ) and sign(O HallYoncam ): These are O HallXonCam and O HallYonCam The sign function returns 1 or -1.

[0104] In one exemplary embodiment, for example, the input hall code is orgHall = {5, -3}, and its transformation relationship is O. h2c ={2,-1}.

[0105] Calculate the absolute value: |O HallXonCam |=|2|=2;|O HallYonCam |=|-1|=1

[0106] Calculation symbol: sign(O) HallXonCam ) = 1; sign(O HallYonCam ) = -1

[0107] Convert to pixel coordinates:

[0108] C x =5*(2-1)*1=5

[0109] C y =-3*(1-1)*(-1)=0

[0110] Therefore, the transformed pixel coordinates are (C x C y ) = (5,0).

[0111] In this embodiment of the disclosure, the Hall compensation for motion compensation of the target subject is determined based on the motion vector of the Hall coordinate system, which can be achieved by means of, for example... Figure 6 As shown in the diagram. Figure 6 This is a flowchart illustrating the determination of Hall compensation according to an exemplary embodiment, including the following steps.

[0112] In step S61, the motion speed of the target body in adjacent frames is determined based on the calibration value, the frame interval between adjacent frames, and the motion vector of the Hall coordinate system.

[0113] The calibration value is the Hall effect value corresponding to each pixel size. In a laboratory environment, the conversion relationship between pixel units and Hall units is determined by using a known physical movement distance and the corresponding pixel displacement. For example, the calibration value can be calculated by moving the lens or sensor a certain distance, recording the corresponding pixel displacement, and then calculating the calibration value.

[0114] In step S62, the ratio between the motion speed and the preset gain value is determined as the Hall compensation value for motion compensation of the target body.

[0115] According to an exemplary embodiment of this disclosure, the Hall motion velocity can be accurately converted into a Hall compensation value through a calibration process, ensuring the accuracy of motion compensation. The gain value can be adjusted according to the specific conditions of different devices, improving versatility and adaptability. By calculating the Hall compensation value, the motion compensation of the OIS system can be precisely controlled, ensuring image stability. The calculation of the Hall compensation value takes into account changes in the Hall motion velocity, enabling dynamic adjustment of motion compensation to adapt to different shooting scenarios.

[0116] In one exemplary embodiment, a gain value is determined for converting the Hall motion velocity into a gain value. The gain value is used to determine the gain in the horizontal and vertical directions. This gain is transmitted to the OIS hardware at the frame rate, representing the desired motion direction and velocity in the OIS, to obtain a Hall compensation value, thereby counteracting hand motion. Without changing the original OIS calculation logic, the motion vector is processed with a gain value to obtain the Hall velocity corresponding to each degree per second. The determination method is as follows:

[0117]

[0118] Among them, gyro fx This represents the Hall compensation value in the horizontal direction;

[0119] gyro fy This represents the Hall compensation value in the vertical direction.

[0120] In this embodiment of the disclosure, the calculated Hall compensation value is transmitted to the OIS hardware, which then adjusts the position of the lens based on these signals to achieve motion compensation.

[0121] In this embodiment of the disclosure, the motion velocity of the target subject in adjacent frames is determined based on the calibration value, the frame interval between adjacent frames, and the motion vector in the Hall coordinate system. This can be achieved using methods such as... Figure 7 As shown, Figure 7 This is a flowchart illustrating a method for determining motion speed according to an exemplary embodiment, including the following steps.

[0122] In step S71, the motion vector in the Hall coordinate system is converted into a motion vector in the Hall coordinate system with the corresponding unit of measurement, thus obtaining the motion vector in the Hall coordinate system after unit conversion.

[0123] In step S72, a second product between the calibration value and the frame interval of adjacent frames is determined.

[0124] In step S73, the ratio between the unit-converted Hall coordinate system motion vector and the second product is determined as the motion velocity of the target subject in the adjacent frame.

[0125] According to exemplary embodiments of this disclosure, a calibration process accurately converts pixel-level motion vectors into Hall-level motion vectors, ensuring the accuracy of motion compensation. Recording and calculating time intervals ensures real-time motion compensation, responding promptly to camera shake. Calculating Hall motion velocity allows for precise control of the OIS system's motion compensation, ensuring image stability. Furthermore, since the calculation of Hall motion velocity considers changes in time intervals, motion compensation is dynamically adjusted to adapt to different shooting scenarios.

[0126] In one exemplary embodiment, the average optical flow within the region where the target subject is located is calculated using optical flow methods or other motion estimation algorithms to obtain the motion vector:

[0127] mv = {mv x ,mv y}

[0128] The motion vectors in the acquired image coordinate system are converted into motion vectors in the Hall coordinate system according to the installation position and orientation of the Hall sensor. The conversion method is as follows:

[0129] mv′ x =mv x *sign(O HallXonCam )

[0130] mv′ y =mv y *sign(O HallYonCam )

[0131] Among them, mv′ x Let mv' be the horizontal motion vector in the Hall coordinate system. y This is the motion vector in the vertical direction in the Hall coordinate system.

[0132] The obtained mv′ x and mv′ y Perform a reverse calculation to adjust the motion vector of the Hall coordinate system to match the installation method of the Hall sensor. The adjustment method is as follows:

[0133] mv' x ’ =mv′ x [|O HallXonCam |-1]*(-1)

[0134] mv' y ’ =mv′ y [|O HallyonCam |-1]*(-1)

[0135] Among them, |O HallXonCam | Indicates the horizontal mounting method of the Hall sensor. |O HallYonCam | indicates the vertical mounting method of the Hall sensor.

[0136] |O HallXonCam |-1 and |O HallyonCam -1 is used to adjust the magnitude of the motion. (-1) is used to reverse the operation, ensuring that the direction of the motion vector is correct.

[0137] Using the calibration values, the obtained motion vector is converted from pixel units to Hall unit calibration values ​​(pixelPerHallX), and the motion speed corresponding to the Hall sensor is calculated.

[0138]

[0139] Where pixelPerHallX represents the number of pixels in the horizontal direction corresponding to each unit of hall code.

[0140] pixelPerHallY: Represents the number of pixels per hall code in the vertical direction.

[0141] T frameIntervel : Represents the time interval in consecutive image frames.

[0142] Using the methods described in this disclosure, the motion vector mv = {mv} in the image coordinate system can be... x ,mv y The input is converted into Hall effect compensation values ​​and passed to the OIS hardware. This allows the OIS hardware to simulate the input of a real gyroscope even when there is no gyroscope in the Hall effect component, enabling the OIS hardware to compensate for image motion and thus improve image stability and clarity.

[0143] In this embodiment of the disclosure, since the tracked target is the target in the image coordinate system, the position of the target in the Hall coordinate system is determined by using the transformation relationship between the Hall coordinate system and the image pixel coordinate system, and then converted into a Hall compensation value (fake gyro). The converted Hall compensation value is then passed to the OIS hardware, thereby simulating the input of a real gyroscope in the absence of a real gyroscope, enabling the OIS hardware to compensate for image motion and achieve optical image stabilization.

[0144] In one exemplary embodiment, the following is employed: Figure 8 The method of taking pictures as shown is explained in this disclosure. Figure 8 This is a schematic diagram illustrating a photographic image according to an exemplary embodiment, such as... Figure 8 As shown.

[0145] In this embodiment, two consecutive image frames (image frame 1 and image frame 2) prior to the capture command are acquired. A target tracking box is detected in these two consecutive image frames, and the coordinates of the target subject are determined, thereby determining the optical flow value of the target subject. Based on the optical flow value of the target subject, the corresponding motion vector of the target subject is determined, i.e., the motion vector is determined, and the Hall compensation value is calculated. At this time, image frame i at the moment the capture command is determined is acquired. The calculated Hall compensation value is used to simulate the speed of a virtual gyroscope and input into the OIS to drive the lens, obtaining the processed original image. Image frame i+1 is processed in the same way. The resulting original images corresponding to different image frames are then fused to obtain a final image.

[0146] According to exemplary embodiments of this disclosure, as Figure 9 The diagram shown further illustrates the comparison of photographic images. Figure 9 This is a schematic diagram illustrating a photographic imaging comparison according to an exemplary embodiment.

[0147] exist Figure 9In general photography, the camera remains stationary. While this allows for clear imaging of static objects on the timeline, it cannot guarantee the stability of the exposure time for moving subjects. Consequently, the raw image (RAW) often exhibits blurry subjects. Furthermore, post-processing techniques such as multi-frame fusion are used to obtain the final image from a stationary camera. Superimposing multiple RAW frames further increases the blurriness of the subject, making the final image even more blurry. However, according to the embodiments of this disclosure, by tracking the target subject and obtaining accurate optical flow information based on optical panning, the current motion speed of the target subject is calculated using this optical flow information. OIS compensation ensures that the target subject can be tracked in every RAW frame, resulting in a clear subject and excellent image quality in the final superimposed image.

[0148] Based on the same concept, this disclosure also provides a photographing device.

[0149] It is understood that the photographing device provided in this disclosure includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. In conjunction with the units and algorithm steps of the various examples disclosed in this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of this disclosure.

[0150] Figure 10 This is a block diagram illustrating a photographing device according to an exemplary embodiment. (Refer to...) Figure 10 The device 100 includes an acquisition unit 101, a determination unit 102, and a processing unit 103.

[0151] The acquisition unit 101 is used to identify the target subject in the preview image in response to the camera application startup, and to acquire the motion vector of the target subject between adjacent frames in the preview image frame queue, thereby obtaining the motion vector of the image coordinate system.

[0152] The determining unit 102 is used to convert the motion vector of the image coordinate system into the motion vector of the Hall coordinate system based on the vector transformation relationship between the image coordinate system and the Hall coordinate system; and to determine the Hall compensation value for motion compensation of the target subject based on the motion vector of the Hall coordinate system.

[0153] The processing unit 103 is used to respond to receiving a photo capture command, perform motion compensation on the target subject in the target image frame based on the Hall compensation value, and obtain a photo capture image. The target image frame is the Nth preview image frame closest to the time when the photo capture command is received, where N is a positive integer.

[0154] In one embodiment, the determining unit 102 converts the image coordinate system motion vector into a Hall coordinate system motion vector based on the transformation relationship between the image coordinate system and the Hall coordinate system as follows: determining the sign value of the Hall value mapped from the image coordinate system to the Hall coordinate system, and using the first product between the sign value and the image coordinate system as the initial motion vector of the Hall coordinate system; adjusting the initial motion vector of the Hall coordinate system based on the adjustment parameter, which is the difference between the absolute value of the sign value and the constant term; and using the reverse motion vector of the adjusted initial motion vector of the Hall coordinate system as the motion vector of the Hall coordinate system.

[0155] In one embodiment, the determining unit 102 determines the Hall compensation value for motion compensation of the target subject based on the Hall coordinate system motion vector in the following manner: the motion speed of the target subject in the adjacent frame is determined according to the calibration value, the frame interval of the adjacent frame, and the Hall coordinate system motion vector, where the calibration value is the Hall quantity corresponding to each pixel size; the ratio between the motion speed and the preset gain value is determined as the Hall compensation value for motion compensation of the target subject.

[0156] In one embodiment, the determining unit 102 determines the motion speed of the target subject in adjacent frames based on the calibration value, the frame interval of adjacent frames, and the motion vector in the Hall coordinate system as follows: the motion vector in the Hall coordinate system is converted into a Hall coordinate system motion vector in the corresponding Hall coordinate system motion vector measurement unit to obtain the unit-converted Hall coordinate system motion vector; the second product between the calibration value and the frame interval of adjacent frames is determined; and the ratio between the unit-converted Hall coordinate system motion vector and the second product is determined as the motion speed of the target subject in adjacent frames.

[0157] In one embodiment, the acquisition unit 101 acquires the motion vector of the target subject between adjacent frames in the preview image frame queue in the following manner to obtain the motion vector of the image coordinate system: determining the ROI of the target subject in the preview image; determining the dense optical flow of each pixel in the ROI between adjacent frames in the preview image frame queue; and determining the motion vector of the image coordinate system based on the dense optical flow of each pixel in the ROI.

[0158] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0159] Figure 11This is a block diagram illustrating a photographing device 200 according to an exemplary embodiment. For example, device 200 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0160] Reference Figure 11 The device 200 may include one or more of the following components: processing component 202, memory 204, power component 206, multimedia component 208, audio component 210, input / output (I / O) interface 212, sensor component 214, and communication component 216.

[0161] Processing component 202 typically controls the overall operation of device 200, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 202 may include one or more modules to facilitate interaction between processing component 202 and other components. For example, processing component 202 may include a multimedia module to facilitate interaction between multimedia component 208 and processing component 202.

[0162] Memory 204 is configured to store various types of data to support the operation of device 200. Examples of such data include instructions for any application or method operating on device 200, contact data, phonebook data, messages, pictures, videos, etc. Memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0163] The power supply component 206 provides power to the various components of the device 200. The power supply component 206 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 200.

[0164] Multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 208 includes a front-facing camera and / or a rear-facing camera. When the device 200 is in an operating mode, such as a photo mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0165] Audio component 210 is configured to output and / or input audio signals. For example, audio component 210 includes a microphone (MIC) configured to receive external audio signals when device 200 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 204 or transmitted via communication component 216. In some embodiments, audio component 210 also includes a speaker for outputting audio signals.

[0166] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0167] Sensor assembly 214 includes one or more sensors for providing status assessments of various aspects of device 200. For example, sensor assembly 214 may detect the on / off state of device 200, the relative positioning of components such as the display and keypad of device 200, changes in the position of device 200 or a component of device 200, the presence or absence of user contact with device 200, the orientation or acceleration / deceleration of device 200, and temperature changes of device 200. Sensor assembly 214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 214 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 214 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0168] Communication component 216 is configured to facilitate wired or wireless communication between device 200 and other devices. Device 200 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 216 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 216 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0169] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0170] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, which can be executed by a processor 220 of the device 200 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0171] Figure 12 This is a block diagram illustrating a photographing device 300 according to an exemplary embodiment. For example, device 300 may be provided as a server. (Refer to...) Figure 12 The device 300 includes a processing component 322, which further includes one or more processors, and memory resources represented by memory 332 for storing instructions, such as application programs, that can be executed by the processing component 322. The application programs stored in memory 332 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 322 is configured to execute instructions to perform the methods described above.

[0172] Device 300 may also include a power supply component 326 configured to perform power management of device 300, a wired or wireless network interface 350 configured to connect device 300 to a network, and an input / output (I / O) interface 352. Device 300 may operate on an operating system stored in memory 332, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0173] It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0174] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0175] It is further understood that the terms “center,” “longitudinal,” “lateral,” “front,” “rear,” “up,” “down,” “left,” “right,” “vertical,” “horizontal,” “top,” “bottom,” “inner,” and “outer,” etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation.

[0176] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.

[0177] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0178] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0179] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A photographing method, characterized by, The method includes: In response to the camera application launch, the target subject in the preview image is identified, and the motion vector of the target subject between adjacent frames in the preview image frame queue is obtained to obtain the motion vector of the image coordinate system; Based on the vector transformation relationship between the image coordinate system and the Hall coordinate system, the motion vector of the image coordinate system is converted into the motion vector of the Hall coordinate system; Based on the motion vector of the Hall coordinate system, determine the Hall compensation value for motion compensation of the target body; In response to receiving a photo capture command, motion compensation is performed on the target subject in the target image frame based on the Hall compensation value to obtain a captured image. The target image frame is the Nth preview image frame closest to the moment the photo capture command is received, where N is a positive integer.

2. The method of claim 1, wherein, The process of converting the image coordinate system motion vector into a Hall coordinate system motion vector based on the transformation relationship between the image coordinate system and the Hall coordinate system includes: Determine the sign value of the Hall value mapped from the image coordinate system to the Hall coordinate system, and use the first product between the sign value and the image coordinate system as the initial motion vector of the Hall coordinate system; Based on the adjustment parameters, the initial motion vector of the Hall coordinate system is adjusted, wherein the adjustment parameters are the difference between the absolute value of the sign value and the constant term; The reverse motion vector of the adjusted initial motion vector in the Hall coordinate system is used as the motion vector in the Hall coordinate system.

3. The method according to claim 1 or 2, characterized in that, The step of determining the Hall compensation value for motion compensation of the target subject based on the motion vector of the Hall coordinate system includes: The motion speed of the target body in adjacent frames is determined based on the calibration value, the frame interval between adjacent frames, and the motion vector of the Hall coordinate system. The calibration value is the Hall quantity corresponding to each pixel size. The ratio between the motion speed and the preset gain value is determined as the Hall compensation value for motion compensation of the target subject.

4. The method according to claim 3, characterized in that, Determining the target's motion velocity in adjacent frames based on calibration values, the frame interval between adjacent frames, and the motion vector in the Hall coordinate system includes: The motion vector in the Hall coordinate system is converted into a Hall coordinate system motion vector in the corresponding Hall coordinate system motion vector measurement unit to obtain the unit-converted Hall coordinate system motion vector. Determine the second product between the calibration value and the frame interval of the adjacent frames; The ratio between the unit-converted Hall coordinate system motion vector and the second product is determined as the motion velocity of the target subject in adjacent frames.

5. The method according to claim 1, characterized in that, The step of obtaining the motion vector of the target subject between adjacent frames in the preview image frame queue to obtain the motion vector of the image coordinate system includes: Determine the region of interest (ROI) of the target subject in the preview image; Determine the dense optical flow of each pixel in the region of interest (ROI) between adjacent frames in the preview image frame queue; Based on the dense optical flow of each pixel in the region of interest (ROI), the motion vector of the image coordinate system is determined.

6. A photographing device, characterized in that, The device includes: The acquisition unit is used to identify the target subject in the preview image in response to the camera application startup, and to acquire the motion vector of the target subject between adjacent frames in the preview image frame queue, so as to obtain the motion vector of the image coordinate system. The determining unit is used to convert the motion vector of the image coordinate system into a motion vector of the Hall coordinate system based on the vector transformation relationship between the image coordinate system and the Hall coordinate system; and to determine the Hall compensation value for motion compensation of the target subject based on the motion vector of the Hall coordinate system. The processing unit is configured to, in response to receiving a photo-taking command, perform motion compensation on the target subject in the target image frame based on the Hall compensation value to obtain a photo image, wherein the target image frame is the Nth preview image frame closest to the moment the photo-taking command is received, and N is a positive integer.

7. The apparatus according to claim 6, characterized in that, The determining unit converts the image coordinate system motion vector into a Hall coordinate system motion vector based on the transformation relationship between the image coordinate system and the Hall coordinate system: Determine the sign value of the Hall value mapped from the image coordinate system to the Hall coordinate system, and use the first product between the sign value and the image coordinate system as the initial motion vector of the Hall coordinate system; Based on the adjustment parameters, the initial motion vector of the Hall coordinate system is adjusted, wherein the adjustment parameters are the difference between the absolute value of the sign value and the constant term; The reverse motion vector of the adjusted initial motion vector in the Hall coordinate system is used as the motion vector in the Hall coordinate system.

8. The apparatus according to claim 6 or 7, characterized in that, The determining unit determines the Hall compensation value for motion compensation of the target subject based on the motion vector of the Hall coordinate system in the following manner: The motion speed of the target body in adjacent frames is determined based on the calibration value, the frame interval between adjacent frames, and the motion vector of the Hall coordinate system. The calibration value is the Hall quantity corresponding to each pixel size. The ratio between the motion speed and the preset gain value is determined as the Hall compensation value for motion compensation of the target subject.

9. The apparatus according to claim 8, characterized in that, The determining unit determines the target's motion velocity in adjacent frames based on the calibration value, the frame interval between adjacent frames, and the motion vector in the Hall coordinate system as follows: The motion vector in the Hall coordinate system is converted into a Hall coordinate system motion vector in the corresponding Hall coordinate system motion vector measurement unit to obtain the unit-converted Hall coordinate system motion vector. Determine the second product between the calibration value and the frame interval of the adjacent frames; The ratio between the unit-converted Hall coordinate system motion vector and the second product is determined as the motion velocity of the target subject in adjacent frames.

10. The apparatus according to claim 6, characterized in that, The acquisition unit obtains the motion vector of the target subject between adjacent frames in the preview image frame queue in the following manner, thus obtaining the motion vector in the image coordinate system: Determine the region of interest (ROI) of the target subject in the preview image; Determine the dense optical flow of each pixel in the region of interest (ROI) between adjacent frames in the preview image frame queue; Based on the dense optical flow of each pixel in the region of interest (ROI), the motion vector of the image coordinate system is determined.

11. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the method described in any one of claims 1-5.

12. A storage medium, characterized in that, The storage medium stores instructions that, when executed by a processor, enable the execution of the method described in any one of claims 1-5.

Images