Position calibration method and device

Abstract

The invention discloses a position calibration method and device, and belongs to the technical field of electronic equipment. The position calibration method comprises the following steps: determining first position information of a target object in a preview stream image; wherein the target object comprises one or more objects; according to the first position information, a target position calibration mode is adopted to calibrate the position of the target object in the video stream image, and second position information of the target object in the video stream image is obtained; wherein the target position calibration mode comprises at least one of the following items: a frame rate calibration mode, a motion state calibration mode, a zoom state calibration mode and an output specification calibration mode.

Description

Technical Field

[0001] This application belongs to the field of electronic equipment technology, and specifically relates to a position calibration method and apparatus. Background Technology

[0002] With the development of electronic devices and photography technology, electronic devices typically have two data streams when recording video: a preview data stream and a recording data stream. The preview data stream is used to send the images captured by the camera to the screen for display in real time, while the recording data stream is used for background processing to generate the video.

[0003] Object detection is a commonly used technique in electronic devices. In related techniques, the position obtained by object detection in the preview stream image is used as the position of the object in the video stream image. However, the preview stream image and the video stream image in the same frame have large deviations, and the object position also changes. When the position obtained by object detection in the preview stream image is directly used as the position of the object in the video stream image, it will lead to inaccurate positioning of the object in the video stream image. Summary of the Invention

[0004] The purpose of this application is to provide a position calibration method and apparatus that can solve the problem of inaccurate position of an object in a video stream image.

[0005] In a first aspect, embodiments of this application provide a position calibration method, including: Determine the first position of the target object in the preview stream image; wherein the target object includes one or more objects; Based on the first position information, the target object's position in the video stream image is calibrated using a target position calibration method to obtain the target object's second position information in the video stream image; wherein, the target position calibration method includes at least one of the following: frame rate calibration method, motion state calibration method, zoom state calibration method, and output specification calibration method.

[0006] Secondly, embodiments of this application provide a position calibration device, comprising: The first determining module is used to determine the first position information of the target object in the first preview stream image; wherein, the target object includes one or more objects; The calibration module is used to calibrate the position of the target object in the video stream image according to the first position information and the target position calibration method, so as to obtain the second position information of the target object in the video stream image; wherein, the target position calibration method includes at least one of the following: frame rate calibration method, motion state calibration method, zoom state calibration method, and output specification calibration method.

[0007] Thirdly, embodiments of this application provide an electronic device, which includes a processor and a memory. The memory stores programs or instructions that can run on the processor, and when the programs or instructions are executed by the processor, they implement the steps of the position calibration method provided in embodiments of this application.

[0008] Fourthly, embodiments of this application provide a readable storage medium storing a program or instructions, which, when executed by a processor, implement the steps of the position calibration method provided in embodiments of this application.

[0009] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the steps of the position calibration method provided in embodiments of this application.

[0010] Sixthly, embodiments of this application provide a computer program product, which is stored in a storage medium and executed by at least one processor to implement the steps of the position calibration method provided in embodiments of this application.

[0011] In this embodiment, a first position information of a target object in the preview stream image is determined; wherein the target object includes one or more objects; based on the first position information, a target position calibration method is used to calibrate the position of the target object in the video stream image, thereby obtaining a second position information of the target object in the video stream image; wherein the target position calibration method includes at least one of the following: frame rate calibration method, motion state calibration method, zoom state calibration method, and output specification calibration method. In this way, the position of the object in the video stream image can be calibrated, improving the accuracy of the object's position in the video stream image. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating a position calibration method provided in some embodiments of this application; Figure 2 These are schematic diagrams of the position calibration device provided in some embodiments of this application; Figure 3 These are schematic diagrams of the structure of electronic devices provided in some embodiments of this application; Figure 4 These are schematic diagrams of the hardware structure of electronic devices provided in some embodiments of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0014] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0015] The terminology used in the implementation section of this application is only for explaining specific embodiments of this application and is not intended to limit this application. The terminology involved in the embodiments of this application is explained below.

[0016] An object refers to an entity within an image that has clear semantics or independent features, such as a person, a car, or an animal.

[0017] The preview stream image is an image displayed in real time on the preview interface.

[0018] Video stream images are images processed in the background during recording.

[0019] The position calibration method and apparatus provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0020] It should be noted that the position calibration method provided in this application can be executed by electronic devices such as mobile phones, tablets, laptops, PDAs, and in-vehicle electronic devices. Some embodiments of this application use electronic devices as the executing entity to illustrate the position calibration method provided in this application.

[0021] The position calibration method provided in this application can be applied to video recording scenarios.

[0022] The position calibration method and apparatus provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0023] Figure 1This is a schematic flowchart illustrating a position calibration method provided in some embodiments of this application. The position calibration method includes: Step 101: The electronic device determines the first location information of the target object in the preview stream image; wherein, the target object includes one or more objects; In some embodiments of this application, the electronic device may use an object detection algorithm to identify objects in the preview stream image and obtain the position information of the objects in the preview stream image.

[0024] For example, the position information of the object in the preview stream image is denoted as (x p y p w p h p ), where x p y is the x-coordinate of the top-left corner of the object in the preview stream image; p w is the top-left ordinate of the object in the preview stream image; p h is the width of the object in the preview stream image. p The height of the object in the preview stream image.

[0025] Step 102: Based on the first position information, the electronic device calibrates the position of the target object in the video stream image using a target position calibration method to obtain the second position information of the target object in the video stream image; wherein, the target position calibration method includes at least one of the following: Frame rate calibration method, motion state calibration method, zoom state calibration method, output specification calibration method.

[0026] In some embodiments of this application, the frame rate calibration method is the calibration method used when the frame rate of the preview stream image and the frame rate of the video stream image are different; the motion state calibration method is the calibration method used when electronic image stabilization is applied to the video stream image during recording; the zoom state calibration method is the calibration method used when zooming during recording; and the output specification calibration method is the calibration method used when the size of the preview stream image and the video stream image is inconsistent.

[0027] In some embodiments of this application, when the frame rate of the preview stream image and the frame rate of the video stream image are different, a frame rate calibration method is used for position calibration; when electronic image stabilization is applied to the video stream image during recording, a motion state calibration method is used for position calibration; when zooming occurs during recording, a zoom state calibration method is used for position calibration; and when the sizes of the preview stream image and the video stream image are inconsistent, an output specification calibration method is used for position calibration.

[0028] In some embodiments of this application, when the frame rates of the preview stream image and the video stream image are different and electronic image stabilization is applied to the video stream image during recording, position calibration is performed using a frame rate calibration method and a motion state calibration method; when the frame rates of the preview stream image and the video stream image are different and zooming occurs during recording, position calibration is performed using a frame rate calibration method and a zoom state calibration method; when the frame rates of the preview stream image and the video stream image are different and the sizes of the preview stream image and the video stream image are inconsistent, position calibration is performed using a frame rate calibration method and an output specification calibration method; when electronic image stabilization is applied to the video stream image during recording and zooming occurs during recording, position calibration is performed using a motion state calibration method and a zoom state calibration method; when electronic image stabilization is applied to the video stream image during recording and the sizes of the preview stream image and the video stream image are inconsistent, position calibration is performed using a motion state calibration method and an output specification calibration method; when zooming occurs during recording and the sizes of the preview stream image and the video stream image are inconsistent, position calibration is performed using a zoom state calibration method and an output specification calibration method.

[0029] In some embodiments of this application, when the frame rate of the preview stream image and the frame rate of the video stream image are different, electronic image stabilization is applied to the video stream image during recording, and zooming occurs during recording, position calibration is performed using frame rate calibration, motion state calibration, and zoom state calibration. When the frame rate of the preview stream image and the frame rate of the video stream image are different, electronic image stabilization is applied to the video stream image during recording, and the sizes of the preview stream image and the video stream image are inconsistent, position calibration is performed using frame rate calibration, motion state calibration, and output specification calibration. When the frame rate of the preview stream image and the frame rate of the video stream image are different, zooming occurs during recording, and the sizes of the preview stream image and the video stream image are inconsistent, position calibration is performed using frame rate calibration, zoom state calibration, and output specification calibration. When electronic image stabilization is applied to the video stream image during recording, zooming occurs during recording, and the sizes of the preview stream image and the video stream image are inconsistent, position calibration is performed using motion state calibration, zoom state calibration, and output specification calibration.

[0030] In some embodiments of this application, when the frame rate of the preview stream image and the frame rate of the video stream image are different, when electronic image stabilization is applied to the video stream image during recording, when zooming occurs during recording and the sizes of the preview stream image and the video stream image are inconsistent, position calibration is performed using frame rate calibration, motion state calibration, zoom state calibration, and output specification calibration.

[0031] It should be noted that if at least two calibration methods are used for position calibration, and the output specification calibration method is included among the at least two calibration methods, then the position calibration must be performed first using the non-output specification calibration method, and then the position calibrated using the output specification calibration method must be calibrated again using the position calibrated using the non-output specification calibration method.

[0032] For example, when performing position calibration using zoom state calibration method and output specification calibration method, the position is first calibrated using zoom state calibration method, and then the position calibrated using zoom state calibration method is calibrated using output specification calibration method.

[0033] For example, when performing position calibration using frame rate calibration, zoom state calibration, and output specification calibration, the position is first calibrated using frame rate calibration, then the position calibrated using zoom state calibration is calibrated using zoom state calibration, and finally the position calibrated using zoom state calibration is calibrated using output specification calibration.

[0034] For example, when performing position calibration using frame rate calibration, motion state calibration, zoom state calibration, and output specification calibration, the position is first calibrated using frame rate calibration, then the position calibrated using frame rate calibration is calibrated using motion state calibration, then the position calibrated using zoom state calibration is calibrated using zoom state calibration, and finally the position calibrated using output specification calibration is calibrated using zoom state calibration.

[0035] It should also be noted that if at least three calibration methods are used for position calibration, and the output specification calibration method is included among the at least three calibration methods, the order of the at least two calibration methods other than the output specification calibration method can be set according to actual needs.

[0036] For example, when performing position calibration using frame rate calibration, zoom state calibration, and output specification calibration, one can first use frame rate calibration to perform position calibration, then use zoom state calibration to calibrate the position calibrated using frame rate calibration, and then use output specification calibration to calibrate the position calibrated using zoom state calibration; alternatively, one can first use zoom state calibration to perform position calibration, then use frame rate calibration to calibrate the position calibrated using zoom state calibration, and then use output specification calibration to calibrate the position calibrated using frame rate calibration.

[0037] It should also be noted that if at least two calibration methods are used for position calibration and the output specification calibration method is not included among the at least two calibration methods, the order in which the at least two calibration methods are used can be set according to actual needs.

[0038] For example, when using frame rate calibration and zoom state calibration for position calibration, the position can be calibrated first using frame rate calibration, and then the position calibrated using zoom state calibration can be calibrated using the frame rate calibration method; alternatively, the position can be calibrated first using zoom state calibration, and then the position calibrated using frame rate calibration can be calibrated using the zoom state calibration method.

[0039] In some embodiments of this application, the target position calibration method includes a frame rate calibration method; correspondingly, step 102 may include: acquiring the timestamps of the first video stream image, the first preview stream image, and the second preview stream image; wherein, the first preview stream image is a preview stream image before the first video stream image, and the second preview stream image is a preview stream image after the first video stream image; determining a time weight based on the timestamps of the first video stream image, the first preview stream image, and the second preview stream image; wherein, the time weight is used to measure the proportion of the timestamp of the first video stream image within the timestamp interval, and the timestamp interval is the interval from the timestamp of the first preview stream image to the timestamp of the second preview stream image; determining the second position information of the target object in the first video stream based on the time weight, the third position information of the target object in the first preview stream image, and the fourth position information of the target object in the second preview stream image.

[0040] In some embodiments of this application, the first preview stream image can be a preview stream image corresponding to the timestamp before the timestamp of the first video stream image and the timestamp closest to the timestamp of the first video stream image; the second preview stream image can be a preview stream image corresponding to the timestamp after the timestamp of the first video stream image and the timestamp closest to the timestamp of the first video stream image.

[0041] Under normal circumstances, the preview stream and the recorded stream have the same frame rate, and in this case, they can be synchronized using the request number. However, there may be situations where the frame rates of the preview stream and the recorded stream are inconsistent, potentially leading to frame errors and misaligned object positions. In such cases, a frame rate calibration method is needed to ensure that each recorded stream image captures the accurate object position. The core of the frame rate calibration method is to use timestamps and a cached queue of object results in the preview stream to perform temporal and spatial data alignment and mapping. Specifically, the timestamps of the two preview stream images that are temporally closest to the timestamp of the recorded stream image can be retrieved. These two timestamps are used to determine a time weight, which is then used to determine the target object's position information within the recorded stream image.

[0042] In some embodiments of this application, the time weight can be determined by the following formula (1): α=(Ts-T1) / (T2-T1) (1) In formula (1), α is the time weight, Ts is the timestamp of the first video stream image, T1 is the timestamp of the first preview stream image, and T2 is the timestamp of the second preview stream image.

[0043] In some embodiments of this application, the second location information of the target object in the first video stream can be determined by the following formula (2): (2) In formula (2), the second position information of the target object in the first video stream is (Xv, Yv, Wv, Hv), the third position information of the target object in the first preview stream is (x1, y1, w1, h1), and the fourth position information of the target object in the second preview stream is (x2, y2, w2, h2).

[0044] In some embodiments of this application, before determining the second position information of the target object in the first video stream based on the time weight, the third position information of the target object in the first preview stream image, and the fourth position information of the target object in the second preview stream image, the position calibration method provided in this application may further include: determining the frame rate difference between the preview stream frame rate and the video stream frame rate based on the preview stream frame rate and the video stream frame rate; and adjusting the time weight based on the frame rate difference.

[0045] In some embodiments of this application, the frame rate difference between the preview stream frame rate and the recording stream frame rate can be determined by the following formula (3): β=|Fp-Fv| / max(Fp,Fv) (3) In formula (3), β is the frame rate difference between the preview stream frame rate Fp and the recording stream frame rate Fv.

[0046] In some embodiments of this application, when adjusting the time weight based on the frame rate difference, the time weight can be adjusted based on the frame rate difference and the frame rate difference threshold.

[0047] In some embodiments of this application, when adjusting the time weight based on the frame rate difference and the frame rate difference threshold, the time weight can be adjusted by the following formula (4) when the frame rate difference is greater than or equal to the frame rate difference threshold; and the time weight can be adjusted by the following formula (5) when the frame rate difference is less than the frame rate difference threshold.

[0048] α = 1 - (1 - α) 1+β (4) α = α·(1+0.5β) (5) In some embodiments of this application, the target position calibration method includes a motion state calibration method; accordingly, step 102 may include: converting the first position in the active array domain to the electronic image stabilization distortion input domain to obtain the fifth position information; performing position transformation on the fifth position information according to the electronic image stabilization transformation matrix to obtain the sixth position information; and converting the sixth position information to the active array domain to obtain the second position information.

[0049] In some embodiments of this application, to improve the quality and stability of the recorded video, an Electronic Image Stabilization (EIS) algorithm is typically used in the video stream. The EIS algorithm buffers multiple frames, and the image cropped after EIS warp often differs significantly from the preview image, causing object position deviation. When motion is detected in the electronic device and EIS recording is activated, EIS warp mapping calibration is required. Since the EIS warp matrix is ​​based on the EIS warp in domain, the target object position needs to be transferred from the Active Array Domain (AA Domain) to the EIS warp in domain. After EIS warp cropping and calibration, it is then converted back to the AA Domain.

[0050] The conversion from AA Domain to EIS warp in Domain is actually a linear mapping of the target object's position from the source coordinate system (AA Domain coordinate system) to the target coordinate system (EIS warp in Domain coordinate system).

[0051] Assume the AADomain coordinate system is (x src y src w src h src ), EIS warp in Domain coordinate system is (x dst y dst w dst h dst Coordinate transformation can be performed using the following formula (6): (6) In formula (6), the first position information of the target object in the AADomain coordinate system is (x1, y1, w1, h1), and after transformation, the fifth position information of the target object in the EIS warp inDomain coordinate system is (x2, y2, w2, h2).

[0052] The core of EIS warp calibration is to use warping to counteract jitter. To comprehensively describe translation, scaling, and rotation transformations, a 3x3 affine transformation matrix H is defined. Assume the affine transformation matrix H is as follows:

[0053] Where s is the scaling factor, Let t be the rotation angle. x and t y This represents the translation component.

[0054] Suppose that the sixth position information obtained by transforming the fifth position information (x2, y2, w2, h2) using the electronic image stabilization transformation matrix is ​​(x warp y warp w warp h warp Then we have:

[0055] Then x warp =h 11 ·x2+h 12 ·y2+h 13 =s·cos ·x2-s·sin ·y2+t x ; y warp =h 21 ·x2+h 22 ·y2+h 23 =s·sin ·x2+s·cos ·y2+t y ;w warp =s·w2;h warp =s·h2.

[0056] Then, using the above formula (6) to (x warp y warp w warp h warp The data is then converted to AADomain to obtain the second location information.

[0057] In some embodiments of this application, the target position calibration method includes a zoom state calibration method; accordingly, step 102 may include: obtaining the zoom ratio of the image sensor; determining a scaling compensation factor based on the zoom ratio; and compensating the first position information based on the scaling compensation factor to obtain the second position information.

[0058] When there is no zoom during recording, the image does not need to be cropped, and the object location output by the object detection algorithm is in the AA Domain, which can also be described as the zoom Domain. When zooming occurs during recording, the zoom ratio changes constantly, and the cropping ratio also changes, resulting in different cropping sizes for each frame. Image zoom cropping is typically implemented internally by the Image Signal Processor (ISP). The image detected by the object detection algorithm is the cropped image, which is now in the zoom Domain. Zoom calibration is then required to map the target object location from the zoom Domain to the AA Domain.

[0059] Assuming the zoom ratio is zoomRatio, the hardware sensor size (HW Sensor Size) of the image sensor is (w hw h hw The scaling factor is then scaleFcator = 1 / zoomRatio. Assume the target object's position information in zoomDomain is (x1, y1, w1, h1), and the target object's position information in AADomain is (x2, y2, w2, h2). Then: x2 = x1·scaleFcator + 0.5·w hw • (1-scaleFcator); y2 = y1·scaleFcator + 0.5·h hw • (1-scaleFcator); w2 = w1·scaleFcator; h2 = h1·scaleFcator.

[0060] The coordinates of zoomDomain are mapped to AADomain using the scaling compensation factor scaleFactor; 0.5·w hw • (1-scaleFcator) is the horizontal center alignment compensation, 0.5·h hw • (1-scaleFcator) is the vertical center alignment compensation, which compensates for the center point offset caused by clipping. Through scaling compensation and center alignment compensation, a precise mapping of coordinates from the clipped zoomDomain to the AADomain can be achieved.

[0061] In some embodiments of this application, the target position calibration method includes an output specification calibration method; accordingly, step 102 may include: determining position adjustment parameters based on the dimensions of the preview stream image and the video stream image; wherein, the position adjustment parameters include a scaling factor of the preview stream image relative to the video stream image, a horizontal offset, and a vertical offset; and determining second position information based on the first position information and the position adjustment parameters.

[0062] In some possible implementations of this application, since the video stream image is based on the Video Dimension Domain, it is necessary to convert the object position information from the AA Domain to the Video Dimension Domain. At this point, the output specifications need to be checked to determine the final output video stream image size. Simultaneously, the aspect ratio of the video stream image also needs to be checked. If the aspect ratio of the video stream image matches the aspect ratio of the HW Sensor Size, then only the mapping from the AA Domain to the Video Dimension Domain needs to be performed. If the two aspect ratios are inconsistent, direct mapping will lead to object box distortion or even loss of object box information. In this case, output domain coordinate mapping calibration of the target object position information is required.

[0063] Assume the coordinate system of HW Sensor Size is (x src y src w src h src The video stream image coordinate system is (x dst y dst w dst h dst HW Sensor Size aspect ratio srcRatio=w src / h src The aspect ratio of the video stream image is dstRatio=w dst / h dst .

[0064] When dstRatio is less than or equal to srcRatio, the recorded stream image is narrower horizontally than the preview stream image. To ensure image integrity, vertical fill priority mapping is required, i.e., based on the height h of the recorded stream image. dst Scaling and adaptation are performed based on the baseline, at which point Ratio=h dst / h src xOffset = (w src ·Ratio-w dst ) / 2; yOffset=0.

[0065] When dstRatio is greater than srcRatio, the recorded stream image is narrower in the vertical direction compared to the preview stream image. To ensure image integrity, horizontal fill priority mapping is required, i.e., the width w of the recorded stream image is used as the fill priority mapping. dst Scaling is performed based on the baseline, at which point Ratio=w dst / w src xOffset=0; yOffset=(h src ·Ratio-h dst ) / 2.

[0066] Where Ratio represents the scaling factor from the preview stream image to the recorded stream image, xOffset represents the horizontal offset, and yOffset represents the vertical offset.

[0067] Assuming the target object's location information in the AA Domain is (x1, y1, w1, h1), and the target object's location information in the VideoDimensionDomain is (x2, y2, w2, h2), then: x2=x1·Ratio-xOffset; y2=y1·Ratio-yOffset; w2=w1·Ratio; h2=h1·Ratio.

[0068] In this embodiment, a first position information of a target object in the preview stream image is determined; wherein the target object includes one or more objects; based on the first position information, a target position calibration method is used to calibrate the position of the target object in the video stream image, thereby obtaining a second position information of the target object in the video stream image; wherein the target position calibration method includes at least one of the following: frame rate calibration method, motion state calibration method, zoom state calibration method, and output specification calibration method. In this way, the position of the object in the video stream image can be calibrated, improving the accuracy of the object's position in the video stream image.

[0069] The position calibration method provided in this application can be executed by a position calibration device. This application uses the example of a position calibration device executing the position calibration method to illustrate the position calibration device provided in this application.

[0070] Figure 2 This is a schematic diagram of the structure of a position calibration device provided in some embodiments of this application. The position calibration device 200 includes: The first determining module 201 is used to determine the first position information of the target object in the preview stream image; wherein the target object includes one or more objects; The calibration module 202 is used to calibrate the position of the target object in the video stream image according to the first position information and the target position calibration method, so as to obtain the second position information of the target object in the video stream image; wherein, the target position calibration method includes at least one of the following: frame rate calibration method, motion state calibration method, zoom state calibration method, and output specification calibration method.

[0071] In this embodiment, a first position information of a target object in the preview stream image is determined; wherein the target object includes one or more objects; based on the first position information, a target position calibration method is used to calibrate the position of the target object in the video stream image, thereby obtaining a second position information of the target object in the video stream image; wherein the target position calibration method includes at least one of the following: frame rate calibration method, motion state calibration method, zoom state calibration method, and output specification calibration method. In this way, the position of the object in the video stream image can be calibrated, improving the accuracy of the object's position in the video stream image.

[0072] In some embodiments of this application, the target position calibration method includes a frame rate calibration method; the calibration module 202 is specifically used for: Obtain the timestamps of the first recorded video stream image, the first preview stream image, and the second preview stream image; wherein, the first preview stream image is the preview stream image before the first recorded video stream image, and the second preview stream image is the preview stream image after the first recorded video stream image; The time weight is determined based on the timestamps of the first video stream image, the first preview stream image, and the second preview stream image; wherein, the time weight is used to measure the proportion of the timestamp of the first video stream image in the timestamp interval, and the timestamp interval is the interval from the timestamp of the first preview stream image to the timestamp of the second preview stream image; Based on time weights, the third position information of the target object in the first preview stream image, and the fourth position information of the target object in the second preview stream image, the second position information of the target object in the first recording stream is determined.

[0073] In some embodiments of this application, the object position calibration device 200 further includes: The second determining module is used to determine the frame rate difference between the preview stream image frame rate and the video stream image frame rate based on the preview stream image frame rate and the video stream image frame rate; The adjustment module is used to adjust the time weights based on frame rate differences.

[0074] In some embodiments of this application, the target position calibration method includes a motion state calibration method; the calibration module 202 is specifically used for: The first position in the active array domain will be converted to the electronic image stabilization distortion input domain to obtain the fifth position information; Based on the electronic image stabilization transformation matrix, the fifth position information is transformed to obtain the sixth position information; The sixth position information is converted to the active array domain to obtain the second position information.

[0075] In some embodiments of this application, the target position calibration method includes a zoom state calibration method; the calibration module 202 is specifically used for: Obtain the zoom ratio of the image sensor; Determine the scaling compensation factor based on the zoom ratio; The first position information is compensated based on the scaling compensation factor to obtain the second position information.

[0076] In some embodiments of this application, the target position calibration method includes an output specification calibration method; the calibration module 202 is specifically used for: The position adjustment parameters are determined based on the dimensions of the preview stream image and the video stream image; the position adjustment parameters include the scaling factor of the preview stream image relative to the video stream image, the horizontal offset, and the vertical offset; The second position information is determined based on the first position information and the position adjustment parameters.

[0077] The object position calibration device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0078] The position calibration device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0079] The position calibration device provided in this application embodiment can achieve... Figure 1 The various processes implemented in the position calibration method embodiment will not be described again here to avoid repetition.

[0080] Optionally, such as Figure 3 As shown, this application embodiment also provides an electronic device 300, including a processor 301 and a memory 302. The memory 302 stores a program or instructions that can run on the processor 301. When the program or instructions are executed by the processor 301, they implement the various steps of the position calibration method embodiment provided in this application embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0081] Figure 4 These are schematic diagrams of the hardware structure of electronic devices provided in some embodiments of this application.

[0082] The electronic device 400 includes, but is not limited to, components such as: radio frequency unit 401, network module 402, audio output unit 403, input unit 404, sensor 405, display unit 406, user input unit 407, interface unit 408, memory 409, and processor 410.

[0083] Those skilled in the art will understand that the electronic device 400 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 410 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0084] The processor 410 is configured to: determine first position information of a target object in a preview stream image; wherein the target object includes one or more objects; calibrate the position of the target object in a video stream image using a target position calibration method based on the first position information to obtain second position information of the target object in the video stream image; wherein the target position calibration method includes at least one of the following: frame rate calibration method, motion state calibration method, zoom state calibration method, and output specification calibration method.

[0085] In this embodiment, a first position information of the target object in the preview stream image is determined; based on the first position information, a target position calibration method is used to calibrate the position of the target object in the video stream image, thereby obtaining a second position information of the target object in the video stream image; wherein, the target position calibration method includes at least one of the following: frame rate calibration method, motion state calibration method, zoom state calibration method, and output specification calibration method. In this way, the position of the object in the video stream image can be calibrated, improving the accuracy of the object's position in the video stream image.

[0086] In some embodiments of this application, the target position calibration method includes a frame rate calibration method; the processor 410 is specifically used for: Obtain the timestamps of the first recorded video stream image, the first preview stream image, and the second preview stream image; wherein, the first preview stream image is the preview stream image before the first recorded video stream image, and the second preview stream image is the preview stream image after the first recorded video stream image; The time weight is determined based on the timestamps of the first video stream image, the first preview stream image, and the second preview stream image; wherein, the time weight is used to measure the proportion of the timestamp of the first video stream image in the timestamp interval, and the timestamp interval is the interval from the timestamp of the first preview stream image to the timestamp of the second preview stream image; Based on time weights, the third position information of the target object in the first preview stream image, and the fourth position information of the target object in the second preview stream image, the second position information of the object in the first recording stream is determined.

[0087] In some embodiments of this application, the processor 410 is also used for: Determine the frame rate difference between the preview stream image frame rate and the video stream image frame rate; Adjust the time weights based on frame rate differences.

[0088] In some embodiments of this application, the target position calibration method includes a motion state calibration method; the processor 410 is specifically used for: The first position information in the active array domain is converted to the electronic image stabilization distortion input domain to obtain the fifth position information; Based on the electronic image stabilization transformation matrix, the fifth position information is transformed to obtain the sixth position information; The sixth position information is converted to the active array domain to obtain the second position information.

[0089] In some embodiments of this application, the target position calibration method includes a zoom state calibration method; the processor 410 is specifically used for: Obtain the zoom ratio of the image sensor; Determine the scaling compensation factor based on the zoom ratio; The first position information is compensated based on the scaling compensation factor to obtain the second position information.

[0090] In some embodiments of this application, the target position calibration method includes an output specification calibration method; the processor 410 is specifically used for: The position adjustment parameters are determined based on the dimensions of the preview stream image and the video stream image; the position adjustment parameters include the scaling factor of the preview stream image relative to the video stream image, the horizontal offset, and the vertical offset; The second position information is determined based on the first position information and the position adjustment parameters.

[0091] It should be understood that, in this embodiment, the input unit 404 may include a graphics processing unit (GPU) 4041 and a microphone 4042. The GPU 4041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 406 may include a display panel 4061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 407 includes at least one of a touch panel 4071 and other input devices 4072. The touch panel 4071 is also called a touch screen. The touch panel 4071 may include a touch detection device and a touch controller. Other input devices 4072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0092] The memory 409 can be used to store software programs and various data. The memory 409 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 409 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 409 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0093] Processor 410 may include one or more processing units; optionally, processor 410 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 410.

[0094] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the position calibration method provided in this application and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0095] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0096] This application also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the position calibration method provided in this application and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0097] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0098] This application also provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the position calibration method embodiment provided in this application, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0099] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0100] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0101] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A position calibration method, characterized in that, The method includes: Determine the first position information of the target object in the preview stream image; wherein, the target object includes one or more objects; Based on the first location information, a target location calibration method is used to calibrate the position of the target object in the video stream image, thereby obtaining the second location information of the target object in the video stream image; wherein, the target location calibration method includes at least one of the following: Frame rate calibration method, motion state calibration method, zoom state calibration method, output specification calibration method.

2. The method according to claim 1, characterized in that, The target location calibration method includes a frame rate calibration method; The step of calibrating the position of the target object in the video stream image based on the first position information using a target position calibration method to obtain the second position information of the target object in the video stream image includes: Obtain the timestamps of the first recorded video stream image, the first preview stream image, and the second preview stream image; wherein, the first preview stream image is the preview stream image preceding the first recorded video stream image, and the second preview stream image is the preview image following the first recorded video stream image; A time weight is determined based on the timestamps of the first recorded stream image, the first preview stream image, and the second preview stream image; wherein, the time weight is used to measure the proportion of the timestamp of the first recorded stream image in the timestamp interval, and the timestamp interval is the interval from the timestamp of the first preview stream image to the timestamp of the second preview stream image; Based on the time weight, the third position information of the target object in the first preview stream image, and the fourth position information of the target object in the second preview stream image, the second position information of the target object in the first recording stream is determined.

3. The method according to claim 1, characterized in that, The target position calibration method includes a motion state calibration method; The step of calibrating the position of the target object in the video stream image based on the first position information using a target position calibration method to obtain the second position information of the target object in the video stream image includes: The first position information in the active array domain is converted to the electronic image stabilization distortion input domain to obtain the fifth position information; The fifth position information is transformed according to the electronic image stabilization transformation matrix to obtain the sixth position information; The sixth position information is converted to the active array domain to obtain the second position information.

4. The method according to claim 1, characterized in that, The target position calibration method includes a zoom state calibration method; The step of calibrating the position of the target object in the video stream image based on the first position information using a target position calibration method to obtain the second position information of the target object in the video stream image includes: Obtain the zoom ratio of the image sensor; Determine the scaling compensation factor based on the zoom ratio; The first position information is compensated according to the scaling compensation factor to obtain the second position information.

5. The method according to claim 1, characterized in that, The target position calibration method includes the output specification calibration method; The step of calibrating the position of the target object in the video stream image based on the first position information using a target position calibration method to obtain the second position information of the target object in the video stream image includes: Based on the dimensions of the preview stream image and the video stream image, position adjustment parameters are determined; wherein, the position adjustment parameters include the scaling factor of the preview stream image relative to the video stream image, the horizontal offset, and the vertical offset; The second location information is determined based on the first location information and the location adjustment parameters.

6. A position calibration device, characterized in that, The device includes: The first determining module is used to determine the first position information of the target object in the preview stream image; wherein, the target object includes one or more objects; A calibration module is configured to calibrate the position of the target object in the video stream image based on the first position information and using a target position calibration method, thereby obtaining second position information of the target object in the video stream image; wherein the target position calibration method includes at least one of the following: Frame rate calibration method, motion state calibration method, zoom state calibration method, output specification calibration method.

7. The apparatus according to claim 6, characterized in that, The target location calibration method includes a frame rate calibration method; the calibration module is specifically used for: Obtain the timestamps of the first recorded video stream image, the first preview stream image, and the second preview stream image; wherein, the first preview stream image is the preview stream image preceding the first recorded video stream image, and the second preview stream image is the preview stream image following the first recorded video stream image; A time weight is determined based on the timestamps of the first recorded stream image, the first preview stream image, and the second preview stream image; wherein, the time weight is used to measure the proportion of the timestamp of the first recorded stream image in the timestamp interval, and the timestamp interval is the interval from the timestamp of the first preview stream image to the timestamp of the second preview stream image; Based on the time weight, the third position information of the target object in the first preview stream image, and the fourth position information of the target object in the second preview stream image, the second position information of the target object in the first recording stream is determined.

8. The apparatus according to claim 6, characterized in that, The target position calibration method includes a motion state calibration method; the calibration module is specifically used for: The first position information in the active array domain is converted to the electronic image stabilization distortion input domain to obtain the fifth position information; The fifth position information is transformed according to the electronic image stabilization transformation matrix to obtain the sixth position information; The sixth position information is converted to the active array domain to obtain the second position information.

9. The apparatus according to claim 6, characterized in that, The target position calibration method includes a zoom state calibration method; the calibration module is specifically used for: Obtain the zoom ratio of the image sensor; Determine the scaling compensation factor based on the zoom ratio; The first position information is compensated according to the scaling compensation factor to obtain the second position information.

10. The apparatus according to claim 6, characterized in that, The target position calibration method includes an output specification calibration method; the calibration module is specifically used for: Based on the dimensions of the preview stream image and the video stream image, position adjustment parameters are determined; wherein, the position adjustment parameters include the scaling factor of the preview stream image relative to the video stream image, the horizontal offset, and the vertical offset; The second location information is determined based on the first location information and the location adjustment parameters.

Images