A system and method for calculating camera image stabilization performance

Abstract

This application provides a system and method for calculating the image stabilization performance of a camera. The system includes a processor configured to perform the following actions: acquiring a first image sequence when the pan-tilt mount is at a target jitter frequency and the camera component is not stabilized; acquiring a second image sequence when the pan-tilt mount is at the target jitter frequency and the camera component is stabilized; determining a jitter dynamic blur amount based on the first image sequence and still images; determining a measured comprehensive blur amount based on the second image sequence and still images; generating a reference dynamic blur amount curve based on the jitter dynamic blur amount; generating a measured dynamic blur amount curve based on the measured comprehensive blur amount; and calculating the image stabilization performance at the target jitter frequency based on the first intersection point of the reference dynamic blur amount curve and a preset straight line, and the second intersection point of the measured dynamic blur amount curve and the preset straight line. This technical solution enables accurate determination of the image stabilization performance of the camera component, making the image stabilization evaluation more objective.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a system and method for calculating the image stabilization performance of a camera. Background Technology

[0002] Image stabilization can be divided into electronic image stabilization and optical image stabilization. Electronic image stabilization does not involve hardware; it achieves stabilization through digital processing, primarily by increasing ISO sensitivity and reducing shutter speed. Optical image stabilization, also known as shift-based image stabilization, works by moving the lens or CCD (Charge-Coupled Device, i.e., the photosensitive element) in the opposite direction of the shake to counteract the effects of the shake. This avoids or reduces shake during the capture of optical signals, improving image quality and reducing image blur.

[0003] Optical image stabilization can include lens-shift optical image stabilization and CCD-shift optical image stabilization. Lens-shift optical image stabilization achieves optical image stabilization by moving the lens. For example, a gyroscope can detect shake information and send it to a processor. The processor calculates the amount of displacement to be compensated based on the shake information, and moves the lens according to the direction of the shake and the amount of displacement, thereby effectively reducing image blur.

[0004] CCD-based optical image stabilization achieves optical image stabilization by moving the CCD. The CCD is fixed on a support that can move up, down, left, and right. A gyroscope senses jitter information (such as the direction and amplitude of jitter) and sends the jitter information to a processor. The processor calculates the amount of displacement to be compensated based on the jitter information and moves the CCD according to the jitter direction and the amount of displacement, thereby effectively reducing image blur.

[0005] Clearly, image stabilization is a crucial indicator for cameras, representing their effectiveness in preventing shake. For example, better image stabilization means better image quality and a clearer picture even under shaky conditions. Conversely, poorer image stabilization results in poorer image quality and a blurrier picture even under shaky conditions.

[0006] Obviously, for cameras, it's necessary to know their image stabilization capabilities to reflect the effectiveness of their stabilization. However, there's currently no reasonable way to accurately determine a camera's image stabilization performance. Summary of the Invention

[0007] This application provides a system for calculating the image stabilization performance of a camera, comprising:

[0008] Camera components;

[0009] A pan-tilt bracket that can support the camera assembly to allow the camera assembly to rotate horizontally and vertically at least one of the two.

[0010] A preset image containing only a checkerboard pattern is set within the visible area of ​​the camera component;

[0011] The camera component is defined such that its visible area covers the size of the preset image, so that the image captured by the camera component includes the checkerboard pattern;

[0012] Processor, the processor being configured to perform:

[0013] When the gimbal support is stationary, a still image is acquired;

[0014] When the gimbal bracket is at a preset target shaking frequency and the camera component is not stabilizing, acquire the first image sequence corresponding to each of the multiple shutter times;

[0015] When the gimbal bracket is at the preset target shaking frequency and the camera component has its image stabilization function enabled, the second image sequence corresponding to each of the multiple shutter times is acquired.

[0016] For each shutter speed, the amount of motion blur corresponding to that shutter speed is determined based on the first image sequence and the still image corresponding to that shutter speed; the measured overall blur amount corresponding to that shutter speed is determined based on the second image sequence and the still image corresponding to that shutter speed.

[0017] A reference dynamic blur curve is generated based on the jitter dynamic blur amount corresponding to all shutter speeds, and a measured dynamic blur curve is generated based on the measured comprehensive blur amount corresponding to all shutter speeds.

[0018] Based on the first intersection point of the reference dynamic blur curve and the preset straight line, and the second intersection point of the measured dynamic blur curve and the preset straight line, the image stabilization performance of the camera component at the target shaking frequency is calculated; wherein, the preset straight line is used to indicate the maximum tolerance value of the image stabilization performance.

[0019] This application provides a system for calculating the image stabilization performance of a camera, comprising:

[0020] Camera components;

[0021] A pan-tilt bracket that can support the camera assembly to allow the camera assembly to rotate horizontally and vertically at least one of the two.

[0022] A preset image containing only a checkerboard pattern is set within the visible area of ​​the camera component;

[0023] The camera component is defined such that its visible area covers the size of the preset image, so that the image captured by the camera component includes the checkerboard pattern;

[0024] Processor, the processor being configured to perform:

[0025] When the gimbal bracket is at a preset target jitter frequency and the camera component is not stabilizing, the third image sequence captured by the camera component is obtained.

[0026] When the gimbal bracket is at the preset target jitter frequency and the camera component has its image stabilization function enabled, the fourth image sequence captured by the camera component is acquired.

[0027] For each of the multiple points of interest in the chessboard, the center stability of the point of interest is determined based on its coordinate position in each frame of the third image in the third image sequence and its coordinate position in each frame of the fourth image in the fourth image sequence; wherein, the center stability is used to reflect the image stabilization capability of the camera component at the point of interest.

[0028] The overall stability is determined based on the center stability of the multiple points of interest; wherein, the overall stability is used to reflect the image stabilization capability of the camera assembly across the entire frame;

[0029] The image stabilization performance of the camera component at the target shaking frequency is calculated based on the overall stability; wherein, the greater the overall stability, the better the image stabilization performance.

[0030] This application provides a method for calculating the image stabilization performance of a camera, wherein a gimbal bracket supports a camera assembly, allowing the camera assembly to rotate horizontally and vertically at least once; the visible area of ​​the camera assembly covers the size of a preset image containing only a checkerboard pattern, and the image captured by the camera assembly includes the checkerboard pattern; the method includes:

[0031] When the gimbal support is stationary, a still image is acquired;

[0032] When the gimbal bracket is at a preset target shaking frequency and the camera component is not stabilizing, acquire the first image sequence corresponding to each of the multiple shutter times;

[0033] When the gimbal bracket is at the preset target shaking frequency and the camera component has its image stabilization function enabled, the second image sequence corresponding to each of the multiple shutter times is acquired.

[0034] For each shutter speed, the amount of motion blur corresponding to that shutter speed is determined based on the first image sequence and the still image corresponding to that shutter speed; the measured overall blur amount corresponding to that shutter speed is determined based on the second image sequence and the still image corresponding to that shutter speed.

[0035] A reference dynamic blur curve is generated based on the jitter dynamic blur amount corresponding to all shutter speeds, and a measured dynamic blur curve is generated based on the measured comprehensive blur amount corresponding to all shutter speeds.

[0036] Based on the first intersection point of the reference dynamic blur curve and the preset straight line, and the second intersection point of the measured dynamic blur curve and the preset straight line, the image stabilization performance of the camera component at the target shaking frequency is calculated; wherein, the preset straight line is used to indicate the maximum tolerance value of the image stabilization performance.

[0037] This application provides a method for calculating the image stabilization performance of a camera, wherein a gimbal bracket supports a camera assembly, allowing the camera assembly to rotate horizontally and vertically at least once; the visible area of ​​the camera assembly covers the size of a preset image containing only a checkerboard pattern, and the image captured by the camera assembly includes the checkerboard pattern; the method includes:

[0038] When the gimbal bracket is at a preset target jitter frequency and the camera component is not stabilizing, the third image sequence captured by the camera component is obtained.

[0039] When the gimbal bracket is at the preset target jitter frequency and the camera component has its image stabilization function enabled, the fourth image sequence captured by the camera component is acquired.

[0040] For each of the multiple points of interest in the chessboard, the center stability of the point of interest is determined based on its coordinate position in each frame of the third image in the third image sequence and its coordinate position in each frame of the fourth image in the fourth image sequence; wherein, the center stability is used to reflect the image stabilization capability of the camera component at the point of interest.

[0041] The overall stability is determined based on the center stability of the multiple points of interest; wherein, the overall stability is used to reflect the image stabilization capability of the camera assembly across the entire frame;

[0042] The image stabilization performance of the camera component at the target shaking frequency is calculated based on the overall stability; wherein, the greater the overall stability, the better the image stabilization performance.

[0043] As can be seen from the above technical solutions, in this embodiment, a preset image containing a checkerboard pattern is set in the test environment, and the visible area of ​​the camera component covers the size of the preset image, so that the image captured by the camera component includes the checkerboard pattern. When the pan-tilt bracket is at the target jitter frequency and the camera component's image stabilization function is not enabled, an image sequence containing the checkerboard pattern is acquired; when the pan-tilt bracket is at the target jitter frequency and the camera component's image stabilization function is enabled, another image sequence containing the checkerboard pattern is acquired. Based on the above two image sequences, the image stabilization performance of the camera component at the target jitter frequency can be calculated, thereby revealing the image stabilization performance of the camera component. This image stabilization performance reflects the image stabilization effect of the camera component, and the image stabilization performance of the camera component can be accurately determined. The above method can accurately evaluate the static and dynamic image stabilization effects, making the image stabilization evaluation of the camera component more objective. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings of the embodiments of this application.

[0045] Figure 1 This is a schematic diagram of the gimbal bracket in one embodiment of this application;

[0046] Figure 2 This is a schematic diagram of a test environment in one embodiment of this application;

[0047] Figure 3 This is a schematic diagram of a preset image (i.e., a chessboard pattern) in one embodiment of this application;

[0048] Figure 4 This is a schematic flowchart of the method for calculating camera image stabilization performance in this application;

[0049] Figure 5 This is a schematic diagram of the first image in one embodiment of this application;

[0050] Figure 6 This is a schematic diagram illustrating the determination of the image stabilization level in one embodiment of this application;

[0051] Figure 7 This is a schematic flowchart of the method for calculating camera image stabilization performance in this application;

[0052] Figure 8 This is a schematic diagram of multiple points of interest on a chessboard grid in one embodiment of this application;

[0053] Figure 9A This is a schematic diagram of the device used to calculate the image stabilization performance of a camera in this application;

[0054] Figure 9B This is a schematic diagram of the device used to calculate the image stabilization performance of a camera, as described in this application. Detailed Implementation

[0055] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “the,” and “the” as used in this application and claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any and all possible combinations comprising one or more of the associated listed items.

[0056] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" may also be interpreted as "when," "when," or "in response to a determination."

[0057] To determine a camera's image stabilization performance and reflect its effectiveness, this application proposes a system for calculating camera image stabilization performance. In practical applications, if the camera supports electronic image stabilization (EIS), the system calculates its EIS performance. If the camera supports optical image stabilization (OIS), the system calculates its OIS performance. If the camera supports both EIS and OIS, the system calculates the overall image stabilization performance.

[0058] In one possible implementation, the system for calculating camera stabilization performance may include:

[0059] 1. Camera Components. Camera components may include, but are not limited to, lenses and CCDs, and there are no restrictions on the structure of these camera components. For example, a camera component may be a camera that supports image stabilization, such as electronic image stabilization and / or optical image stabilization, and there are no restrictions on the type of image stabilization.

[0060] 2. Pan / Tilt Mount. A pan / tilt mount supports the camera assembly, allowing it to rotate horizontally and vertically at least once. For example, the pan / tilt mount is used to allow the camera assembly to rotate horizontally; or, the pan / tilt mount is used to allow the camera assembly to rotate vertically; or, the pan / tilt mount is used to allow both horizontal and vertical rotation of the camera assembly.

[0061] See Figure 1 The diagram shown is a structural schematic of a pan-tilt mount. This pan-tilt mount may include, but is not limited to, a base 11, a horizontal rotating platform 12, and a vertical rotating platform 13. The structure of the pan-tilt mount is not limited, as long as it enables the camera assembly to rotate horizontally and vertically at least one of these rotations. Figure 1 In the middle, from bottom to top, are the base 11, the horizontal rotating platform 12, and the vertical rotating platform 13.

[0062] See Figure 1 As shown, the camera assembly can be fixed to the vertical rotating stage 13, meaning the camera assembly is fixed to the vertical rotating stage 13. Based on this, when the horizontal rotating stage 12 rotates horizontally, it will cause the vertical rotating stage 13 to also rotate horizontally, causing the camera assembly fixed on the vertical rotating stage 13 to rotate horizontally as well. When the vertical rotating stage 13 rotates vertically, the camera assembly fixed on the vertical rotating stage 13 will also rotate vertically.

[0063] The base 11 is used to fix the gimbal bracket and drive the horizontal rotating platform 12. In other words, the base 11 can drive the horizontal rotating platform 12 to move horizontally, so that the horizontal rotating platform 12 can rotate horizontally.

[0064] The horizontal rotary table 12 is used to rotate horizontally to cause the camera assembly to rotate horizontally, and to drive the vertical rotary table 13. For example, the horizontal rotary table 12 can rotate horizontally, and during this horizontal rotation, the camera assembly can rotate horizontally along with the horizontal rotary table 12. The horizontal rotary table 12 can also drive the vertical rotary table 13 to move vertically, so that the vertical rotary table 13 can rotate vertically.

[0065] The vertical turntable 13 is used to rotate vertically to allow the camera assembly to rotate vertically. For example, the vertical turntable 13 can rotate vertically, and during the vertical rotation, the camera assembly is fixed to the vertical turntable 13, so that the camera assembly rotates vertically along with the vertical turntable 13.

[0066] The pan-tilt mount can vibrate the camera assembly at any frequency. That is, when the pan-tilt mount is at a certain vibration frequency, the camera assembly will also vibrate at that vibration frequency. There is no restriction on the vibration frequency of the pan-tilt mount. For example, the vibration frequency can be 1Hz, 2Hz, 3Hz, 4Hz, 5Hz, 6Hz, 7Hz, 8Hz, 9Hz, 10Hz, 11Hz, 12Hz, 13Hz, 14Hz, 15Hz, etc. There are no restrictions on this.

[0067] The jitter frequency can be a horizontal jitter frequency and a vertical jitter frequency. The jitter frequency of the horizontal rotating stage 12 controlling the camera assembly to rotate horizontally is the horizontal jitter frequency, and the jitter frequency of the vertical rotating stage 13 controlling the camera assembly to rotate vertically is the vertical jitter frequency.

[0068] 3. A preset image containing only a checkerboard pattern is set within the visible area of ​​the camera component. The camera component is limited such that its visible area covers the size of the preset image (i.e., the preset image is located within the visible area of ​​the camera component), so that the image captured by the camera component includes the checkerboard pattern.

[0069] See Figure 2 As shown, a test environment can be set up, which can include a light screen, which is a screen that can receive light and displays a preset image containing only a checkerboard pattern.

[0070] See Figure 3 The image shown is a schematic diagram of the preset image. This preset image only contains a chessboard pattern; that is, it can include multiple black sub-regions and multiple white sub-regions. Sub-regions surrounding black sub-regions are all white sub-regions, and sub-regions surrounding white sub-regions are all black sub-regions, meaning black and white sub-regions alternate. Figure 3 In this example, we take the case where the width and height of the black sub-region are the same, the width of the black sub-region is the same as the width of the white sub-region, and the height of the black sub-region is the same as the height of the white sub-region. In practical applications, the width and height of the black sub-region may differ, as may the width and height of the black sub-region and the white sub-region, respectively; no restrictions are imposed on these.

[0071] See also Figure 2 As shown, the test environment may also include a gimbal bracket and a camera assembly. For example, the gimbal bracket and camera assembly are on a round table. The gimbal bracket enables the camera assembly to rotate horizontally and vertically at least once. The connection relationship between the gimbal bracket and the camera assembly is described in the above embodiment.

[0072] In this test environment, the visible area of ​​the camera component needs to cover the size of the preset image; that is, the image captured by the camera component will include the preset image. Figure 3 The chessboard pattern shown.

[0073] 4. Processor. When the gimbal is at the target jitter frequency (i.e., the camera component is at the target jitter frequency) and the camera component's image stabilization is not enabled, the processor acquires an image sequence containing a checkerboard pattern using the camera component. When the gimbal is at the target jitter frequency and the camera component's image stabilization is enabled, the processor acquires another image sequence containing a checkerboard pattern using the camera component. Based on these two image sequences, the processor can calculate the image stabilization performance of the camera component at the target jitter frequency, thus determining the image stabilization performance and reflecting the effectiveness of image stabilization. For example, the processor can use a static evaluation method to calculate the image stabilization performance of the camera component at the target jitter frequency, i.e., evaluate the static image stabilization effect of the camera component. Alternatively, the processor can also use a dynamic evaluation method to calculate the image stabilization performance of the camera component at the target jitter frequency, i.e., evaluate the dynamic image stabilization effect of the camera component. Alternatively, the processor can also use both static and dynamic evaluation methods to calculate the image stabilization performance of the camera component at the target jitter frequency, i.e., simultaneously evaluate the static and dynamic image stabilization effects of the camera component. The following describes, with reference to specific embodiments, the process of calculating the anti-shake performance of the processor using a static evaluation method, a dynamic evaluation method, and both static and dynamic evaluation methods.

[0074] In one possible implementation, the processor can use a static evaluation method to calculate the image stabilization performance of the camera assembly at the target shake frequency, i.e., to evaluate the static image stabilization effect of the camera assembly. In this implementation, the difference between images with and without image stabilization can be evaluated using the blur level of a single image as a reference to obtain the image stabilization performance. See also Figure 4 As shown, the processor can calculate the image stabilization performance of the camera components using the following steps, which may include:

[0075] Step 401: Acquire a still image when the gimbal support is stationary. For example, capture an image of the gimbal support when it is stationary using a camera component, and record this image as a still image.

[0076] Step 402: When the gimbal bracket is at the preset target shaking frequency and the camera components are not in the image stabilization function, acquire the first image sequence corresponding to each of the multiple shutter times.

[0077] Step 403: When the gimbal bracket is at the preset target shaking frequency and the camera component has the image stabilization function enabled, acquire the second image sequence corresponding to each of the multiple shutter times.

[0078] For example, the pan-tilt unit can be controlled to operate at a preset target jitter frequency. This target jitter frequency can be any jitter frequency, such as 1Hz, 2Hz, 3Hz, 4Hz, 5Hz, 6Hz, 7Hz, 8Hz, 9Hz, 10Hz, 11Hz, 12Hz, 13Hz, 14Hz, 15Hz, etc., without limitation. When the pan-tilt unit is operating at the target jitter frequency, the camera assembly is also operating at that target jitter frequency. Since the processing method is the same for each target jitter frequency, the following explanation will use one target jitter frequency as an example.

[0079] For example, the type of shutter speed can be determined, such as shutter speeds including but not limited to 1 / 256 second, 1 / 128 second, 1 / 64 second, 1 / 32 second, 1 / 16 second, 1 / 8 second, 1 / 4 second, 1 / 2 second, etc., indicating that image sequences at these shutter speeds need to be acquired, without limiting the type of shutter speed.

[0080] In summary, when the gimbal bracket is at the target shaking frequency, the camera assembly can be controlled to not have image stabilization enabled. In this case, firstly, the shutter speed of the camera assembly is controlled to 1 / 256 second, and a first image sequence a11 at this shutter speed of 1 / 256 second is acquired. This first image sequence a11 can include multiple first images, such as 200 first images. Then, the shutter speed of the camera assembly can be controlled to 1 / 128 second, and a first image sequence a12 at this shutter speed of 1 / 128 second is acquired. This first image sequence a12 can also include multiple first images, such as 200 first images. This process can be repeated to obtain the first image sequence at each shutter speed, and each first image sequence can include multiple first images.

[0081] For example, when the gimbal bracket is at the target shaking frequency, the camera assembly can also be controlled to activate image stabilization. In this case, the shutter speed of the camera assembly is first controlled to 1 / 256 second, and a second image sequence a21 at this shutter speed of 1 / 256 second is acquired. This second image sequence a21 may include multiple frames of second images, such as 200 frames of second images. Then, the shutter speed of the camera assembly can be controlled to 1 / 128 second, and a second image sequence a22 at this shutter speed of 1 / 128 second is acquired. This second image sequence a22 may include multiple frames of second images, such as 200 frames of second images. This process can be repeated to obtain second image sequences at various shutter speeds, and each second image sequence may include multiple frames of second images.

[0082] Step 404: For each shutter speed, determine the amount of motion blur corresponding to that shutter speed based on the first image sequence and the still image. Also, determine the measured overall blur amount corresponding to that shutter speed based on the second image sequence and the still image.

[0083] In one possible implementation, for each shutter speed, the shake motion blur and the measured overall blur corresponding to that shutter speed can be determined. The shake motion blur represents the deviation information between the first image sequence and the still image, and the measured overall blur represents the deviation information between the second image sequence and the still image. For example, the shake motion blur and the measured overall blur can be determined using the following steps.

[0084] Step 4041: For each shutter speed (taking one shutter speed as an example), select the first sub-region from all sub-regions of the still image, and select the second sub-region from each frame of the first image sequence corresponding to the shutter speed. The first sub-region and the second sub-region correspond to the same sub-region of the checkerboard.

[0085] See Figure 3 The image shown is a schematic diagram of a chessboard, as described in the preset image above. This chessboard can include multiple sub-regions, such as a total of 30 sub-regions. The sub-region in the first row and first column can be denoted as sub-region (1,1), the sub-region in the first row and second column as sub-region (1,2), the sub-region in the second row and first column as sub-region (2,1), the sub-region in the second row and second column as sub-region (2,2), and so on.

[0086] The still image captured by the camera component can include 30 sub-regions of the chessboard pattern. The size of these 30 sub-regions is similar to that of the preset image, and will not be described again here. In summary, the still image can include multiple black sub-regions and multiple white sub-regions. Each black sub-region is surrounded by a white sub-region, and each white sub-region is surrounded by a black sub-region.

[0087] Obviously, since the still image is the image when the gimbal support is in a stationary state, that is, the image when there is no shaking, the pixel value of each pixel in the black sub-region is the first value (such as 0), and the pixel value of each pixel in the white sub-region is the second value (such as 255).

[0088] For each frame of the first image captured by the camera component, the first image may also include 30 sub-regions of the checkerboard pattern. That is, the first image may include multiple black sub-regions and multiple white sub-regions, with each black sub-region surrounded by a white sub-region, and each white sub-region surrounded by a black sub-region. However, the size of these 30 sub-regions will differ from that of the still image because:

[0089] Since the first image is the image of the gimbal support at the target jitter frequency, that is, the image when jitter occurs, the first image is a blurred image (the jitter causes the first image to be blurred). The pixel values ​​of the pixels in the black sub-region are not all the first value, but are within a range of pixel values. The pixel values ​​of the pixels in the white sub-region are not all the second value, but are also within a range of pixel values.

[0090] Clearly, in this first image, for the black sub-region of the preset image (i.e., the still image), the pixel values ​​of the black sub-region change due to image blurring. Similarly, for the white sub-region of the preset image (i.e., the still image), the pixel values ​​of the white sub-region also change due to image blurring. In other words, the size of the black sub-region in the first image differs from the size of the black sub-region in the still image, and the size of the white sub-region in the first image differs from the size of the white sub-region in the still image. For example, when the size of the white sub-region increases, the size of the black sub-region surrounding the white sub-region decreases, or vice versa.

[0091] For example, targeting Figure 3 The preset image shown is a still image; an example of the first image can be found in [link to example image]. Figure 5 As shown, the size of the white sub-region of the first image is larger than the size of the white sub-region of the still image, and the size of the black sub-region of the first image is smaller than the size of the black sub-region of the still image.

[0092] It should be noted that since the pixel values ​​of the pixels in the black sub-region of the first image are not all of the first value, but fall within a range of pixel values, and the pixel values ​​of the pixels in the white sub-region are not all of the second value, but also fall within a range of pixel values, it is necessary to redefine the black and white sub-regions in the first image in order to distinguish them.

[0093] For example, the black and white sub-regions in the first image can be determined as follows: pixels with pixel values ​​ranging from [minimum pixel value, pixel boundary value] can be assigned to the black sub-region, and pixels with pixel values ​​ranging from (pixel boundary value, maximum pixel value) can be assigned to the white sub-region. That is, the pixel value range of pixels in the black sub-region can be [minimum pixel value, pixel boundary value], and the pixel value range of pixels in the white sub-region can be (pixel boundary value, maximum pixel value).

[0094] After dividing the pixels into black or white sub-regions using the above method, we can obtain... Figure 5 The first image is shown. In the black sub-region of this first image, the pixel value of a pixel is taken as a first value. However, in practical applications, the pixel value of a pixel in the black sub-region is between [minimum pixel value, pixel boundary value], not the first value. Similarly, in the white sub-region of this first image, the pixel value of a pixel is taken as a second value. However, in practical applications, the pixel value of a pixel in the white sub-region is between [pixel boundary value, maximum pixel value], not the second value.

[0095] For example, the minimum pixel value is the minimum value of all pixel values ​​in the first image, the maximum pixel value is the maximum value of all pixel values ​​in the first image, and the pixel boundary value is determined based on the maximum pixel value, such as 20% of the maximum pixel value, 30% of the maximum pixel value, etc., without limitation.

[0096] For example, suppose the minimum pixel value in the first image is 0 and the maximum pixel value is 240. That is, the minimum pixel value is 0 and the maximum pixel value is 240. Then the pixel boundary value could be 48. Therefore, pixels with values ​​in the range [0, 48] are assigned to the black sub-region, and pixels with values ​​in the range (48, 240) are assigned to the white sub-region. Figure 5 The first image shown.

[0097] In step 4041, a first sub-region can be selected from the still image, and a second sub-region can be selected from each frame of the first image. For example, the first sub-region and the second sub-region corresponding to sub-region 33 of the checkerboard pattern can be selected. That is, the first sub-region is the sub-region in the third row and third column of the still image, and the second sub-region is the sub-region in the third row and third column of the first image. Of course, in practical applications, the first sub-region and the second sub-region corresponding to any sub-region of the checkerboard pattern can be selected, and there is no restriction on this.

[0098] For example, the number of first sub-regions can be at least two, and correspondingly, the number of second sub-regions can be at least two. For instance, the first sub-region and the second sub-domain corresponding to sub-region 33 of the chessboard grid are selected, and the first sub-region and the second sub-domain corresponding to sub-region 44 of the chessboard grid are selected.

[0099] In one possible implementation, the first and second sub-regions corresponding to the central sub-region of the chessboard (such as sub-regions 33 and 43) can be selected. Alternatively, the first and second sub-regions corresponding to the white sub-regions of the chessboard can be selected, such as the first and second sub-regions corresponding to the central sub-region and the surrounding white sub-regions (such as sub-regions 43, 32, and 34). Of course, the above are just examples, and the first and second sub-regions can be selected arbitrarily.

[0100] Step 4042: For each shutter speed (taking one shutter speed as an example), select the third sub-region from all sub-regions of the still image, and select the fourth sub-region from each frame of the second image sequence corresponding to that shutter speed. The third and fourth sub-regions correspond to the same sub-region of the checkerboard.

[0101] For each frame of the second image captured by the camera component, it can include multiple black sub-regions and multiple white sub-regions. Each black sub-region is surrounded by a sub-region that is a white sub-region, and each white sub-region is surrounded by a sub-region that is a black sub-region. Furthermore, the black and white sub-regions in the second image are determined as follows: pixels with pixel values ​​ranging from [minimum pixel value, pixel boundary value] are assigned to the black sub-regions, and pixels with pixel values ​​ranging from (pixel boundary value, maximum pixel value) are assigned to the white sub-regions. That is, the pixel value range of pixels in the black sub-regions is [minimum pixel value, pixel boundary value], and the pixel value range of pixels in the white sub-regions is (pixel boundary value, maximum pixel value). For example, the minimum pixel value is the minimum value of all pixel values ​​in the second image, the maximum pixel value is the maximum value of all pixel values ​​in the second image, and the pixel boundary value is determined based on the maximum pixel value.

[0102] Step 4042 is similar to step 4041, except that the first image is replaced with the second image, the first sub-region is replaced with the third sub-region, and the second sub-region is replaced with the fourth sub-region. It will not be described in detail here.

[0103] Step 4043: Determine the amount of blur between the first image and the still image based on the dimensions of the first sub-region and the second sub-region. For example, the dimensions of the first sub-region are the width of the first sub-region, and the dimensions of the second sub-region are the width of the second sub-region. Alternatively, the dimensions of the first sub-region are the height of the first sub-region, and the dimensions of the second sub-region are the height of the second sub-region. Or, the dimensions of the first sub-region are the width and height of the first sub-region, and the dimensions of the second sub-region are the width and height of the second sub-region.

[0104] For example, regarding horizontal jitter, the size of the first sub-region is its width W0, and the size of the second sub-region is its width W1. The absolute value of the difference between width W0 and width W1 can be calculated, and the amount of blur between the first image and the still image can be determined based on this absolute value. For instance, if only one set of sub-regions exists (such as the first sub-region and the second set of regions), the absolute value corresponding to that set of sub-regions is used as the amount of blur between the first image and the still image. If at least two sets of sub-regions exist, the average of the absolute values ​​corresponding to at least two sets of sub-regions is used as the amount of blur between the first image and the still image.

[0105] For example, regarding vertical jitter, the size of the first sub-region is the height H0 of the first sub-region, and the size of the second sub-region is the height H1 of the second sub-region. The absolute value of the difference between the height H0 and the height H1 can be calculated, and the amount of blur between the first image and the still image can be determined based on this absolute value.

[0106] For example, regarding horizontal and vertical jitter, the dimensions of the first sub-region are the width W0 and height H0 of the first sub-region, and the dimensions of the second sub-region are the width W1 and height H1 of the second sub-region. The absolute value 1 of the difference between the width W0 and the width W1 can be calculated, and the absolute value 2 of the difference between the height H0 and the height H1 can be calculated. The square root of the sum of the squares of the absolute values ​​1 and 2 is used to obtain the geometric mean, and the amount of blur between the first image and the still image is determined based on this geometric mean.

[0107] Of course, the above are just a few examples of determining the amount of blur between the first image and the still image. There are no restrictions on the method of determination. The amount of blur is used to represent the size difference between the first image and the still image.

[0108] Step 4044: Determine the amount of blur between the second image and the still image based on the dimensions of the third sub-region and the fourth sub-region. For example, the dimension of the third sub-region is the width of the third sub-region, and the dimension of the fourth sub-region is the width of the fourth sub-region. Alternatively, the dimension of the third sub-region is the height of the third sub-region, and the dimension of the fourth sub-region is the height of the fourth sub-region. Or, the dimension of the third sub-region is the width and height of the third sub-region, and the dimension of the fourth sub-region is the width and height of the fourth sub-region.

[0109] For example, the implementation process of step 4044 is similar to that of step 4043, and will not be repeated here.

[0110] Step 4045: Determine the Shake Motion Blur Amount (SMBA) based on the blur amount between the first image and the still image in each frame of the first image sequence.

[0111] For example, for each frame of the first image in the first image sequence, step 4043 can be used to obtain the blur amount between the first image and the still image, thereby obtaining the blur amount between each frame of the first image and the still image. After obtaining the blur amount between each frame of the first image and the still image, the average value of the blur amount between all the first images and the still images can be calculated, and this average value can be determined as the jitter dynamic blur amount.

[0112] Step 4046: Determine the Measured Comprehensive Bokeh Amount (MCBA) based on the amount of blur between each frame of the second image and the still image in the second image sequence.

[0113] For example, for each frame of the second image sequence, step 4044 can be used to obtain the blur amount between the second image and the still image, thus obtaining the blur amount between each frame of the second image and the still image. After obtaining the blur amount between each frame of the second image and the still image, the average value of the blur amount between all the second images and the still images can be calculated, and this average value can be determined as the measured comprehensive blur amount.

[0114] In summary, based on steps 4041-4046, the jitter dynamic blur amount and the measured comprehensive blur amount can be obtained, and subsequent steps can be performed based on the jitter dynamic blur amount and the measured comprehensive blur amount.

[0115] Step 405: Generate a reference dynamic blur curve based on the shake dynamic blur corresponding to all shutter times, and generate a measured dynamic blur curve based on the measured comprehensive blur corresponding to all shutter times.

[0116] For example, a coordinate system can be established, such as with the bottom left corner as the origin, the horizontal axis to the right as the x-axis, and the vertical axis upwards as the y-axis; or, with the top left corner as the origin, the horizontal axis to the right as the x-axis, and the vertical axis downwards as the y-axis; or, other methods can be used to establish the coordinate system, without restriction. For an example of establishing a coordinate system with the bottom left corner as the origin, the horizontal axis to the right as the x-axis, and the vertical axis upwards as the y-axis, please refer to [link to relevant documentation]. Figure 6 The image shows an example of this coordinate system.

[0117] Based on this coordinate system, the motion blur corresponding to all shutter speeds can be determined by using shutter speed as the x-axis and the corresponding motion blur as the y-axis. A first coordinate point can then be established for each shutter speed within this coordinate system, and a reference motion blur curve can be determined based on all these first coordinate points. For example, see... Figure 6 As shown, the motion blur corresponding to a shutter speed of 1 / 256 second is the first coordinate point corresponding to that shutter speed of 1 / 256 second; the motion blur corresponding to a shutter speed of 1 / 128 second is the same as the first coordinate point corresponding to that shutter speed of 1 / 128 second, and so on, until the motion blur corresponding to a shutter speed of 1 / 2 second is reached. Based on this, the line connecting all the aforementioned first coordinate points can be used as a reference motion blur curve. (See [reference]). Figure 6 As shown.

[0118] Based on this coordinate system, the measured overall blur quantity corresponding to all shutter speeds can be used as the horizontal axis, and the measured overall blur quantity corresponding to that shutter speed can be used as the vertical axis. A second coordinate point can then be determined for each shutter speed within this coordinate system, and the measured dynamic blur quantity curve can be determined based on all these second coordinate points. For example, see... Figure 6 As shown, the measured overall blur amount corresponding to a shutter speed of 1 / 256 second is the second coordinate point corresponding to that shutter speed of 1 / 256 second; the measured overall blur amount corresponding to a shutter speed of 1 / 128 second is the same as the second coordinate point corresponding to that shutter speed of 1 / 128 second, and so on, until the measured overall blur amount corresponding to a shutter speed of 1 / 2 second is the same as the second coordinate point corresponding to that shutter speed of 1 / 2 second. Based on this, the line connecting all the above second coordinate points can be used as the measured dynamic blur amount curve, see [reference]. Figure 6 As shown.

[0119] Step 406: Based on the first intersection of the reference dynamic blur curve and the preset straight line, and the second intersection of the measured dynamic blur curve and the preset straight line, calculate the anti-shake performance of the camera component at the target shaking frequency. The preset straight line can be used to indicate the maximum tolerance value of the anti-shake performance. In other words, the preset straight line can be the straight line indicating the maximum tolerance value of the anti-shake performance (used to reflect the blur amount).

[0120] For example, the Determination Level for Image Stabilization (DLISP) is a blur threshold that can be configured empirically. When the blur level of an image exceeds the DLISP, the human eye will perceive the image as blurry. When the blur level does not exceed the DLISP, the human eye will not perceive the image as blurry. In other words, the DLISP is the critical point at which the human eye perceives blur when viewing an image.

[0121] The maximum tolerance value for image stabilization performance is related to the image resolution. The higher the image resolution, the higher the maximum tolerance value; conversely, the lower the image resolution, the lower the maximum tolerance value. For example, for an image resolution of 2560*1440, the maximum tolerance value for image stabilization performance could be 3.7 pixels, meaning a blur of 3.7 pixels. Of course, 3.7 pixels is just an example and is not a limitation. Maximum tolerance values ​​for image stabilization performance can also be configured for other image resolutions, and there are no limitations on these.

[0122] In summary, the maximum tolerance value for image stabilization can be represented by a straight line. Taking a maximum tolerance value of 3.7 as an example, the maximum tolerance value is used to indicate a straight line with a vertical axis of 3.7. See [link to relevant documentation]. Figure 6 As shown, the straight line with a vertical axis of 3.7 is the aforementioned preset straight line. Clearly, this preset straight line intersects the reference motion blur curve at one point (the first intersection point), and it also intersects the measured motion blur curve at one point (the second intersection point). After obtaining the first and second intersection points, the image stabilization performance of the camera assembly at the target shake frequency can be calculated based on these points.

[0123] In one possible implementation, the image stabilization level can be determined based on the length between the first and second intersection points. For example, the image stabilization level can be the length between the first and second intersection points, see [reference needed]. Figure 6 As shown, the length between the first and second intersection points is 1.7, so the image stabilization level can be 1.7.

[0124] Then, the image stabilization performance of the camera component at the target shake frequency can be determined based on this stabilization level. For example, the image stabilization level can be used to represent the image stabilization performance of the camera component at the target shake frequency; that is, the image stabilization performance can be characterized by this stabilization level. In other words, the image stabilization performance of the camera component at the target shake frequency is stabilization level 1.7. For example, the higher the stabilization level, the better the image stabilization performance; the lower the stabilization level, the worse the image stabilization performance.

[0125] See Figure 6 As shown, assuming the length from point 0 to point 1 on the horizontal axis is length A, then the length from point 1 to point 2 is length A, the length from point 2 to point 3 is length A, and so on. Clearly, the length of any two adjacent points is length A. Since shutter speeds are 1 / 256 second, 1 / 128 second, 1 / 64 second, 1 / 32 second, 1 / 16 second, 1 / 8 second, 1 / 4 second, 1 / 2 second, etc., the shutter speed of point 1 on the horizontal axis is 1 / 256 second, point 2 is 1 / 128 second, point 3 is 1 / 64 second, and so on. This means the difference in shutter speed between two adjacent points is not the same, but rather depends on the shutter speed level.

[0126] Based on this, the length of 1.7 between the first and second intersection points means that the length between the first and second intersection points is 1.7 times the length A, that is, the ratio between the length A and the length A is 1.7.

[0127] In summary, the image stabilization performance, or stabilization level, of the camera component at the target shaking frequency can be calculated based on the first intersection point of the reference motion blur curve and the preset straight line, and the second intersection point of the measured motion blur curve and the preset straight line. Clearly, when the target shaking frequency is 1 Hz, the stabilization level at 1 Hz can be calculated; when the target shaking frequency is 2 Hz, the stabilization level at 2 Hz can be calculated; when the target shaking frequency is 3 Hz, the stabilization level at 3 Hz can be calculated, and so on.

[0128] As can be seen from the above technical solutions, in this embodiment, when the gimbal bracket is at the target shaking frequency and the camera component's image stabilization function is not enabled, a first image sequence containing a checkerboard pattern is acquired; when the gimbal bracket is at the target shaking frequency and the camera component's image stabilization function is enabled, a second image sequence containing a checkerboard pattern is acquired. Based on the above two image sequences, the image stabilization performance of the camera component at the target shaking frequency can be calculated, thereby determining the image stabilization performance of the camera component. The image stabilization performance reflects the image stabilization effect of the camera component, and the image stabilization performance of the camera component is accurately determined. This method can evaluate the static image stabilization effect, making the image stabilization evaluation of the camera component more objective. The above method uses the blur level of a single image as a reference to evaluate the static image stabilization effect, which can improve the safe shutter speed level.

[0129] In one possible implementation, the processor can use a dynamic evaluation method to calculate the image stabilization performance of the camera assembly at the target shake frequency, i.e., to evaluate the dynamic image stabilization effect of the camera assembly. In this implementation, the stability between consecutive frames is evaluated, such as center stability and overall stability, see [link to relevant documentation]. Figure 7 As shown, the image stabilization performance of a camera component can be calculated using the following steps, which include:

[0130] Step 701: When the gimbal bracket is at the preset target jitter frequency and the camera component is not using image stabilization, acquire the third image sequence captured by the camera component.

[0131] Step 702: When the gimbal bracket is at the preset target jitter frequency and the camera component has its image stabilization function enabled, acquire the fourth image sequence captured by the camera component.

[0132] For example, the pan-tilt unit can be controlled to operate at a preset target jitter frequency. This target jitter frequency can be any jitter frequency, such as 1Hz, 2Hz, 3Hz, 4Hz, 5Hz, 6Hz, 7Hz, 8Hz, 9Hz, 10Hz, 11Hz, 12Hz, 13Hz, 14Hz, 15Hz, etc., without limitation. When the pan-tilt unit is operating at the target jitter frequency, the camera assembly is also operating at that target jitter frequency. Since the processing method is the same for each target jitter frequency, the following explanation will use one target jitter frequency as an example.

[0133] In summary, when the gimbal bracket is at the target shaking frequency, the camera assembly can be controlled to not have image stabilization enabled. In this case, a third image sequence a31 at the target shutter speed is acquired. This third image sequence a31 may include multiple frames of third images, such as 200 frames. For example, the target shutter speed can be configured empirically and can be any shutter speed, such as 1 / 256 second, 1 / 128 second, 1 / 64 second, 1 / 32 second, 1 / 16 second, 1 / 8 second, 1 / 4 second, or 1 / 2 second.

[0134] For example, when the gimbal bracket is at the target jitter frequency, the camera component can also be controlled to enable the image stabilization function. In this case, a fourth image sequence a41 at the target shutter speed can be acquired. The fourth image sequence a41 may include multiple fourth images, such as 200 fourth images.

[0135] Step 703: For each point of interest in the chessboard, based on the coordinate position of the point of interest in each frame of the third image sequence and the coordinate position of the point of interest in each frame of the fourth image sequence, determine the center stability of the point of interest. The center stability is used to reflect the image stabilization capability of the camera component at the point of interest, and can reflect the objective and basic performance of the camera component.

[0136] For example, see Figure 3The image shown is a schematic diagram of a chessboard (i.e., the preset image mentioned above). This chessboard includes multiple black sub-regions and multiple white sub-regions. Sub-regions surrounding black sub-regions are all white sub-regions, and vice versa. Multiple points of interest (POIs) can be configured on this chessboard; the number of POIs is not limited and can be configured based on experience.

[0137] Multiple points of interest on the chessboard include the intersection of the four sub-regions of the chessboard, for example, see [link to relevant documentation]. Figure 8 As shown, the intersections of the four sub-regions of the chessboard are displayed. All of these intersections can be configured as points of interest in the chessboard, or only some of these intersections can be configured as points of interest in the chessboard; there is no restriction on this.

[0138] Of course, in practical applications, the intersection of two sub-regions of the chessboard can also be configured as the point of interest of the chessboard, or any pixel of the chessboard can be configured as the point of interest of the chessboard, without any restrictions.

[0139] In step 703, based on the third and fourth image sequences, it is necessary to calculate the center stability of each interest point among all interest points. The calculation process for the center stability of each interest point is the same. Taking an interest point as an example, the center stability of the interest point can be calculated using the following steps:

[0140] Step 7031: Determine the first average coordinate value (mean of coordinate values) of the point of interest based on its coordinate position in each frame of the third image in the third image sequence, and determine the second average coordinate value (mean of coordinate values) of the point of interest based on its coordinate position in each frame of the fourth image in the fourth image sequence.

[0141] For example, a third image sequence may include n frames of third images, such as 200 frames. First, determine the coordinate position of the point of interest in each frame of the third image, such as (x1, y1), (x2, y2), ..., (xn, yn). (x1, y1) represents the coordinate position of the point of interest in the first third image, (x2, y2) represents the coordinate position of the point of interest in the second third image, and so on, with (xn, yn) representing the coordinate position of the point of interest in the nth third image. Based on this, the first mean coordinate (xe, ye) of the point of interest can be calculated using the following formula:

[0142]

[0143] For example, a fourth image sequence may consist of n fourth images, such as 200 fourth images. The coordinates of the point of interest (POI) in each fourth image are determined, such as (x1', y1'), (x2', y2'), ..., (xn', yn'). (x1', y1') represents the coordinates of the POI in the first fourth image, (x2', y2') represents the coordinates in the second fourth image, and so on, with (xn', yn') representing the coordinates in the nth fourth image. Based on this, the second mean coordinate value (xe', ye') of the POI can be calculated. The formula for calculating the second mean coordinate value (xe', ye') is similar to that for the first mean coordinate value (xe, ye), and will not be repeated here.

[0144] Step 7032: Determine the first standard deviation of the point of interest based on the coordinate position of the point of interest in each frame of the third image in the third image sequence and the first mean of the coordinates, and determine the second standard deviation of the point of interest based on the coordinate position of the point of interest in each frame of the fourth image in the fourth image sequence and the second mean of the coordinates.

[0145] For example, the coordinate position of the point of interest in each frame of the third image sequence is (x1, y1), (x2, y2), ..., (xn, yn), and the mean of the first coordinate is (xe, ye). Based on this, the first standard deviation d of the point of interest can be calculated using the following formula, which is a two-dimensional standard deviation.

[0146]

[0147] For example, the coordinates of the point of interest in each frame of the fourth image in the fourth image sequence are (x1', y1'), (x2', y2'), ..., (xn', yn'), and the mean of the second coordinates is (xe', ye'). The second standard deviation d' of the point of interest can be calculated. The formula for calculating the second standard deviation d' is similar to that for calculating the first standard deviation d, and will not be repeated here.

[0148] Step 7033: Determine the center stability of the point of interest based on the first standard deviation and the second standard deviation.

[0149] In one possible implementation, the central stability can be determined using the following formula:

[0150]

[0151] In the above formula, S j d is used to represent the central stability of the j-th point of interest. j d is used to represent the first standard deviation of the j-th point of interest. j 'The second standard deviation used to represent the j-th point of interest.'

[0152] In another possible implementation, the center stability of the interest point can be determined based on the first standard deviation, the second standard deviation, and the image scaling factor λ, for example, using the following formula:

[0153]

[0154] In the above formula, S j d represents the central stability of the j-th interest point. j Let d represent the first standard deviation of the j-th point of interest. j ' represents the second standard deviation of the j-th point of interest, and λ represents the image scaling ratio.

[0155] For example, the image scaling ratio λ represents the ratio between the image captured when the camera component's image stabilization is off and the image captured when the camera component's image stabilization is on. This image scaling ratio λ can be a ratio value configured based on experience or a calculated ratio value. It only needs to be calculated once and stored. For instance, the calculation method for the image scaling ratio λ can include: acquiring an image captured by the camera component with image stabilization off when the gimbal is at a preset target jitter frequency and the camera component's image stabilization is off; acquiring an image captured by the camera component with image stabilization on when the gimbal is at the preset target jitter frequency and the camera component's image stabilization is on; and determining the image scaling ratio λ based on the checkerboard height in the image with image stabilization off and the checkerboard height in the image with image stabilization on; or, determining the image scaling ratio λ based on the checkerboard width in the image with image stabilization off and the checkerboard width in the image with image stabilization on.

[0156] For example, the checkerboard height can represent the height of the checkerboard within the visible area of ​​the camera component in an image with image stabilization off or on. And the checkerboard width can represent the width of the checkerboard within the visible area of ​​the camera component in an image with image stabilization off or on.

[0157] In step 701, when the gimbal bracket is at the target jitter frequency and the camera component is not using image stabilization, in addition to acquiring the third image sequence, the camera component can also capture one frame of image, which is recorded as the image with image stabilization off (Poff), i.e., Poff is acquired. In step 702, when the gimbal bracket is at the target jitter frequency and the camera component is using image stabilization, in addition to acquiring the fourth image sequence, the camera component can also capture one frame of image, which is recorded as the image with image stabilization on (Pon), i.e., Pon is acquired.

[0158] In one possible implementation, the height 1 of the checkerboard pattern (i.e., the preset image) in Poff and the height 2 of the checkerboard pattern in Pon can be determined. Since both Poff and Pon are captured at the target jitter frequency, meaning both Poff and Pon suffer from image blurring, height 1 and height 2 are different from the actual height of the checkerboard pattern. Furthermore, since Poff is captured without image stabilization and Pon is captured with image stabilization enabled, height 1 and height 2 are different. Based on this, the ratio of height 1 to height 2 can be determined as the image scaling ratio λ.

[0159] In another possible implementation, the width 1 of the checkerboard pattern in Poff and the width 2 of the checkerboard pattern in Pon can be determined. Width 1 is different from the width of the checkerboard pattern itself, and width 2 is also different from the width of the checkerboard pattern itself. Poff is when image stabilization is not enabled, and Pon is when image stabilization is enabled, i.e., width 1 and width 2 are different. The ratio of width 1 to width 2 is determined as the image scaling ratio λ.

[0160] In summary, the central stability of each point of interest can be determined, denoted as central stability S. j .

[0161] Step 704: Determine the overall stability based on the center stability of all points of interest. This overall stability reflects the stabilization capability of the camera components across the entire frame. For example, this overall stability is a quantification of the subjective evaluation of the image by the human eye, taking into account the subjective effects caused by lens distortion, rolling shutter effect, image distortion, and drift caused by motion sensor noise during the stabilization process.

[0162] In one possible implementation, the overall stability can be obtained by weighting the central stability of all interest points (POCs) together with the central stability of each POC and its corresponding weight coefficient. For example, the overall stability can be calculated using the following formula:

[0163] S whole =S1*p1+S2*p2…+S k *p k

[0164] Assuming there are k points of interest, where j ranges from 1 to k, then S1 represents the central stability of the first point of interest, p1 represents the weight coefficient corresponding to the first point of interest, S2 represents the central stability of the second point of interest, p2 represents the weight coefficient corresponding to the second point of interest, and so on. k p represents the center stability of the k-th point of interest. k S represents the weight coefficient corresponding to the k-th point of interest. wholeIndicates overall stability.

[0165] In one possible implementation, the weight coefficients corresponding to each point of interest can be the same, i.e., the aforementioned weight coefficients p1, p2, ..., p... k They can be the same.

[0166] In another possible implementation, for each point of interest, the smaller the distance between the point of interest and the center of the chessboard, the larger the weight coefficient of the point of interest; conversely, the larger the distance between the point of interest and the center of the chessboard, the smaller the weight coefficient of the point of interest.

[0167] Of course, the above are just examples of the weight coefficients corresponding to each point of interest. There are no restrictions on this. You can configure the weight coefficients corresponding to each point of interest based on experience, and the weight coefficients can be values ​​between 0 and 1.

[0168] Step 705: Calculate the image stabilization performance of the camera component at the target shake frequency based on the overall stability. For example, the overall stability can be used to represent the image stabilization performance of the camera component at the target shake frequency; that is, the image stabilization performance can be characterized by the overall stability. For instance, the larger the overall stability, the better the image stabilization performance; the smaller the overall stability, the worse the image stabilization performance.

[0169] Obviously, when the target jitter frequency is 1 Hz, the overall stability at 1 Hz can be calculated; when the target jitter frequency is 2 Hz, the overall stability at 2 Hz can be calculated; when the target jitter frequency is 3 Hz, the overall stability at 3 Hz can be calculated, and so on.

[0170] As can be seen from the above technical solutions, in this embodiment, when the gimbal bracket is at the target jitter frequency and the camera component's image stabilization function is not enabled, a third image sequence containing a checkerboard pattern is acquired; when the gimbal bracket is at the target jitter frequency and the camera component's image stabilization function is enabled, a fourth image sequence containing a checkerboard pattern is acquired. Based on the above two image sequences, the image stabilization performance of the camera component at the target jitter frequency can be calculated, thereby determining the image stabilization performance of the camera component. The image stabilization performance reflects the image stabilization effect of the camera component, and the image stabilization performance of the camera component is accurately determined. This allows for the evaluation of dynamic image stabilization effects, making the image stabilization evaluation of the camera component more objective.

[0171] In one possible implementation, the processor can calculate the image stabilization performance of the camera assembly at the target shake frequency using both static and dynamic evaluation methods, i.e., assessing the static and dynamic image stabilization effects of the camera assembly. In this implementation, it can employ... Figure 4The process shown calculates the image stabilization level of the camera components at the target shake frequency, and uses... Figure 7 The illustrated process calculates the overall stability of the camera assembly at the target shake frequency, and determines the image stabilization performance of the camera assembly at the target shake frequency based on the image stabilization level and the overall stability. In other words, image stabilization performance can be characterized by the image stabilization level and the overall stability. Clearly, a higher image stabilization level results in better image stabilization performance, and a lower level results in worse performance. Similarly, a higher overall stability level results in better image stabilization performance, and a lower overall stability level results in worse performance.

[0172] As can be seen from the above technical solutions, in this embodiment, the image stabilization performance of the camera component at the target shaking frequency can be calculated based on two image sequences, thereby revealing the image stabilization performance of the camera component. This performance reflects the image stabilization effect of the camera component and allows for accurate assessment. It enables the evaluation of both static and dynamic image stabilization effects, making the image stabilization evaluation of the camera component more objective. In other words, static evaluation assesses the image stabilization effect of a single frame while also performing dynamic image stabilization, resulting in a more objective image stabilization evaluation.

[0173] Based on the same concept as the above method, this application proposes a device for calculating camera stabilization performance. The gimbal bracket supports the camera assembly, allowing it to rotate horizontally and vertically at least once. The visible area of ​​the camera assembly covers the size of a preset image containing only a checkerboard pattern, and the image captured by the camera assembly includes the checkerboard pattern. See [link to relevant documentation]. Figure 9A The diagram shown is a structural schematic of the device, which may include:

[0174] The acquisition module 911 is used to acquire a still image when the gimbal support is in a stationary state;

[0175] When the gimbal bracket is at a preset target shaking frequency and the camera component is not stabilizing, acquire the first image sequence corresponding to each of the multiple shutter times;

[0176] When the gimbal bracket is at the preset target shaking frequency and the camera component has its image stabilization function enabled, the second image sequence corresponding to each of the multiple shutter times is acquired.

[0177] The determination module 912 is used to determine the amount of motion blur corresponding to each shutter speed based on the first image sequence corresponding to that shutter speed and the still image; and to determine the measured overall blur amount corresponding to that shutter speed based on the second image sequence corresponding to that shutter speed and the still image.

[0178] The generation module 913 is used to generate a reference dynamic blur curve based on the dynamic blur amount corresponding to all shutter speeds, and to generate a measured dynamic blur curve based on the measured comprehensive blur amount corresponding to all shutter speeds; the calculation module 914 is used to calculate the image stabilization performance of the camera component at the target shake frequency based on the first intersection point of the reference dynamic blur curve and the preset straight line, and the second intersection point of the measured dynamic blur curve and the preset straight line; wherein, the preset straight line is used to indicate the maximum tolerance value of the image stabilization performance.

[0179] Based on the same concept as the above method, this application proposes a device for calculating camera stabilization performance. The gimbal bracket supports the camera assembly, allowing it to rotate horizontally and vertically at least once. The visible area of ​​the camera assembly covers the size of a preset image containing only a checkerboard pattern, and the image captured by the camera assembly includes the checkerboard pattern. See [link to relevant documentation]. Figure 9B The diagram shown is a structural schematic of the device, which may include:

[0180] The acquisition module 921 is used to acquire a third image sequence captured by the camera component when the gimbal bracket is at a preset target jitter frequency and the camera component is not equipped with the anti-shake function.

[0181] And, when the gimbal bracket is at the preset target jitter frequency and the camera component has its image stabilization function enabled, the fourth image sequence captured by the camera component is acquired;

[0182] The determining module 922 is used to determine the center stability of each of the multiple points of interest in the chessboard, based on the coordinate position of the point of interest in each frame of the third image sequence and the coordinate position of the point of interest in each frame of the fourth image sequence; wherein the center stability is used to reflect the image stabilization capability value of the camera component at the point of interest.

[0183] Furthermore, the overall stability is determined based on the center stability of the plurality of points of interest; wherein the overall stability is used to reflect the image stabilization capability of the camera component across the entire frame;

[0184] The calculation module 923 is used to calculate the image stabilization performance of the camera component at the target shaking frequency based on the overall stability; wherein, the greater the overall stability, the better the image stabilization performance.

[0185] Based on the same concept as the methods described above, embodiments of this application also provide a machine-readable storage medium storing a plurality of computer instructions. When executed by a processor, these computer instructions can implement the method for calculating camera stabilization performance disclosed in the examples above. The machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device, and can contain or store information such as executable instructions, data, etc. For example, the machine-readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid-state drive, any type of storage disk (such as optical disc, DVD, etc.), or similar storage media, or combinations thereof.

[0186] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0187] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0188] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0189] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0190] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0191] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0192] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A system for calculating the image stabilization performance of a camera, characterized in that, include: Camera components; A pan-tilt bracket that can support the camera assembly to allow the camera assembly to rotate horizontally and vertically at least one of the two. A preset image containing only a checkerboard pattern is set within the visible area of ​​the camera component; The camera component is defined such that its visible area covers the size of the preset image, so that the image captured by the camera component includes the checkerboard pattern; Processor, the processor being configured to perform: When the gimbal bracket is at a preset target jitter frequency and the camera component is not stabilizing, the third image sequence captured by the camera component is obtained. When the gimbal bracket is at the preset target jitter frequency and the camera component has its image stabilization function enabled, the fourth image sequence captured by the camera component is acquired. For each of the multiple points of interest in the chessboard, the center stability of the point of interest is determined based on its coordinate position in each frame of the third image in the third image sequence and its coordinate position in each frame of the fourth image in the fourth image sequence; wherein, the center stability is used to reflect the image stabilization capability of the camera component at the point of interest. The overall stability is determined based on the center stability of the multiple points of interest; wherein, the overall stability is used to reflect the image stabilization capability of the camera assembly across the entire frame; The image stabilization performance of the camera component at the target shaking frequency is calculated based on the overall stability; wherein, the greater the overall stability, the better the image stabilization performance. The step of determining the center stability of the interest point based on its coordinate position in each frame of the third image in the third image sequence and its coordinate position in each frame of the fourth image in the fourth image sequence includes: determining a first mean of the coordinates of the interest point based on its coordinate position in each frame of the third image in the third image sequence, and determining a first standard deviation of the interest point based on its coordinate position in each frame of the third image in the third image sequence and the first mean of the coordinates; determining a second mean of the coordinates of the interest point based on its coordinate position in each frame of the fourth image in the fourth image sequence, and determining a second standard deviation of the interest point based on its coordinate position in each frame of the fourth image in the fourth image sequence and the second mean of the coordinates; and determining the center stability of the interest point based on the first standard deviation and the second standard deviation. The step of determining the center stability of the point of interest based on the first standard deviation and the second standard deviation includes: determining the center stability of the point of interest based on the first standard deviation, the second standard deviation and the image scaling ratio; wherein the image scaling ratio represents the ratio between the image captured when the camera component does not have image stabilization enabled and the image captured when the camera component has image stabilization enabled.

2. The system according to claim 1, characterized in that, The pan-tilt bracket includes a base, a horizontal rotating platform, and a vertical rotating platform, with the camera assembly fixed to the vertical rotating platform; wherein: The base is used to fix the pan-tilt bracket and drive the horizontal rotating platform; the horizontal rotating platform is used to rotate horizontally to make the camera assembly rotate horizontally and drive the vertical rotating platform; the vertical rotating platform is used to rotate vertically to make the camera assembly rotate vertically.

3. The system according to claim 1, characterized in that, The methods for determining the image scaling ratio include: When the gimbal bracket is at the preset target jitter frequency and the camera component is not stabilizing, acquire an image with stabilization disabled by the camera component; When the gimbal bracket is at the preset target shaking frequency and the camera component has its stabilization function enabled, the stabilization image captured by the camera component is obtained. The image scaling ratio is determined based on the checkerboard height in the image with stabilization off and the checkerboard height in the image with stabilization on; wherein, the checkerboard height represents the height of the checkerboard within the visible area of ​​the camera component in the image with stabilization off or the image with stabilization on.

4. The system according to claim 1, characterized in that, The determination of the center stability of the point of interest based on the first standard deviation, the second standard deviation, and the image scaling ratio includes: The central stability of the point of interest is determined by the following formula: Among them, S j d represents the central stability of the j-th interest point. j Let d represent the first standard deviation of the j-th point of interest. j ' represents the second standard deviation of the j-th point of interest, and λ represents the image scaling ratio.

5. The system according to claim 1, characterized in that, The determination of overall stability based on the central stability of the multiple points of interest includes: The overall stability is obtained by weighting the central stability of each interest point and the corresponding weight coefficient. Specifically, for each interest point, the smaller the distance between the interest point and the center of the chessboard, the larger the weight coefficient of the interest point.

6. The system according to any one of claims 1-5, characterized in that, The chessboard grid includes multiple black sub-regions and multiple white sub-regions, with each black sub-region surrounded by a white sub-region, and each white sub-region surrounded by a black sub-region; wherein, the multiple points of interest of the chessboard grid include the intersection of the four sub-regions of the chessboard grid.

7. A method for calculating the image stabilization performance of a camera, characterized in that, A pan-tilt bracket supports a camera assembly, allowing the camera assembly to rotate horizontally and vertically at least once; the visible area of ​​the camera assembly covers the size of a preset image containing only a checkerboard pattern, and the image captured by the camera assembly includes the checkerboard pattern; the method includes: When the gimbal bracket is at a preset target jitter frequency and the camera component is not stabilizing, the third image sequence captured by the camera component is obtained. When the gimbal bracket is at the preset target jitter frequency and the camera component has its image stabilization function enabled, the fourth image sequence captured by the camera component is acquired. For each of the multiple points of interest in the chessboard, the center stability of the point of interest is determined based on its coordinate position in each frame of the third image in the third image sequence and its coordinate position in each frame of the fourth image in the fourth image sequence; wherein, the center stability is used to reflect the image stabilization capability of the camera component at the point of interest. The overall stability is determined based on the center stability of the multiple points of interest; wherein, the overall stability is used to reflect the image stabilization capability of the camera assembly across the entire frame; The image stabilization performance of the camera component at the target shaking frequency is calculated based on the overall stability; wherein, the greater the overall stability, the better the image stabilization performance. The step of determining the center stability of the interest point based on its coordinate position in each frame of the third image in the third image sequence and its coordinate position in each frame of the fourth image in the fourth image sequence includes: determining a first mean of the coordinates of the interest point based on its coordinate position in each frame of the third image in the third image sequence, and determining a first standard deviation of the interest point based on its coordinate position in each frame of the third image in the third image sequence and the first mean of the coordinates; determining a second mean of the coordinates of the interest point based on its coordinate position in each frame of the fourth image in the fourth image sequence, and determining a second standard deviation of the interest point based on its coordinate position in each frame of the fourth image in the fourth image sequence and the second mean of the coordinates; and determining the center stability of the interest point based on the first standard deviation and the second standard deviation. The step of determining the center stability of the point of interest based on the first standard deviation and the second standard deviation includes: determining the center stability of the point of interest based on the first standard deviation, the second standard deviation and the image scaling ratio; wherein the image scaling ratio represents the ratio between the image captured when the camera component does not have image stabilization enabled and the image captured when the camera component has image stabilization enabled.

Images