Electronic image stabilization control method, program product, electronic device and storage medium

Abstract

The invention provides an electronic image stabilization control method, a program product, electronic equipment and a storage medium, and the method comprises the steps: converting a current frame image based on a homography matrix corresponding to the current frame image, and obtaining a first image region; determining whether a preset cutting area in the current frame image is completely located in the first image area or not; in response to the fact that the preset cutting area is not completely located in the first image area, adjusting the homography matrix; and carrying out conversion again based on the adjusted homography matrix until the preset cutting area is completely located in the first image area, and carrying out cutting based on the preset cutting area to obtain a stable image corresponding to the current frame image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an electronic image stabilization control method, program product, electronic device and storage medium. Background Technology

[0002] Electronic image stabilization (EIS) is a method for obtaining stable and clear video sequences in shaky environments. EIS methods mainly include image-based methods and gyroscope-based methods. However, regardless of the method, achieving video sequence stabilization inevitably results in the loss of some field of view. This is because stabilization methods require the target's position relative to the previous frame to remain as unchanged as possible. In motion, some targets in the previous frame will not be present in the current frame. After perspective transformation or other transformations, the current frame's image is mapped to the positions of the previous frame's image. The areas mapped from the current frame to targets that were not present in the previous frame will lack targets, thus forming black borders.

[0003] To address the aforementioned shortcomings, related technologies generally employ mosaic and motion inpainting methods for image inpainting. These methods involve finding the missing target in the current frame or other frames and filling in the missing target area. However, these methods are time-consuming and exhibit significant instability. Summary of the Invention

[0004] To address the existing technical problems, this application provides an electronic image stabilization control method, program product, electronic device, and storage medium that can effectively improve electronic image stabilization efficiency and image stability.

[0005] In a first aspect, embodiments of this application provide an electronic image stabilization control method, comprising: transforming the current frame image based on the homography matrix corresponding to the current frame image to obtain a first image region; determining whether a preset cropping region in the current frame image is entirely located within the first image region; adjusting the homography matrix in response to the fact that the preset cropping region is not entirely located within the first image region; transforming again based on the adjusted homography matrix until the preset cropping region is entirely located within the first image region; and cropping based on the preset cropping region to obtain a stable image corresponding to the current frame image.

[0006] In a second aspect, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the electronic image stabilization control method as described in any embodiment of this application.

[0007] Thirdly, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the electronic image stabilization control method as described in any embodiment of this application.

[0008] Fourthly, a storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the electronic image stabilization control method as described in any embodiment of this application.

[0009] In the electronic image stabilization control method provided in the above embodiments, it is determined whether the preset cropping region is entirely located within the first image region. In response to the preset cropping region not being entirely located within the first image region, the homography matrix is ​​adjusted, and a new first image region is obtained based on the adjusted homography matrix. This process continues until the preset cropping region is entirely located within the first image region. Then, cropping is performed based on the preset cropping region to obtain a stable image corresponding to the current frame image. Thus, by adjusting the homography matrix, the first image region mapped from the current frame image to the previous frame image can be adjusted. Therefore, when the entire cropping region falls within this first image region, the resulting stable image of the current frame will not contain mapped targets that do not exist relative to the previous frame image, thus avoiding black borders. Furthermore, there is no need to search for and fill missing targets from the current frame or other frames, resulting in a faster response speed and improved stability and efficiency of electronic image stabilization.

[0010] The computer program product, electronic device and storage medium provided in the above embodiments belong to the same concept as the corresponding electronic image stabilization control method embodiments, and thus have the same technical effects as the corresponding electronic image stabilization control method embodiments, which will not be repeated here. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating an embodiment of the electronic image stabilization control method of this application; Figure 2 This is a schematic diagram of a preset cropping area B and a first image area C in one embodiment of this application; Figure 3 This is a schematic diagram of a preset cropping area B and a first image area C in another embodiment of this application; Figure 4 This is a schematic diagram of a corner point located outside the first image area in one embodiment of this application; Figure 5A This is a schematic diagram of the image effect before applying the electronic image stabilization control method in one embodiment of this application; Figure 5B This is a schematic diagram of the image effect after applying the electronic image stabilization control method in one embodiment of this application; Figure 6A This is a schematic diagram of the image effect before applying the electronic image stabilization control method in another embodiment of this application; Figure 6B This is a schematic diagram of the image effect after applying the electronic image stabilization control method in another embodiment of this application; Figure 7 This is a flowchart illustrating the electronic image stabilization control method in another embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device in one embodiment of this application. Detailed Implementation

[0012] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] In the following description, the phrase "some embodiments" refers to a subset of all possible embodiments. It should be noted that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0015] In the following description, the terms "first," "second," and "third" are used merely to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that "first," "second," and "third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0016] like Figure 1 As shown, one embodiment of this application provides an electronic image stabilization control method, including: S10: The current frame image is transformed based on the homography matrix corresponding to the current frame image to obtain the first image region.

[0017] In one embodiment, the current frame image can be a frame image in a video sequence, that is, a frame image that needs to be electronically stabilized. The homography matrix represents the perspective transformation relationship between the current frame image and the mapping region, and can be used to perspective transform multiple corner points of the current frame image into multiple corner points of the mapping region, where the mapping region is the first image region.

[0018] In one embodiment, transforming the current frame image can refer to performing perspective transformation on the current frame image. For example, performing perspective transformation on multiple corner points of the current frame image. For instance, if the boundary of the current frame image is a rectangle, then the corner points are the four corner points of the rectangle.

[0019] In one embodiment, the first image region can also be a quadrilateral. For example, at least one corner point of the first image region is obtained by translating the corresponding corner point in the current frame image. The translation amount (i.e., translation distance) can be determined based on the homography matrix corresponding to the current frame image.

[0020] S20: Determine whether the preset cropping region in the current frame image is entirely located within the first image region.

[0021] In one embodiment, the preset cropping region can be a portion of the current frame image; for example, the preset cropping region can also be a rectangle.

[0022] For example, such as Figure 2 The diagram shows a pre-defined cropping region entirely within the first image region. The pre-defined cropping region (region B) is a portion of the current frame image (region A) whose boundary is proportionally scaled down. The first image region (region C) may not completely overlap with the pre-defined cropping region, or it may at least partially overlap with it. The geometric center of the pre-defined cropping region may or may not coincide with the geometric center of the current frame image.

[0023] For example, such as Figure 3 The diagram shows that the preset cropping area B is not entirely located within the first image area C.

[0024] In one embodiment, determining whether the preset cropping region in the current frame image is entirely located within the first image region may include: determining whether all corner points of the preset cropping region in the current frame image are entirely located within the first image region.

[0025] That is, when all corner points of the preset cropping area in the current frame image are located within the first image area, it is determined that the preset cropping area in the current frame image is located within the first image area; when at least one corner point of the preset cropping area in the current frame image is not located within the first image area, it is determined that the preset cropping area in the current frame image is not located within the first image area.

[0026] Here, there is no area located within the first image region, meaning that part or all of the area is located outside the first image region.

[0027] S30: In response to the fact that the preset cropping area is not entirely located within the first image area, adjust the homography matrix.

[0028] In one embodiment, the preset cropping area is not entirely located within the first image area; it can mean that at least one corner point in the preset cropping area is located outside the first image area.

[0029] In one embodiment, adjusting the homography matrix may include adjusting the translation amount in the homography matrix, such as adjusting the translation amount in a first direction and / or a second direction. The first direction may be the x-axis direction, and the second direction may be the y-axis direction.

[0030] In one embodiment, transforming the current frame image based on the homography matrix corresponding to the current frame image to obtain a first image region may include: translating the start point and end point of the current frame image in the first direction based on at least one translation amount in the first direction of the homography matrix to obtain the start point and end point of the first image region in the first direction; and / or translating the start point and end point of the current frame image in the second direction based on at least one translation amount in the second direction of the homography matrix to obtain the start point and end point of the first image region in the second direction.

[0031] The starting point and the ending point are called corner points. For example, the corner points of the current frame image include the starting point and the ending point in the first direction, as well as the starting point and the ending point in the second direction.

[0032] S40: Based on the adjusted homography matrix, the image is re-transformed until the preset cropping area is entirely within the first image area. Then, cropping is performed based on the preset cropping area to obtain a stable image corresponding to the current frame image.

[0033] In one embodiment, a new first image region is obtained by re-transforming based on the adjusted homography matrix, and it is further determined whether the preset cropping region in the current frame image is entirely located within the first image region; in response to the preset cropping region not being entirely located within the first image region, the homography matrix is ​​further adjusted, and a re-transformation is performed based on the adjusted homography matrix until the preset cropping region is entirely located within the first image region.

[0034] In one embodiment, cropping based on the preset cropping region to obtain a stable image corresponding to the current frame image can mean cropping out the preset cropping region from the current frame image as the stable image corresponding to the current frame image. Here, the preset cropping region is the entire preset cropping region located within the first image region that is ultimately obtained.

[0035] In one embodiment, the process of ensuring that the preset cropping area is entirely within the first image area may include: ensuring that the preset cropping area is entirely within the first image area until the number of adjustments reaches a preset number or before the preset number of adjustments is reached.

[0036] The number of adjustments refers to the number of times the homography matrix is ​​adjusted.

[0037] In this way, by adjusting the homography matrix, the first image region mapped from the current frame image to the previous frame image can be adjusted. When the entire area to be cropped falls into this first image region, the cropped stable image of the current frame will not contain the mapped target that does not exist relative to the previous frame image, so there will be no black border. Moreover, there is no need to search for and fill the missing target from the current frame or other frames, resulting in a faster response speed and thus improving the stability and efficiency of electronic image stabilization.

[0038] In some embodiments, the method may further include: In response to the fact that the preset cropping area is entirely located within the first image area, cropping is performed based on the preset cropping area to obtain a stable image corresponding to the current frame image.

[0039] In one embodiment, if the preset cropping area is entirely within the first image area, then there is no need to adjust the homography matrix, and a stable image can be obtained directly by cropping.

[0040] Specifically, cropping based on the preset cropping region to obtain the stable image corresponding to the current frame image can mean cropping out the preset cropping region from the current frame image as the stable image corresponding to the current frame image. Here, the preset cropping region is the entire preset cropping region located within the first image region mentioned above.

[0041] In this way, by setting a fixed preset cropping area, there is no need to frequently change the cropping reference. The homography matrix can be flexibly adjusted to change the mapping area so that the cropping area meets the requirements of mapping black edge removal. When the requirements are met, a stable image can be directly cropped, thus improving the efficiency of electronic image stabilization and black edge removal.

[0042] In some embodiments, determining whether the preset cropping region in the current frame image is entirely located within the first image region may include: Determine the first vector corresponding to each edge of the first image region; Determine a second vector corresponding to the starting point of an edge from each corner point of the preset cropping area to the first image area; wherein the starting point is an endpoint of the edge determined based on the vector direction of the first vector; Calculate the cross product of the second vector from the corner point to an edge and the first vector corresponding to the edge; If the cross product of the corner point and each edge is greater than 0, then the corner point is determined to be located within the first image region; In response to the fact that all corner points are located within the first image area, it is determined that the preset cropping area in the current frame image is entirely located within the first image area.

[0043] In one embodiment, determining the first vector corresponding to each edge of the first image region may include: determining the first vector corresponding to each edge based on the coordinates of the corner points corresponding to each edge in the first image region. Here, the corner points corresponding to each edge are the two endpoints of each edge.

[0044] The vector direction can be either clockwise or counterclockwise, depending on the first image region.

[0045] In one embodiment, for each corner point of the preset cropping region, based on the second vector formed by the corner point to the starting point of each edge in the first image region, the cross product of each second vector and the first vector of the corresponding edge is calculated. When the cross product of the corner point and each edge is positive, the corner point is determined to be located within the first image region. Here, the corresponding edge is the edge corresponding to the starting point of the formed second vector.

[0046] In this context, the starting point of an edge in the first image region refers to the starting point of the edge as determined by the direction of the vector. For example, for an edge extending along the first direction, the starting point is the starting point of the edge in the first direction, and for an edge extending along the second direction, the starting point is the starting point of the edge in the second direction.

[0047] For example, such as Figure 4 As shown, taking the clockwise direction as the vector direction, for a corner point a in the preset cropping area, the edge closest to corner point a in the first image area is c, and the starting point of c is point b. At this time, the cross product between the second vector from corner point a to point b and the first vector corresponding to edge c is less than 0, so corner point a is outside the first image area.

[0048] In one embodiment, all corner points are located within the first image region, meaning that the cross product of all corner points and each edge in the first image region is greater than 0.

[0049] Thus, the method based on vector cross product can accurately and efficiently calculate whether each corner point of the preset cropping area is located within the first image area. Therefore, when all corner points are located within the first image area, it can be determined that the preset cropping area is entirely located within the first image area, which can improve the accuracy and efficiency of the calculation.

[0050] In some embodiments, adjusting the homography matrix may include: The target corner point is determined to be one that does not exist within the first image area in the preset cropping area; Determine the distance between the target corner point and the nearest edge to the target corner point in the first image region; In response to the nearest edge being the left edge, the distance is subtracted from the first translation amount in the first direction of the homography matrix; In response to the nearest edge being the right edge, the distance is added to the first translation in the first direction of the homography matrix; the first direction is the direction from the left edge to the right edge. In response to the nearest edge being the top edge, the distance is subtracted from the second translation amount in the second direction of the homography matrix. In response to the nearest edge being the bottom edge, the second translation amount in the second direction of the homography matrix is ​​added to the distance; the second direction is the direction from the top edge to the bottom edge.

[0051] In one embodiment, the target corner point is a corner point in the preset cropping area that is not located within the first image area. There may be one or more target corner points.

[0052] In one embodiment, for each target corner point, the distance from the target corner point to the nearest edge in the first image region is determined. The nearest edge in the first image region to the target corner point can refer to the shortest perpendicular distance from the target corner point to that edge, or it can refer to the shortest distance between the target corner point and the starting point of that edge, etc. Here, the distance is the absolute value of the distance, which is a value greater than or equal to 0.

[0053] In one embodiment, the first image region may be a quadrilateral, including a left side located on the left, a right side located on the right, an upper side located on the top, and a lower side located on the bottom, wherein the left side, right side, upper side, and lower side together constitute the first image region.

[0054] In one embodiment, the first direction is the x-axis direction, and the left and right sides are the two sides of the first image region in the first direction. When the nearest side is the left side, the first translation amount in the homography matrix in the first direction is subtracted from the distance between the target corner point and the left side, and the result is used as the first translation amount of the adjusted homography matrix in the first direction. When the nearest side is the right side, the first translation amount in the homography matrix in the first direction is added to the distance between the target corner point and the right side, and the result is used as the first translation amount of the adjusted homography matrix in the first direction.

[0055] In one embodiment, the second direction is the y-axis direction. The top and bottom edges are the two sides of the first image region in the second direction, respectively. When the nearest edge is the top edge, the second translation amount in the homography matrix in the second direction is subtracted from the distance between the target corner point and the top edge, and the result is used as the adjusted second translation amount of the homography matrix in the second direction. When the nearest edge is the bottom edge, the second translation amount in the homography matrix in the second direction is added to the distance between the target corner point and the bottom edge, and the result is used as the adjusted second translation amount of the homography matrix in the second direction.

[0056] Thus, based on the distance between the corner point located outside the first image region and the first image region, the translation amount in the corresponding direction in the homography matrix is ​​adjusted. This allows for targeted adjustment of the homography matrix according to the portion of the preset cropping region located outside the first image region, thereby adjusting the converted first image region so that the first image region can contain the entire preset cropping region, improving the accuracy of image stabilization and increasing efficiency.

[0057] For example, such as Figure 5A and 5B The images shown are schematic diagrams of the image stabilization control method before and after applying the method that includes adjusting the homography matrix. It can be seen that the black borders are significantly removed. Since this effect cannot be expressed by line drawings or other means, this comparison image is used to represent the above effect.

[0058] In special circumstances, such as when the video sequence containing the image is subject to excessive jitter or is in a scene with large-scale movement, it may be impossible to adjust the preset cropping area to be entirely within the first image area. In such cases, continuously adjusting the homography matrix will affect the efficiency of electronic image stabilization.

[0059] Therefore, in some embodiments, the re-transformation based on the adjusted homography matrix until the entire preset cropping region is located within the first image region includes: Based on the adjusted homography matrix, if the preset cropping area is still not entirely within the first image area, the homography matrix is ​​adjusted again until the preset number of adjustments is reached and the preset cropping area is still not entirely within the first image area. The first translation amount and the second translation amount in the last adjusted homography matrix are then obtained. The coordinates of the preset cropping area are corrected based on the first translation amount and the second translation amount, so that the preset cropping area is entirely located within the first image area.

[0060] Here, the preset number of times can be a fixed value, such as 5 times, 10 times, 15 times, etc. Alternatively, the preset number of times can also be adjusted based on the jitter level of the video sequence in which the current frame image is located. For example, the jitter level of the video sequence is determined based on the current frame image and the previous frame image. The higher the jitter level, the larger the preset number of times, thus providing more times to adjust the homography matrix.

[0061] In one embodiment, the homography matrix after the last adjustment is the homography matrix obtained after the final adjustment when the number of adjustments reaches a preset number. Here, the number of adjustments refers to the number of times the homography matrix has been adjusted.

[0062] In one embodiment, the first translation amount is the first translation amount in the first direction, and the second translation amount is the second translation amount in the second direction.

[0063] Thus, when the range of motion is large, it is impossible to adjust the preset cropping area to be entirely within the first image area. In this case, the preset number of adjustments can be used to avoid continuously adjusting for a long time, which would affect the image effect. Alternatively, the coordinates of the preset cropping area can be adjusted based on the homography matrix of the last adjustment (that is, the one closest to the preset cropping area being entirely within the first image area) instead of adjusting the boundary of the first image area, so that the area to be cropped can be entirely within the first image area.

[0064] In some embodiments, correcting the coordinates of the preset cutting area based on the first translation amount and the second translation amount may include: In response to the first translation amount being positive, the coordinates of the first starting point of the preset cutting area are adjusted to the first translation amount; In response to the first translation amount being negative, the coordinates of the first end point of the preset cropping region are adjusted to the sum of the first translation amount and the width of the current frame image; wherein the first start point and the first end point are the two endpoints of the preset cropping region in the first direction, and the width is the dimension along the first direction.

[0065] In one embodiment, if the first translation amount in the homography matrix after the last adjustment is greater than 0, then the coordinates of the first starting point of the preset cropping region in the first direction are adjusted to the first translation amount. If the first translation amount in the homography matrix after the last adjustment is less than 0, then the coordinates of the first ending point of the preset cropping region in the first direction are adjusted to the sum of the first translation amount and the width of the current frame image.

[0066] Here, the first starting point is the starting point of the preset clipping region in the first direction, such as the x-axis, and the first ending point is the ending point of the preset clipping region in the first direction, such as the x-axis. For example, when the vector direction is clockwise, the starting point is the endpoint on the left, which is the point with the smallest x-axis coordinate value, and the ending point is the endpoint on the right, which is the point with the largest x-axis coordinate value.

[0067] In one embodiment, the width of the current frame image refers to the size of the current frame image along a first direction, that is, the span in the x-axis direction.

[0068] If the preset cropping area cannot be completely located within the first image area within a preset number of attempts, the endpoint coordinates of the preset cropping area are translated based on the translation amount in the x-direction of the last adjusted homography matrix, thereby translating the preset cropping area into the first image area in the x-direction.

[0069] In some embodiments, correcting the coordinates of the preset cutting area based on the first translation amount and the second translation amount may include: In response to the second translation amount being positive, the coordinates of the second starting point of the preset cutting area are adjusted to the second translation amount; In response to the second translation amount being negative, the coordinates of the second end point of the preset cropping region are adjusted to the sum of the second translation amount and the height of the current frame image; wherein the second start point and the second end point are the two endpoints of the preset cropping region in the second direction, and the height is the dimension along the second direction.

[0070] In one embodiment, if the second translation amount in the homography matrix after the last adjustment is greater than 0, then the coordinates of the second starting point of the preset cropping region in the second direction are adjusted to the second translation amount. If the second translation amount in the homography matrix after the last adjustment is less than 0, then the coordinates of the second ending point of the preset cropping region in the second direction are adjusted to the sum of the second translation amount and the height of the current frame image.

[0071] Here, the second starting point is the starting point of the preset clipping area in the second direction, such as the y-axis, and the second ending point is the ending point of the preset clipping area in the second direction, such as the y-axis. For example, when the vector direction is clockwise, the starting point is the upper endpoint, which is the point with the smallest y-axis coordinate value, and the ending point is the lower endpoint, which is the point with the largest y-axis coordinate value.

[0072] In one embodiment, the height of the current frame image refers to the size of the current frame image along the second direction, that is, the span in the y-axis direction.

[0073] Thus, if the preset cropping area cannot be completely located within the first image area within a preset number of attempts, the endpoint coordinates of the preset cropping area are translated based on the translation amount in the y-direction of the last adjusted homography matrix, thereby translating the preset cropping area into the first image area in the y-direction.

[0074] For example, such as Figure 6A and 6B The images shown are schematic diagrams illustrating the image effects before and after applying an electronic image stabilization control method that includes adjusting the cropping region boundary based on translation amount in a scene with large-scale motion. It can be seen that the black borders are significantly removed. Since this effect cannot be expressed using line drawings or other methods, this comparison image is used to represent the above effect.

[0075] As one possible implementation, embodiments of this application provide an electronic image stabilization boundary control method, such as... Figure 7 As shown, the specific steps include: S1. Map the four corner points of the image according to the homography matrix.

[0076] The image is the current frame image, and after mapping, the mapped region (i.e., the first image region) is obtained.

[0077] S2. Determine whether the four corner points of the clipping region are within the mapping region. If so, output the homography matrix. The clipping region is the preset clipping region.

[0078] The cropping region in electronic image stabilization is generally a portion of the original image (i.e., the current frame image). For example, 10% is cropped from each of the four sides of the original image to form the cropping region. After obtaining the homography matrix, the original... Figure 4 Each corner point undergoes perspective transformation to form a mapped region. It is necessary to determine whether the four corner points of the clipping region are within the mapped region.

[0079] The cross product method is used to determine whether the four corner points of the clipping region are within the mapped region. The clockwise direction of the mapped region is taken as the vector direction. That is, for each edge of the mapped region, the cross product of the edge vector and the vector from a corner point to the edge starting point is calculated. If all cross products are positive, the point is inside the polygon; if the cross product of the point and the edge vector is negative, the point is outside the edge.

[0080] For corner points that are not within the mapped area, proceed to step S3.

[0081] S3. Calculate the distance from the corner point to the edge of the mapped region.

[0082] Determine the straight-line distance from the corner point to the nearest edge within the mapping region. If it is the left edge, the distance is recorded as the distance in the x-direction and is taken as positive; if it is the top edge, the distance is recorded as the distance in the y-direction and is taken as positive; if it is the right edge, the distance is recorded as the distance in the x-direction and is taken as negative; if it is the bottom edge, the distance is recorded as the distance in the y-direction and is taken as negative.

[0083] S4. Adjust the homography matrix according to the distance.

[0084] Iterate through the straight-line distances from the corner points to the edges calculated in S1. If the distance is in the x-direction (i.e., the first direction), subtract the x-direction distance from the x-direction translation in the homography matrix; if the distance is in the y-direction (i.e., the second direction), subtract the y-direction distance from the y-direction translation in the homography matrix. Repeat the steps in S1 based on the new homography matrix to obtain a new homography matrix from the corner points to the edges.

[0085] S5. Determine if the number of adjustments is less than 10.

[0086] Repeat the above steps 10 times until all corner points are within the mapped area, or until they cannot be adjusted to the mapped area. After 10 repetitions, proceed to step S6.

[0087] S6. Cut from the original image according to the adjusted translation amount.

[0088] After adjustment using the homography matrix, the mapped region includes the clipping region, allowing for direct clipping.

[0089] Furthermore, in scenes with significant motion, it's difficult to ensure all corner points are within the mapping area. In this case, the x and y-direction translations of the last adjusted homography matrix are obtained. If the x-direction translation is positive, the starting x-coordinate of the original cropping region is changed to the x-direction translation in the homography matrix; if the x-direction translation is negative, the ending x-coordinate of the original cropping region is changed to the x-direction translation in the homography matrix plus the image width. Similarly, if the y-direction translation is positive, the starting y-coordinate of the original cropping region is changed to the y-direction translation in the homography matrix; if the y-direction translation is negative, the ending y-coordinate of the original cropping region is changed to the y-direction translation in the homography matrix plus the image height. The new cropping region is then used to crop the image from the original image to obtain the stabilized region.

[0090] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the electronic image stabilization control method as described in any embodiment of this application.

[0091] This application also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the electronic image stabilization control method as described in any embodiment of this application.

[0092] For example, Figure 8 This is a structural diagram of an electronic device according to an embodiment of this application, such as... Figure 8 As shown, the electronic device includes a processor 41, a communication interface 42, a memory 43, and a communication bus 44. The processor 41, the communication interface 42, and the memory 43 communicate with each other through the communication bus 44. The memory 43 is used to store computer programs. When the processor 41 executes the program stored in the memory 43, it implements the steps of the method described in any one or more of the foregoing method embodiments.

[0093] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0094] The communication interface is used for communication between the aforementioned terminal and other devices.

[0095] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0096] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0097] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform the method described in any of the above embodiments.

[0098] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are generated entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0099] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0100] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An electronic image stabilization control method, characterized in that, include: The current frame image is transformed based on the homography matrix corresponding to the current frame image to obtain the first image region; Determine whether the preset cropping region in the current frame image is entirely located within the first image region; In response to the fact that the preset cropping area is not entirely located within the first image area, the homography matrix is ​​adjusted; Based on the adjusted homography matrix, the image is transformed again until the preset cropping area is completely located within the first image area. Then, cropping is performed based on the preset cropping area to obtain a stable image corresponding to the current frame image.

2. The electronic image stabilization control method according to claim 1, characterized in that, The method further includes: In response to the fact that the preset cropping area is entirely located within the first image area, cropping is performed based on the preset cropping area to obtain a stable image corresponding to the current frame image.

3. The electronic image stabilization control method according to claim 1, characterized in that, Determining whether the preset cropping region in the current frame image is entirely located within the first image region includes: Determine the first vector corresponding to each edge of the first image region; Determine a second vector corresponding to the starting point of an edge from each corner point of the preset cropping area to the first image area; wherein the starting point is an endpoint of the edge determined based on the vector direction of the first vector; Calculate the cross product of the second vector from the corner point to an edge and the first vector corresponding to the edge; If the cross product of the corner point and each edge is greater than 0, then the corner point is determined to be located within the first image region; In response to the fact that all corner points are located within the first image area, it is determined that the preset cropping area in the current frame image is entirely located within the first image area.

4. The electronic image stabilization control method according to claim 1, characterized in that, The adjustment of the homography matrix includes: The target corner point is determined to be one that does not exist within the first image area in the preset cropping area; Determine the distance between the target corner point and the nearest edge to the target corner point in the first image region; In response to the nearest edge being the left edge, the distance is subtracted from the first translation amount in the first direction of the homography matrix; In response to the nearest edge being the right edge, the distance is added to the first translation amount in the first direction of the homography matrix; the first direction is the direction from the left edge to the right edge. In response to the nearest edge being the top edge, the distance is subtracted from the second translation amount in the second direction of the homography matrix. In response to the nearest edge being the bottom edge, the second translation amount in the second direction of the homography matrix is ​​added to the distance; the second direction is the direction from the top edge to the bottom edge.

5. The electronic image stabilization control method according to claim 4, characterized in that, The process of re-transforming based on the adjusted homography matrix until the entire preset cropping region is located within the first image region includes: Based on the adjusted homography matrix, if the preset cropping area is still not entirely within the first image area, the homography matrix is ​​adjusted again until the preset number of adjustments is reached and the preset cropping area is still not entirely within the first image area. The first translation amount and the second translation amount in the last adjusted homography matrix are then obtained. The coordinates of the preset cropping area are corrected based on the first translation amount and the second translation amount, so that the preset cropping area is entirely located within the first image area.

6. The electronic image stabilization control method according to claim 5, characterized in that, The step of correcting the coordinates of the preset cutting area based on the first translation amount and the second translation amount includes: In response to the first translation amount being positive, the coordinates of the first starting point of the preset cutting area are adjusted to the first translation amount; In response to the first translation amount being negative, the coordinates of the first end point of the preset cropping region are adjusted to the sum of the first translation amount and the width of the current frame image; wherein the first start point and the first end point are the two endpoints of the preset cropping region in the first direction, and the width is the dimension along the first direction.

7. The electronic image stabilization control method according to claim 5, characterized in that, The step of correcting the coordinates of the preset cutting area based on the first translation amount and the second translation amount includes: In response to the second translation amount being positive, the coordinates of the second starting point of the preset cutting area are adjusted to the second translation amount; In response to the second translation amount being negative, the coordinates of the second end point of the preset cropping region are adjusted to the sum of the second translation amount and the height of the current frame image; wherein the second start point and the second end point are the two endpoints of the preset cropping region in the second direction, and the height is the dimension along the second direction.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the electronic image stabilization control method as described in any one of claims 1 to 7.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the electronic image stabilization control method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the electronic image stabilization control method as described in any one of claims 1 to 7.

Images