Method, device, electronic device, and storage medium for generating transition video

Abstract

The present invention provides a method, device, electronic device, and storage medium for generating a transition video. The method comprises: receiving an image pair, the image pair comprising a first image and a second image; extracting n key feature points from the first image and the second image respectively to form a first point set and a second point set; forming a first triangulation region set of the first image according to the first point set; forming a second triangulation region set of the second image according to the second point set; determining a matching relationship between the first triangulation region set and the second triangulation region set; calculating a first affine matrix and a second affine matrix respectively; generating an intermediate triangulation region set according to the first triangulation region set and the second triangulation region set based on the first affine matrix and the second affine matrix; generating an intermediate triangulation region set; generating an intermediate frame; and generating a transition video based on the first image, the intermediate frame, and the second image. The present invention realizes the generation of a transition video.

Description

Background Art

[0002] With the development of Internet information and the continuous upgrading of network technology, traditional pictures and texts can no longer meet the needs of users to quickly obtain information, and mobile video technology has developed rapidly. For OTA (online travel companies), accurately and beautifully displaying information about products for sale based on videos can effectively improve user experience. Therefore, as an important information medium, it is particularly important to produce high-quality videos. There are many forms of videos, which can be mainly divided into two categories: videos generated by pictures and videos actually shot. Since real-time videos require professional equipment and lens processing skills, the cost of real-time videos is high, and the historical accumulation is far less than the number of high-quality pictures. Therefore, based on existing pictures, automatically generating videos through machines and mining the transition mode of image-synthesized videos can greatly save the cost of video generation and improve the richness of image-synthesized videos, which has high application value.

[0003] Therefore, how to generate transition videos based on images is a technical problem that needs to be solved urgently in this field. Summary of the invention

[0004] In order to overcome the defects of the above-mentioned prior art, the present invention provides a method, device, electronic device and storage medium for generating transition video, so that transition video can be generated based on images.

[0005] According to one aspect of the present invention, there is provided a method for generating a transition video, comprising:

[0006] receiving an image pair, the image pair comprising a first image and a second image;

[0007] Extracting n key feature points from the first image and the second image respectively to form a first point set and a second point set, where n is an integer greater than 0;

[0008] forming a first set of triangulated regions of the first image based on the first set of points;

[0009] forming a second set of triangulated regions of the second image based on the second set of points;

[0010] Determine a matching relationship between triangulated regions in the first triangulated region set and the second triangulated region set according to the first point set and the second point set;

[0011] Respectively calculating a first affine matrix and a second affine matrix for converting the matched first triangulation region and the second triangulation region to the intermediate triangulation region;

[0012] Based on the first affine matrix and the second affine matrix, generate an intermediate triangulation region set according to the first triangulation region set and the second triangulation region set;

[0013] Generating an intermediate frame according to the generated set of intermediate triangulation regions;

[0014] A transition video is generated based on the first image, the intermediate frame and the second image.

[0015] In some embodiments of the present application, before extracting n key feature points from the first image and the second image respectively, the process includes:

[0016] Obtaining a similarity score for the image pair;

[0017] The number n of key feature points is calculated according to the similarity score of the image pair.

[0018] In some embodiments of the present application, calculating the number n of key feature points according to the similarity score of the image pair includes:

[0019] When the similarity score s of the image pair is less than 0.5, n is set to 9;

[0020] When the similarity score s of the image pair is greater than or equal to 0.5, n is set to 10*s+4.

[0021] In some embodiments of the present application, the first point set and the second point set respectively include four corner points of the image, center points of four edges and an image centroid point, and the image centroid point is calculated based on the zero-order matrix and the first-order matrix of the image.

[0022] In some embodiments of the present application, when the similarity score s of the image pair is greater than or equal to 0.5, extracting n key feature points from the first image and the second image respectively includes:

[0023] Extract the four corner points, the center points of the four edges and the center of gravity of the image;

[0024] Extract n-9 feature points based on the feature point matching algorithm.

[0025] In some embodiments of the present application, the calculating of the first affine matrix and the second affine matrix of the first triangulation region and the second triangulation region respectively to convert the matched first triangulation region to the intermediate triangulation region comprises:

[0026] Selecting a pair of matching first triangulation region and second triangulation region;

[0027] Calculating vertex coordinates of an intermediate triangulation region according to the selected vertex coordinates of the first triangulation region and the vertex coordinates of the second triangulation region;

[0028] The first affine matrix and the second affine matrix are calculated respectively according to the selected vertex coordinates of the first triangulation area, the vertex coordinates of the second triangulation area, and the vertex coordinates of the intermediate triangulation area.

[0029] In some embodiments of the present application, the vertex coordinates of the intermediate triangulation region, the first affine matrix, and the second affine matrix are calculated based on the fusion coefficients, and the method further includes:

[0030] Based on different fusion coefficients, multiple intermediate frames are generated to form an intermediate frame sequence;

[0031] A transition video is generated based on the first image, the intermediate frame sequence and the second image.

[0032] According to another aspect of the present application, a device for generating a transition video is also provided, comprising:

[0033] A receiving module, configured to receive an image pair, wherein the image pair includes a first image and a second image;

[0034] An extraction module, used to extract n key feature points from the first image and the second image respectively to form a first point set and a second point set, where n is an integer greater than 0;

[0035] A first triangulation module, configured to form a first triangulation region set of the first image according to the first point set;

[0036] A second triangulation module, configured to form a second triangulation region set of the second image according to the second point set;

[0037] A matching module, configured to determine a matching relationship between triangulated regions in the first triangulated region set and the second triangulated region set according to the first point set and the second point set;

[0038] An affine matrix calculation module, used for respectively calculating a first affine matrix and a second affine matrix for converting the matched first triangulation region and the second triangulation region to the intermediate triangulation region;

[0039] An intermediate triangulation generation module, used to generate an intermediate triangulation region set according to the first triangulation region set and the second triangulation region set based on the first affine matrix and the second affine matrix;

[0040] An intermediate frame generation module, used for generating an intermediate frame according to the generated intermediate triangulation region set;

[0041] A transition module is used to generate a transition video based on the first image, the intermediate frame and the second image.

[0042] According to another aspect of the present invention, an electronic device is provided, comprising: a processor; and a storage medium storing a computer program, wherein the computer program, when executed by the processor, performs the steps of the method for generating a transition video as described above.

[0043] According to yet another aspect of the present invention, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for generating a transition video as described above are executed.

[0044] Compared with the prior art, the advantages of the present invention are:

[0045] By extracting the key matching feature points of the image, the feature points and the contour points are combined into a point set to obtain the triangulation area of ​​the image. Based on the corresponding matching points, the triangulation area of ​​the two images is obtained, and the intermediate transition image is constructed and combined through affine transformation to generate a transition video. Therefore, the present application builds a model based on existing image data, which can efficiently extract the key points of the picture, obtain the matching area, and generate the transition video through shape gradient, which can greatly save the labor cost of video production and shooting, ensure the richness of the image-generated video, and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and other features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

[0047] Figure 1 A flow chart of a method for generating a transition video according to an embodiment of the present invention is shown.

[0048] Figure 2 A schematic diagram showing key feature points of an image according to an embodiment of the present invention is shown.

[0049] Figure 3 A flowchart of generating a transition video based on an intermediate frame sequence according to another embodiment of the present invention is shown.

[0050] Figure 4 A module diagram of an apparatus for generating a transition video according to an embodiment of the present invention is shown.

[0051] Figure 5 A schematic diagram of a computer-readable storage medium in an exemplary embodiment of the present disclosure is schematically shown.

[0052] Figure 6 A schematic diagram of an electronic device in an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0053] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0054] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0055] See first Figure 1 , Figure 1 A schematic diagram of a method for generating a transition video according to an embodiment of the present invention is shown. The method for generating a transition video comprises the following steps:

[0056] Step S110: receiving an image pair, wherein the image pair includes a first image and a second image.

[0057] Step S120: extracting n key feature points from the first image and the second image respectively to form a first point set and a second point set, where n is an integer greater than 0.

[0058] Specifically, before step S120, the similarity score of the image pair can be obtained; and the number n of key feature points can be calculated based on the similarity score of the image pair. The similarity score of the image pair can be obtained based on various image similarities, and the present application is not limited thereto. The similarity of the image pair can be normalized to a value between 0 and 1. The number of key feature points to be extracted can be calculated and determined based on the obtained similarity score of the image pair.

[0059] Specifically, calculating the number n of key feature points according to the similarity score of the image pair includes: when the similarity score s of the image pair is less than 0.5, setting n to 9; when the similarity score s of the image pair is greater than or equal to 0.5, setting n to 10*s+4.

[0060] In this embodiment, if the similarity between two images is less than 0.5, it means that the images are not similar, and the conventional feature point matching algorithm is usually unable to effectively extract effective features. Therefore, this embodiment can construct a set of matching points with fixed attributes based on this: the four corner points of the image, the center points of the four edges, and the center of gravity of the image. Figure 2 As shown, Figure 2 A schematic diagram showing nine key feature points of an image 200 according to an embodiment of the present invention is shown.

[0061] Specifically, the image centroid point can be calculated based on the image zero-order matrix and the first-order matrix. m The coordinates (x c ,y c ) can be calculated as follows:

[0062]

[0063] Among them, M 00 is the zero-order matrix of the image, M 10 and M 01 is the first-order matrix of the image.

[0064] Zero-order matrix M 00 It can be calculated according to the following formula:

[0065]

[0066] Among them, V(i,j) is the grayscale value of image (i,j), I is the number of columns of image pixels, and J is the number of rows of image pixels.

[0067] The first-order matrix M 10 and M 01 It can be calculated according to the following formula:

[0068]

[0069]

[0070] Furthermore, when the similarity score s of the image pair is greater than or equal to 0.5, the extracting of n key feature points from the first image and the second image respectively includes: extracting four corner points of the image, the center points of the four edges and the center of gravity of the image; extracting n-9 feature points based on a feature point matching algorithm. The feature point matching algorithm may include but is not limited to SIFT (Scale-invariant feature transform), SURF (Speeded Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), etc. According to these algorithms, the similarity of feature points can be calculated, and the n-9 feature point pairs with the highest similarity can be extracted as key feature point packages.

[0071] Step S130: forming a first triangulation region set of the first image according to the first point set.

[0072] Step S140: forming a second triangulation region set of the second image according to the second point set.

[0073] Specifically, the above steps S130 and S140 may adopt Delaunay triangulation to obtain a triangulation region set formed by a plurality of triangulation regions of the first image and the second image.

[0074] The definition of triangulation is: suppose V is a finite point set in the two-dimensional real number field, edge e is a closed line segment formed by the points in the point set as endpoints, and E is the set of e. Then a triangulation T = (V, E) of the point set V is a plane graph G, which satisfies the following conditions: except for the endpoints, the edges in the plane graph do not contain any points in the point set; there are no intersecting edges; all faces in the plane graph are triangular faces, and the union of all triangular faces is the convex hull of the scattered point set V. For Delaunay triangulation, the Delaunay edge can be defined as: suppose an edge e in E (with two endpoints a and b), e is called a Delaunay edge if it satisfies the following conditions: there is a circle passing through points a and b, and the circle (note that it is the circle, and at most three points on the circle are in the same circle) does not contain any other points in the point set V. This property is also called the empty circle property. The Delaunay triangulation can be defined as: if a triangulation T of the point set V contains only Delaunay edges, then the triangulation is called a Delaunay triangulation. Therefore, assuming that T is any triangulation of V, then T is a Delaunay triangulation of V, only if the interior of the circumscribed circle of each triangle in T does not contain any point in V. Delaunay triangulation can be implemented by using a flanging algorithm, a point-by-point insertion algorithm, a split-and-merge algorithm, a Bowyer-Watson algorithm, etc., but the present application is not limited thereto.

[0075] Step S150: determining a matching relationship between triangulated regions in the first triangulated region set and the second triangulated region set according to the first point set and the second point set.

[0076] Specifically, since the triangulated regions are generated according to the aforementioned point sets, and there is a matching relationship between the key feature points in the aforementioned first point set and the second point set, the triangulated regions in the first triangulated region set and the second triangulated region set also have a matching relationship.

[0077] Thus, multiple pairs of matching triangulated regions can be obtained according to the matching relationship.

[0078] Step S160: respectively calculating a first affine matrix and a second affine matrix for converting the matched first triangulation region and the second triangulation region to the intermediate triangulation region.

[0079] Specifically, step S160 can be implemented through the following steps: select a pair of matching first triangulation regions and the second triangulation regions; calculate the vertex coordinates of the intermediate triangulation region based on the vertex coordinates of the selected first triangulation region and the vertex coordinates of the second triangulation region; calculate the first affine matrix and the second affine matrix respectively based on the vertex coordinates of the selected first triangulation region, the vertex coordinates of the second triangulation region and the vertex coordinates of the intermediate triangulation region.

[0080] In some specific implementations, a blending function of two images can be set to control the blending coefficient of the two images. The coordinates of the triangulated area corresponding to the triangulated area of ​​the intermediate frame are obtained by interpolating the blending coefficient. This is shown in the following formula:

[0081] V M =(1-α)V1+αV2

[0082] Where V1 is the vertex coordinates of the first triangulated area selected, V2 is the vertex coordinates of the second triangulated area selected, and V M are the vertex coordinates of the intermediate triangulation region.

[0083] The first affine matrix M1 and the second affine matrix M2 are shown in the following formula:

[0084] V M =M1*V1

[0085] V M =M2*V2

[0086] The above formula can be further broken down into:

[0087]

[0088] Among them, x m and m are the vertex coordinates of the middle triangulation area, x11 and y12 are the vertex coordinates of the first triangulation area, M1. M2 can also be obtained by splitting in the same way.

[0089] Therefore, M1 and M2 can be obtained by solving the known vertex coordinates of the first triangulation region, the vertex coordinates of the second triangulation region, and the vertex coordinates of the intermediate triangulation region.

[0090] Step S170: Based on the first affine matrix and the second affine matrix, generate an intermediate triangulation region set according to the first triangulation region set and the second triangulation region set.

[0091] Step S170 can be expressed as follows:

[0092] Z=(1-α)M1A+αM2B

[0093] Among them, Z is the middle triangulation area, A and B are a pair of matching first triangulation areas and second triangulation areas, M1 and M2 are the first affine matrix and the second affine matrix, and α is the relaxation coefficient.

[0094] Step S180: generating an intermediate frame according to the generated set of intermediate triangulation regions.

[0095] Step S190: Generate a transition video based on the first image, the intermediate frame and the second image.

[0096] In the method for generating a transition video provided by the present invention, by extracting the key matchable feature points of the image, the feature points and the image contour points are combined into a point set, and the triangulation area of ​​the image is obtained. Based on the corresponding matching points, the triangulation areas of the two images are obtained, and the intermediate transition images are constructed and combined by affine transformation to generate a transition video. Thus, the present application builds a model based on existing image data, can efficiently extract key points of the picture, obtain matching areas, generate transition videos by shape gradient, can greatly save the labor cost of video production and shooting, ensure the richness of image-generated videos, and improve user experience.

[0097] See below Figure 3 , Figure 3 A flowchart of generating a transition video based on an intermediate frame sequence according to another embodiment of the present invention is shown. Figure 3 The following steps are shown:

[0098] Step S181: generating a plurality of intermediate frames based on different fusion coefficients to form an intermediate frame sequence;

[0099] Step S191: Generate a transition video based on the first image, the intermediate frame sequence and the second image.

[0100] Therefore, a plurality of intermediate frames can be smoothly transitioned based on the gradual adjustment of the fusion coefficient to form a transition video with smoother transition.

[0101] The method for generating transition videos based on key feature points proposed in the present invention realizes the gradual change of matching areas in the process of generating transition videos based on two images, improves the utilization rate of images, greatly reduces the cost of video acquisition and production, and improves the richness of image-synthesized videos.

[0102] The above is merely an illustrative description of multiple implementations of the present invention, and the present invention is not limited thereto.

[0103] The present invention also provides a device for generating a transition video. Figure 4A schematic diagram of a device for generating a transition video according to an embodiment of the present invention is shown. The device 400 for generating a transition video includes a receiving module 410, an extraction module 420, a first segmentation module 430, a second segmentation module 440, a matching module 450, an affine matrix calculation module 460, an intermediate segmentation generation module 470, an intermediate frame generation module 480, and a transition module 490.

[0104] The receiving module 410 is used to receive an image pair, where the image pair includes a first image and a second image;

[0105] The extraction module 420 is used to extract n key feature points from the first image and the second image respectively to form a first point set and a second point set, where n is an integer greater than 0;

[0106] The first triangulation module 430 is used to form a first triangulation region set of the first image according to the first point set;

[0107] The transition module 490 is used to form a second triangulation region set of the second image according to the second point set;

[0108] The matching module 450 is used to determine a matching relationship between the triangulated regions in the first triangulated region set and the second triangulated region set according to the first point set and the second point set;

[0109] The affine matrix calculation module 460 is used to respectively calculate the first affine matrix and the second affine matrix of the matched first triangulation region and the second triangulation region converted to the intermediate triangulation region;

[0110] The intermediate triangulation generation module 470 is used to generate an intermediate triangulation region set according to the first triangulation region set and the second triangulation region set based on the first affine matrix and the second affine matrix;

[0111] The intermediate frame generation module 480 is used to generate an intermediate frame according to the generated intermediate triangulation region set;

[0112] The transition module 490 is used to generate a transition video based on the first image, the intermediate frame and the second image.

[0113] In the device for generating transition video provided by the present invention, by extracting the key matchable feature points of the image, the feature points and the image contour points are combined into a point set to obtain the triangulation area of ​​the image. Then, based on the corresponding matching points, the triangulation areas of the two images are obtained, and the intermediate transition images are constructed and combined by affine transformation to generate the transition video. Thus, the present application builds a model based on the existing image data, can efficiently extract the key points of the picture, obtain the matching area, generate the transition video by shape gradient, can greatly save the labor cost of video production and shooting, ensure the richness of the image-generated video, and improve the user experience.

[0114] Figure 4 The present invention is merely a schematic illustration of the apparatus for generating transition videos provided by the present invention. Without violating the concept of the present invention, the splitting, merging, and adding of modules are all within the protection scope of the present invention. The apparatus for generating transition videos provided by the present invention can be implemented by software, hardware, firmware, plug-ins, and any combination thereof, and the present invention is not limited thereto.

[0115] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, and when the program is executed by, for example, a processor, the steps of the method for generating a transition video described in any of the above embodiments can be implemented. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to cause the terminal device to execute the steps of various exemplary embodiments of the present invention described in the method for generating a transition video described in the above-mentioned section of this specification.

[0116] refer to Figure 5 As shown, a program product 400 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0117] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0118] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0119] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the tenant computing device, partially on the tenant device, as a separate software package, partially on the tenant computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device may be connected to the tenant computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0120] In an exemplary embodiment of the present disclosure, an electronic device is further provided, which may include a processor and a memory for storing executable instructions of the processor, wherein the processor is configured to execute the steps of the method for generating a transition video in any of the above embodiments by executing the executable instructions.

[0121] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0122] Refer to the following Figure 6 The electronic device 600 according to this embodiment of the present invention is described. Figure 6 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0123] like Figure 6 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0124] The storage unit stores a program code, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the method for generating a transition video in the above description of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 Follow the steps shown in .

[0125] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0126] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0127] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0128] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable tenants to interact with the electronic device 600, and / or may communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 650. In addition, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 through a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0129] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation methods of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above-mentioned method for generating a transition video according to the implementation methods of the present disclosure.

[0130] Compared with the prior art, the advantages of the present invention are:

[0131] The present invention extracts the key matching feature points of the image, forms a point set with the feature points and the contour points of the image, and obtains the triangulation area of ​​the image. Based on the corresponding matching points, the triangulation areas of the two images are obtained, and the intermediate transition images are constructed and combined through affine changes to generate a transition video. Thus, the present application builds a model based on existing image data, can efficiently extract the key points of the picture, obtain the matching area, generate the transition video through shape gradient, can greatly save the labor cost of video production and shooting, ensure the richness of the image-generated video, and improve the user experience.

[0132] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A method for generating a transition video, characterized in that: include: receiving an image pair, the image pair comprising a first image and a second image; Obtaining a similarity score of the image pair, calculating the number n of key feature points according to the similarity score of the image pair, when the similarity score s of the image pair is less than 0.5, setting n to 9; when the similarity score s of the image pair is greater than or equal to 0.5, setting n to 10*s+4; extracting n key feature points from the first image and the second image respectively to form a first point set and a second point set, n is an integer greater than 0, the first point set and the second point set respectively include four corner points of the image, the center points of four edges and the image center of gravity, the image center of gravity is calculated according to the zero-order matrix and the first-order matrix of the image, when the similarity score s of the image pair is greater than or equal to 0.5, extracting n key feature points from the first image and the second image respectively includes: extracting four corner points of the image, the center points of four edges and the image center of gravity, and extracting n-9 feature points based on a feature point matching algorithm; forming a first set of triangulated regions of the first image based on the first set of points; forming a second set of triangulated regions of the second image based on the second set of points; Determine a matching relationship between triangulated regions in the first triangulated region set and the second triangulated region set according to the first point set and the second point set; Respectively calculating a first affine matrix and a second affine matrix for converting the matched first triangulation region and the second triangulation region to an intermediate triangulation region; Based on the first affine matrix and the second affine matrix, generate an intermediate triangulation region set according to the first triangulation region set and the second triangulation region set; Generate an intermediate frame according to the generated set of intermediate triangulation regions; A transition video is generated based on the first image, the intermediate frame and the second image.

2. The method for generating a transition video according to claim 1, wherein: The first affine matrix and the second affine matrix of converting the matched first triangulation region and the second triangulation region to the intermediate triangulation region respectively include: Selecting a pair of matching first triangulation region and second triangulation region; Calculating vertex coordinates of an intermediate triangulation region according to the selected vertex coordinates of the first triangulation region and the vertex coordinates of the second triangulation region; The first affine matrix and the second affine matrix are calculated respectively according to the selected vertex coordinates of the first triangulation area, the vertex coordinates of the second triangulation area, and the vertex coordinates of the intermediate triangulation area.

3. The method for generating a transition video according to claim 2, wherein: The vertex coordinates of the intermediate triangulation region, the first affine matrix, and the second affine matrix are calculated based on the fusion coefficients, and the method further includes: Based on different fusion coefficients, multiple intermediate frames are generated to form an intermediate frame sequence; A transition video is generated based on the first image, the intermediate frame sequence and the second image.

4. A device for generating a transition video, characterized in that: include: A receiving module, configured to receive an image pair, wherein the image pair includes a first image and a second image; an extraction module, obtaining a similarity score of the image pair, calculating the number n of key feature points according to the similarity score of the image pair, and when the similarity score s of the image pair is less than 0.5, setting n to 9; and when the similarity score s of the image pair is greater than or equal to 0.5, setting n to 10*s+4; extracting n key feature points from the first image and the second image respectively to form a first point set and a second point set, where n is an integer greater than 0, and the first point set and the second point set respectively include four corner points of the image, the center points of four edges, and the image center of gravity, and the image center of gravity is calculated according to the zero-order matrix and the first-order matrix of the image. When the similarity score s of the image pair is greater than or equal to 0.5, extracting n key feature points from the first image and the second image respectively includes: extracting four corner points of the image, the center points of four edges, and the image center of gravity, and extracting n-9 feature points based on a feature point matching algorithm; a first subdivision module, used to form a first triangulation region set of the first image according to the first point set; A second triangulation module, configured to form a second triangulation region set of the second image according to the second point set; A matching module, configured to determine a matching relationship between triangulated regions in the first triangulated region set and the second triangulated region set according to the first point set and the second point set; An affine matrix calculation module, used for respectively calculating a first affine matrix and a second affine matrix of the matched first triangulation region and the second triangulation region converted to an intermediate triangulation region; An intermediate triangulation generation module, used to generate an intermediate triangulation region set according to the first triangulation region set and the second triangulation region set based on the first affine matrix and the second affine matrix; An intermediate frame generation module, used for generating an intermediate frame according to the generated intermediate triangulation region set; A transition module is used to generate a transition video based on the first image, the intermediate frame and the second image.

5. An electronic device, characterized in that: The electronic device comprises: processor; A storage medium having a computer program stored thereon, wherein the computer program, when executed by the processor, executes the method for generating a transition video as described in any one of claims 1 to 3.

6. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method for generating a transition video according to any one of claims 1 to 3 is executed.

Images