Video synthesis method, device, equipment, medium and product

Abstract

This disclosure provides a video synthesis method, apparatus, device, medium, and product. The method includes: performing image segmentation on image frames in a video to be processed to obtain a region image corresponding to a target object and a background image excluding the region image; determining a mirror image corresponding to the region image; expanding the region image in a first expansion direction to obtain at least one first expanded image; the at least one first expanded image includes the region image; expanding the mirror image in a second expansion direction to obtain at least one second expanded image; rendering the background image, at least one second expanded image, and at least one first expanded image to obtain a target rendered image at the end of rendering; and synthesizing the target rendered image corresponding to the image frame to obtain a target video corresponding to the video to be processed. By detecting and expanding video images, the method solves the problem of relatively simple video processing methods.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and in particular to a video synthesis method, device, equipment, medium and product. Background Art

[0002] With the rapid development of the video field, the demand for increasing the fun of videos in video playback fields such as short videos and live videos is increasing. Especially in the real-time video acquisition scenario, the current video processing method is mostly to add interesting content such as bullet screen and stickers to the video.

[0003] However, the existing means of improving the fun of videos are relatively simple in their processing methods and the content is not rich enough, resulting in low video dissemination speed and low utilization rate. Summary of the invention

[0004] The embodiments of the present disclosure provide a video synthesis method, device, equipment, medium and product for split video synthesis to overcome the technical problem of the relatively single video processing method in the prior art.

[0005] In a first aspect, an embodiment of the present disclosure provides a video synthesis method, including:

[0006] Performing image segmentation on the image frames in the video to be processed to obtain a region image corresponding to the target object and a background image other than the region image;

[0007] Determine a mirror image corresponding to the regional image;

[0008] Performing expansion processing on the regional image in a first expansion direction to obtain at least one first expanded image; at least one first expanded image includes the regional image;

[0009] Performing an expansion process on the mirror image in a second expansion direction to obtain at least one second expanded image; the first expansion direction and the second expansion direction are different;

[0010] Rendering the background image, at least one of the second extended images, and at least one of the first extended images to obtain a target rendered image when rendering is completed;

[0011] Perform video synthesis on the target rendered image corresponding to the image frame to obtain a target video corresponding to the video to be processed.

[0012] In a second aspect, an embodiment of the present disclosure provides a device for video synthesis, including:

[0013] An image segmentation unit is used to perform image segmentation on the image frame in the video to be processed to obtain a region image corresponding to the target object and a background image other than the region image;

[0014] A mirror processing unit, used to determine a mirror image corresponding to the regional image;

[0015] A first expansion unit, configured to perform expansion processing on the regional image in a first expansion direction to obtain at least one first expanded image; at least one first expanded image includes the regional image;

[0016] A second expansion unit, configured to expand the mirror image in a second expansion direction to obtain at least one second expanded image; the first expansion direction and the second expansion direction are different;

[0017] An image rendering unit, configured to render the background image, at least one of the second extended images, and at least one of the first extended images to obtain a target rendered image when rendering is completed;

[0018] The video synthesis unit is used to perform video synthesis on the target rendered image corresponding to the image frame to obtain a target video corresponding to the video to be processed.

[0019] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0020] The memory stores computer-executable instructions;

[0021] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the video synthesis method as described in the first aspect and various possible designs of the first aspect.

[0022] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the video synthesis method described in the first aspect and various possible designs of the first aspect is implemented.

[0023] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the video synthesis method described in the first aspect and various possible designs of the first aspect.

[0024] In this embodiment, the image frame in the video to be processed is segmented to obtain the regional image and the background image where the target object is located, and the regional image and the mirror image of the regional image are used to expand the image. Specifically, the regional image can be expanded in a first expansion direction to obtain at least one first expanded image, and the mirror image can be expanded in a second expansion direction to obtain at least one second expanded image. Through the expansion of the image, the target object can be cloned. Then, by rendering at least one first expanded image, at least one second expanded image and a background image, a target rendered image with a target object clone is obtained. The target rendered image is obtained by rendering a background image, at least one second expanded image and at least one first expanded image, and the expanded images in the first expansion direction and the second expansion direction can be displayed to improve the display efficiency and fun of the image. The target rendered image corresponding to the image frame is synthesized to obtain the target video of the video to be processed. By automatically completing the image rendering and the automatic synthesis of the target video, the video acquisition efficiency and accuracy can be improved, ensuring that the playback speed and efficiency of the target video are effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0026] Figure 1 A schematic diagram of an application of a fractional video synthesis method provided by an embodiment of the present disclosure;

[0027] Figure 2 A flowchart of an embodiment of a video synthesis method provided by an embodiment of the present disclosure;

[0028] Figure 3 An example diagram of a mirror image provided by an embodiment of the present disclosure;

[0029] Figure 4 A flowchart of another embodiment of a video synthesis method provided by the embodiment of the present disclosure;

[0030] Figure 5 An example diagram of a displacement point provided for an embodiment of the present disclosure;

[0031] Figure 6 A flowchart of another embodiment of a video synthesis method provided by the embodiment of the present disclosure;

[0032] Figure 7 An example diagram of a target rendered image provided by an embodiment of the present disclosure;

[0033] Figure 8 A flowchart of another embodiment of a video synthesis method provided by the embodiment of the present disclosure;

[0034] Fig. 9 A structural example diagram of a video synthesis device provided by an embodiment of the present disclosure;

[0035] Fig.10 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0037] The technical solution disclosed in the present invention can be applied to the field of video. It can obtain a rendered image by performing split-view extraction, expansion and rendering on the video, and synthesize the video using the rendered images of each video frame. In this process, everything from image frame extraction to image segmentation, image expansion and rendering synthesis can be completed automatically, and a split-view video can be obtained at the end of rendering, thereby improving the display efficiency of the split-view video.

[0038] In the related technologies, various live videos and short videos have high requirements for real-time video processing. At present, the more common way to process videos in real time is to add bullet screens, stickers and other content to the video to improve the fun of video playback. However, this processing method is relatively simple and does not improve the speed of video dissemination and recommendation.

[0039] In order to solve the above technical problems, the present disclosure considers adding a clone of an object in motion in a video to increase the interest of the video. Therefore, it is considered to automatically clone each frame of the image, and then synthesize the cloned video frames into a target video. In order to obtain a cloned image, the regional image and the mirror image of the target object can be expanded in different directions, and then the expanded images are uniformly rendered to obtain the corresponding target rendered image. In this way, the object clone in the image can be realized to utilize video synthesis, improve the synthesis efficiency and interest of the video, and have a positive effect on improving the playback rate of the video.

[0040] In an embodiment of the present disclosure, the image frames in the video to be processed are segmented to obtain the regional image and the background image where the target object is located, the regional image is expanded in a first expansion direction and the mirror image is expanded in a second expansion direction to obtain at least one first expanded image and at least one second expanded image, and a target rendered image having a clone of the target object is obtained by rendering at least one first expanded image and at least one second expanded image and the background image. The target rendered image is obtained by rendering the background image, at least one second expanded image and at least one first expanded image, and can display a forward clone and a reverse clone, thereby improving the image display efficiency. The target rendered image corresponding to the image frame is subjected to video synthesis to obtain the target video of the video to be processed. By automatically completing the image rendering and the automatic synthesis of the target video, the video acquisition efficiency and accuracy can be improved, ensuring that the playback volume and playback speed of the target video are effectively improved.

[0041] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0042] Figure 1 This is an application diagram of the video synthesis method according to the present disclosure. The application diagram according to the embodiment of the present disclosure may include an electronic device and a user terminal connected to the electronic device through a local area network or a wide area network. It is assumed that the electronic device can be a personal computer, an ordinary server, a super personal computer, a cloud server or other type of server. The present disclosure does not make too many restrictions on the specific type of the electronic device. The client can be, for example, a mobile phone, a tablet computer, a personal computer, a smart home appliance, a wearable device or other terminal device. The present disclosure does not make too many restrictions on the specific type of the client. Figure 1 As shown, taking the electronic device as the cloud server 1, the user terminal 2 can be configured in a terminal device such as a mobile phone 21, a tablet computer 22, etc., as an example, any user terminal 2 can provide a video to be processed to the cloud server 1. The cloud server 1 can obtain the video to be processed sent by any user terminal 2, and synthesize the target video based on the video synthesis method provided in the embodiment of the present disclosure.

[0043] Afterwards, the cloud server 1 can send the synthesized target video to the user terminal 2. In addition, in actual applications, after the target video is confirmed to be released by the user terminal 2, the target video can be released. If the target video is successfully released, other user terminals 3, such as Figure 1 The mobile phone 3 shown can obtain the target video and play the target video. Of course, in actual application, the target video can also be recommended to other user terminals 3 to improve the play rate of the target video.

[0044] refer to Figure 2 , Figure 2 The present disclosure provides a flowchart of an embodiment of a video synthesis method, which can be configured as a video synthesis device, and the video synthesis device can be located in an electronic device. The video synthesis method may include the following steps:

[0045] 201: Perform image segmentation on the image frames in the video to be processed to obtain a region image corresponding to the target object and a background image other than the region image.

[0046] Optionally, the image frame can be obtained by sampling from the video to be processed. In practical applications, the video to be processed may correspond to a frame rate ((Frame rate)) = number of frames (Frames) / time (Time), and the unit of the frame rate is frames per second (f / s, frames per second, fps). When sampling from the video to be processed, the sampling frequency can be set to include the frame rate of the video to be processed. If the sampling frequency is set to be the same as the frame rate of the image frame, it can be ensured that each image frame in the video can be sampled, and the technical solution of the present disclosure is used for image processing to obtain the corresponding target rendered image.

[0047] Of course, in practical applications, the sampling frequency of the video to be processed can also be set according to actual needs, and is not limited to being equal to the frame rate. The higher the sampling frequency, the more image frames are collected per unit time, and the lower the possibility of discontinuous images. The lower the sampling frequency, the fewer image frames are collected per unit time, and the higher the possibility of discontinuous images.

[0048] The image frame can be obtained by sampling from the video to be processed. The image frame may include at least one, and each image frame can obtain a corresponding target rendered image through steps 201-204.

[0049] The area image may be an image of the area where the contour of the target object is located, or may be an irregular image. The background image may be an image corresponding to other areas in the video to be processed except the contour of the target object.

[0050] Optionally, the target object may include a movable object such as a human body, an object, an animal, etc. A common target object may include, for example, a person or an animal in motion.

[0051] 202: Determine a mirror image corresponding to the regional image.

[0052] The mirror image can be obtained by mirror-flipping the regional image. The image content of the mirror image is the same as that of the regional image, but the display direction is opposite.

[0053] Mirror images can include any of horizontal mirror images, vertical mirror images, and diagonal mirror images. Among them, a water surface mirror image refers to a mirror image that is centered on the vertical axis of the image without distinguishing between the left and right sides of the image. A vertical mirror image can be a mirror image that is centered on the horizontal axis of the image without distinguishing between the top and bottom sides of the image. A diagonal mirror image can be a mirror image that is centered on the intersection of the horizontal axis and the vertical axis of the image, which is equivalent to performing horizontal mirroring and vertical mirroring on the image in sequence. For ease of understanding, refer to Figure 3 In the triangle 301 shown, vertex A is on the left side of side length B. After horizontal mirroring 301, vertex A' is on the right side of side length B'.

[0054] 203: Perform expansion processing on the regional image in a first expansion direction to obtain at least one first expanded image; the at least one first expanded image includes the regional image.

[0055] 204: Perform expansion processing on the mirror image in a second expansion direction to obtain at least one second expanded image.

[0056] The first expansion direction and the second expansion direction are different.

[0057] In some embodiments, the first extension direction and the second extension direction may be opposite. That is, the angle between the first extension direction and the second extension direction is 180 degrees. Of course, the first extension direction and the second extension direction may also be determined based on two angles formed by the perpendicular line of the horizontal line, the perpendicular line of the horizontal line may be used as the common side of the two angles, and the directions pointed to by the other sides of the two angles may be the first extension reverse direction and the second extension direction, respectively. Assume that the two angles are represented by angle A and angle B. In practical applications, the angles of angle A and angle B may be equal or unequal, and the angles of angle A and angle B are between 0 and 180 degrees. Among them, when the angle is 90 degrees, the first extension direction and the second extension direction are opposite. When angle A and angle B are the same, the first extension direction and the second extension reverse direction are symmetrical relative to the perpendicular line of the horizontal line.

[0058] Optionally, taking the vertical line of the horizontal line as an example, the first expansion direction may include the right side of the vertical line of the horizontal line, and the second expansion direction may include the right side of the vertical line of the horizontal line. In practical applications, the horizontal line may include the horizontal line corresponding to the center point of the target object. For ease of understanding, taking the image displayed on the display screen and a person viewing the image on the display screen as an example, the left side of the image on the display screen that can be viewed by the human eye may be reverse, and the right side of the image on the display screen that can be viewed by the human eye may be forward. Forward and reverse may specifically refer to the left expansion or right expansion of the image. The regional image may be expanded to the right in the first expansion direction to obtain at least one first expanded image, and expanded to the left in the second expansion direction to obtain at least one second expanded image.

[0059] 205: Render the background image, the at least one second extended image, and the at least one first extended image to obtain a target rendered image when rendering is completed.

[0060] Optionally, the target rendered image may be obtained by rendering a background image, at least one second extended image and at least one first extended image respectively, and may specifically include a rendered image obtained when multi-layer rendering of the background image, at least one second extended image and at least one first extended image is completed.

[0061] 206: Perform video synthesis on the target rendered image corresponding to the image frame to obtain a target video corresponding to the video to be processed.

[0062] The target video can be obtained by synthesizing the target rendered images corresponding to multiple image frames. When synthesizing the target rendered images corresponding to the image frames, the target rendered images corresponding to the image frames can be synthesized according to the timestamps of the image frames. The timestamps of the image frames are the timestamps of the corresponding target rendered images. During the video synthesis process, the frame rate of the target video can be determined according to the frame rate of the video to be processed, and the synthesis time of the video can be accurately controlled by the frame rate of the video.

[0063] In the disclosed embodiment, the image frames in the video to be processed are segmented to obtain the regional image and the background image where the target object is located, and the regional image and the background image of the regional image are used to expand the image to obtain at least one first expanded image and at least one second expanded image. By rendering at least one first expanded image and at least one second expanded image and the background image, a target rendered image having a clone of the target object is obtained. The target rendered image is obtained by rendering the background image, at least one second expanded image and at least one first expanded image, and can display a positive clone and a reverse clone, thereby improving the image display efficiency. Afterwards, the target rendered image corresponding to the image frame can be synthesized to obtain the target video of the video to be processed. By automatically completing the image rendering and the automatic synthesis of the target video, the video acquisition efficiency and accuracy can be improved, ensuring that the playback volume and playback speed of the target video are effectively improved.

[0064] In addition, the video to be processed involved in the present disclosure can be acquired in real time, by extracting image frames from the video acquired in real time. Then, the image frame is segmented into regions of the target object, so as to use the region image obtained by segmentation to perform mirror flipping, realize the expansion of the positive region image and the expansion of the mirror image, and obtain at least one first extended image and at least one second extended image. By rendering the background image other than the region image, at least with the first extended image and at least one second extended image, a target rendered image is obtained, so as to use the target rendered image to synthesize the video. This process is all completed automatically, and does not require manual intervention, such as manual copying of common drawing software, etc. The video synthesis efficiency is relatively high, which can solve the problem of low efficiency in manually completing the synthesis of cloned videos in the prior art.

[0065] In order to obtain an accurate first extended image, such as Figure 4 FIG. 1 is a flowchart of another embodiment of a video synthesis method provided by an embodiment of the present disclosure, which is different from other embodiments in that, performing expansion processing on a regional image in a first expansion direction to obtain at least one first expanded image may include:

[0066] 401: Determine the displacement speed of the target object corresponding to the image frame.

[0067] The displacement speed may refer to the moving speed of the target object in the video to be processed. For example, assuming that the target object is at position N when the video is played, and moves to position M when the second frame is played, the displacement speed of the target object can be calculated based on the difference between position N and position M combined with the time difference between the two image frames.

[0068] 402: Based on the original position point of the regional image, displacement processing is performed according to the displacement speed and the first expansion direction to obtain at least one positive displacement point including the original position point.

[0069] The regional image may refer to marking the pixels belonging to the target object in the image frame to obtain the pixels of the regional image in the image frame. The coordinates of the pixels corresponding to the regional image may also be represented by the pixel coordinate system of the image frame. The pixels not marked as the regional image are the pixels of the background image.

[0070] Among them, the original position point of the regional image may refer to the pixel coordinate point corresponding to the key point of the target object in the image frame. The key point may refer to a pixel point that can confirm the position of the regional image, and the coordinates of the pixel point are represented by the pixel coordinate system of the entire image frame. The original position point may, for example, include the coordinate point of the center point of the regional image in the image frame. For example, when the target object is a human body, it can also be the coordinates of a key position point in the human body, such as the coordinates of the pixel points such as the center point of the eyebrows, the tip of the nose, and the center point. The meaning of the position corresponding to the original position point is not limited here. Similarly, the positive displacement point may include the pixel coordinate point located in the image frame after the original position point is displaced. Both the original position point and the positive displacement point may include coordinate points located in the pixel coordinate system of the image frame.

[0071] Optionally, the first expansion direction may include a positive displacement angle. The first expansion direction may be determined by the trajectory of the target object in the video frame. For example, taking the movement from position N to position M as an example, the line connecting the two positions may have an angle relative to the horizontal line or the vertical line, and the angle is the positive displacement angle. Assuming the positive displacement angle is 45 degrees, when the target object is displaced, the distance moved to the upper right and at a straight line with an angle of 45 degrees to the horizontal line is calculated according to the displacement speed to obtain a point, which is a positive displacement point. For ease of understanding, as Figure 5 As shown, assuming that P0 is the original position point, P0 is displaced 45 degrees in the positive direction, and the obtained positive displacement point may be P1.

[0072] 403: Copy the regional image to the positive displacement point to obtain a first extended image corresponding to the regional image at at least one positive displacement point.

[0073] Copying the regional image to the positive displacement point may include moving the pixel value corresponding to the pixel point of the regional image to the corresponding coordinate point according to the positive displacement point, that is, the positive coordinate point of the first extended image in the image frame may be determined according to the coordinate point of the regional image in the image frame and the positive displacement point, and the first extended image composed of the pixel value corresponding to the positive coordinate point may be obtained by assigning the pixel value at the coordinate point in the regional image to the corresponding positive coordinate point. Of course, in practical applications, the regional image may include multiple pixel points, and each pixel point may be moved one by one according to the positive displacement point to obtain the first extended image composed of the multiple pixel points after the movement. In the first extended image, except for the coordinate value of the pixel point of the regional image after the movement is true, the coordinates of other pixel points are set to transparent attributes, and are all transparent when displayed without color, so as not to cause display occlusion to other images, such as the background image.

[0074] The positive displacement point may include the original position point, that is, without displacing the regional image, a first extended image corresponding to the original position point is generated, which can ensure that the regional image at the original position can be displayed normally. The coordinates of the pixel points of the first extended image corresponding to the original position point do not change.

[0075] In the disclosed embodiment, the displacement speed of the target object can be obtained, and the positive displacement can be performed on the basis of the original position point of the target object to obtain at least one accurate positive displacement point. By using the at least one positive displacement point, the regional image is confirmed according to the positive displacement point to obtain the first extended image corresponding to the positive displacement point, improve the positioning accuracy of the first extended image, and obtain the first extended image corresponding to the at least one positive displacement point with higher positioning accuracy.

[0076] In a possible design, based on the original position point of the regional image, displacement processing is performed according to the displacement speed and the first expansion direction to obtain at least one positive displacement point including the original position point, which may include:

[0077] Determines the region width for region images and the image width for image frames.

[0078] Based on the region width and the image width, combined with the displacement speed, the forward displacement amount and the forward offset step length of the region image in the first expansion direction are calculated.

[0079] According to the original position of the regional image, at least one positive position point is determined according to each positive offset and the amount of positive displacement, and at least one positive displacement point consisting of the original position point and the at least one positive position point is obtained.

[0080] Optionally, the area width of the area image may refer to the horizontal width of the object contour area. The image width may also refer to the horizontal width of the image frame.

[0081] The number of positive displacements can be automatically calculated based on the region width and the image width. For example, the quotient of half the image width and the region width can be calculated, and then the result obtained can be rounded to obtain the number of positive displacements. However, in actual applications, the number of positive displacements can also be preset. For example, it can be set by an operation and maintenance personnel or by a front-end user using the video synthesis method. In a more common scenario, the number of positive displacements can be set to 3.

[0082] The forward offset step can be determined by taking the quotient of the width difference between half the image width and the area width and the forward displacement amount, with the displacement speed as a constraint. Usually, the quotient of the width difference and the forward displacement amount can be calculated, and then the quotient result can be rounded to obtain the initial forward step. If the displacement speed is greater than a certain threshold, the step is reduced, and if the displacement speed is less than a certain threshold, the step is increased.

[0083] In the embodiment of the present disclosure, when determining the positive displacement point, the forward displacement amount and the positive offset step in the first expansion direction can be accurately calculated based on the area width of the regional image and the image width of the image frame. The positive position point can be accurately located by the forward displacement amount and the positive offset step to obtain a positive displacement point with higher position accuracy.

[0084] In order to obtain an accurate second extended image, such as Figure 6 FIG. 1 is a flowchart of another embodiment of a video synthesis method provided by an embodiment of the present disclosure, which is different from other embodiments in that, the mirror image is extended in a second extension direction to obtain at least one second extended image, including:

[0085] 601: Determine the displacement speed of the target object corresponding to the image frame.

[0086] Some steps in this embodiment are the same as some steps in other embodiments, and will not be repeated here.

[0087] 602: Based on the original position point of the regional image, displacement processing is performed based on the displacement speed and the second expansion direction to obtain at least one reverse displacement point corresponding to the original position point.

[0088] The regional image may refer to marking the pixels belonging to the target object in the image frame to obtain the pixels of the regional image in the image frame. The coordinates of the pixels corresponding to the regional image may also be represented by the pixel coordinate system of the image frame. The pixels not marked as the regional image are the pixels of the background image.

[0089] Among them, the original position point of the regional image may refer to the pixel coordinate point corresponding to the key point of the target object in the image frame. The key point may refer to a pixel point that can confirm the position of the regional image, and the coordinates of the pixel point are represented by the pixel coordinate system of the entire image frame. The original position point may, for example, include the coordinate point of the center point of the regional image in the image frame. For example, when the target object is a human body, it may also be the coordinates of a key position point in the human body, such as the coordinates of the pixel points such as the center point of the eyebrows, the tip of the nose, and the center point. The meaning of the position corresponding to the original position point is not limited here. Similarly, the reverse displacement point may include the pixel coordinate point in the image frame obtained after the original position point is displaced. Both the original position point and the reverse displacement point may include coordinate points in the pixel coordinate system of the image frame.

[0090] Optionally, the second extension direction may include a reverse displacement angle. The second extension direction may be determined by the trajectory of the target object in the video frame. For example, taking the movement from position N to position M as an example, the line connecting the two positions may have an angle relative to the horizontal line or the vertical line, and the angle is the reverse displacement angle. The reverse displacement process is similar to the forward displacement and will not be described in detail here.

[0091] 603: Copy the mirror image to the reverse displacement point to obtain a second extended image corresponding to the mirror image at at least one reverse displacement point.

[0092] Copying the regional image to the reverse displacement point may include moving the pixel value corresponding to the pixel point of the regional image to the corresponding coordinate point according to the reverse displacement point, that is, the reverse coordinate point of the second extended image in the image frame may be determined according to the coordinate point of the regional image in the image frame and the reverse displacement point, and by assigning the pixel value at the coordinate point in the regional image to the corresponding reverse coordinate point, the second extended image composed of the pixel value corresponding to the reverse coordinate point can be obtained. Of course, in practical applications, the pixel points of the regional image may include multiple pixels, and each pixel point may be moved one by one according to the reverse displacement point to obtain a second extended image composed of multiple pixels after the movement. In the second extended image, except for the coordinate value of the pixel point of the regional image after the movement is true, the coordinates of other pixel points are set to transparent attributes, and are all transparent when displayed, without color, so as not to cause display occlusion to other images, such as the background image.

[0093] In the disclosed embodiment, the displacement speed of the target object can be obtained, and reverse displacement can be performed on the basis of the original position point of the target object to obtain at least one accurate reverse displacement point. By using the at least one reverse displacement point, the regional image is confirmed according to the reverse displacement point to obtain the second extended image corresponding to the reverse displacement point, improve the positioning accuracy of the second extended image, and obtain the second extended image corresponding to at least one reverse displacement point with higher positioning accuracy.

[0094] As an embodiment, based on the original position point of the regional image, displacement processing is performed based on the displacement speed and the second expansion direction to obtain at least one reverse displacement point corresponding to the original position point, including:

[0095] Determine the region width of the region image and the image width of the image frame;

[0096] Based on the area width and the image width, combined with the displacement speed, the reverse displacement amount and the reverse displacement step length of the area image in the second expansion direction are calculated;

[0097] According to the original position of the regional image, at least one reverse position point is determined according to the reverse offset step length and the reverse offset quantity, and at least one reverse displacement point formed by the at least one reverse position point is obtained.

[0098] Optionally, the area width of the area image may refer to the horizontal width of the object contour area. The image width may also refer to the horizontal width of the image frame.

[0099] The reverse displacement amount can be automatically calculated based on the region width and the image width. For example, the quotient of half the image width and the region width can be calculated, and then the result obtained can be rounded to obtain the reverse displacement amount. However, in actual applications, the reverse displacement amount can also be preset. For example, it can be set by the operation and maintenance personnel or by the front-end user who uses the video synthesis method. In more common scenarios, the reverse displacement amount can be set to 3.

[0100] The reverse offset step can be determined by taking the quotient of the width difference between half the image width and the area width and the reverse displacement amount, with the displacement speed as a constraint. Usually, the quotient of the width difference and the reverse displacement amount can be calculated, and then the quotient result can be rounded to obtain the initial reverse step. If the displacement speed is greater than a certain threshold, the step is reduced, and if the displacement speed is less than a certain threshold, the step is increased.

[0101] In the embodiment of the present disclosure, when determining the reverse displacement point, the reverse displacement amount and the reverse offset step in the second expansion direction can be accurately calculated based on the area width of the regional image and the image width of the image frame. The reverse position point can be accurately located by the reverse displacement amount and the reverse offset step to obtain a reverse displacement point with higher position accuracy.

[0102] As an embodiment, determining the displacement speed of the target object corresponding to the image frame includes:

[0103] Determine the previous image frame of the image frame corresponding to the video to be processed.

[0104] Determining a first position of the target object in a previous image frame and a second position of the target object in the image frame;

[0105] Calculating a displacement corresponding to the target object according to the first position and the second position;

[0106] The displacement speed of the target object in the image frame is calculated by using the image frame display time interval and displacement amount of the video to be processed.

[0107] Optionally, according to the first position and the second position, calculating the displacement corresponding to the target object may include: determining the frame rate of the video to be processed, calculating the time interval between two image frames according to the frame rate of the video to be processed, and calculating the position difference according to the first position and the second position. The position difference may refer to the absolute difference between the first position and the second position, which may be obtained by performing a triangular calculation on the lateral displacement difference and the longitudinal displacement difference of the two positions, that is, using the lateral difference and the longitudinal difference to coordinate the two right-angled sides of a right triangle, and the displacement is the length of the hypotenuse of the right triangle. The quotient of the displacement and the time interval may be the displacement speed.

[0108] In the disclosed embodiment, the first position of the target object in the previous image frame and the second position of the current image frame can be obtained, and the displacement corresponding to the target object can be calculated by calculating the first position and the second position. The displacement speed of the target image can be accurately calculated by the displacement and the time interval between the two image frames. The displacement speed of the target object can be accurately calculated by the displacement and the time interval between two adjacent image frames to obtain a high-precision displacement speed.

[0109] In order to ensure the accuracy of image display so that the user can normally view the target object, in a possible design, rendering the background image, at least one second extended image, and at least one first extended image to obtain a target rendered image at the end of rendering may include:

[0110] According to the first rendering order of the background image, the second rendering order of at least one second extended image and the third rendering order of at least one first extended image, the background image, at least one second extended image and at least one first extended image are superimposed and rendered in sequence to obtain the target rendered image when the rendering is completed.

[0111] Optionally, rendering the background image, the at least one second extended image, and the at least one first extended image to obtain a target rendered image at the end of rendering may include: rendering the background image, the at least one second extended image, and the at least one first extended image respectively to obtain the target rendered image at the end of rendering. The target rendered image may be obtained by rendering the background image, the at least one second extended image, and the at least one first extended image respectively.

[0112] In the disclosed embodiment, the rendering order may be set for the background image, at least one second extended image, and at least one first extended image, respectively, so as to achieve accurate rendering of the target rendered image, avoid image display overlap problems caused by random rendering between images, and ensure the effectiveness of the display.

[0113] As a possible implementation manner, sequentially superimposing and rendering a background image, at least one second extended image, and at least one first extended image to obtain a target rendered image at the end of rendering may include:

[0114] Render the background image at the bottom layer.

[0115] Determine reverse center distances corresponding to center points of at least one second extended image and the background image respectively;

[0116] Rendering at least one second extended image in sequence according to the reverse center distance from large to small;

[0117] Determine the forward center distances respectively corresponding to the center points of at least one second extended image and the background image;

[0118] At least one first extended image is rendered in sequence according to the order of forward center distance from large to small, until the rendering of the last first extended image is completed, so as to obtain a target rendered image.

[0119] The reverse center distance between the second extended image and the center point of the background image can be calculated by the reverse displacement point of the second extended image and the center point of the background image. Similarly, the forward center distance between the first extended image and the center point of the background image can be calculated by the forward displacement point of the first extended image and the center point of the background image.

[0120] The target rendered image can be obtained by multi-layer rendering of a background image, at least one second extended image, and at least one first extended image. Specifically, the background image can be rendered at the bottom layer, the second extended image can be rendered at the upper layer of the background image, and the first extended image can be rendered at the upper layer of the second extended image. There is also a rendering order in the rendering process of at least one second extended image, that is, the second extended image with the largest distance from the center point of the image is rendered first, and as the distance decreases, the rendering order becomes later and the rendering order becomes earlier. The rendering process of at least one first extended image is similar, that is, the first extended image with the largest distance from the center point of the image is rendered first, and as the distance decreases, the rendering order becomes later and the rendering order becomes earlier.

[0121] Of course, other rendering orders can also be used. For example, at least one second extended image and at least one first extended image are rendered in the same batch, and the rendering starts from the image corresponding to the largest distance from the center point until the rendering of the last first extended image is completed. The first extended image includes a regional image, and the regional image has the smallest distance from the center point. For ease of understanding, Figure 7 As shown, assuming that the regional image is shown as 701, after the regional image is expanded in the first expansion direction and the second expansion direction, at least one first expanded image displayed in the display screen 700 is 701-703, and at least one second expanded image is 704-705.

[0122] In the disclosed embodiment, the background image is rendered at the bottom layer to ensure that the background image is displayed normally. The first extended image and the second extended image can be rendered at the upper layer of the background image. When rendering the positive second extended image, the second extended image and the first extended image can be rendered to the upper layer in the order of the reverse center distance and the forward center distance from large to small, so that the rendering level of the image is related to the distance, and the image is accurately rendered, ensuring that the regional image can be rendered at the top layer, and obtaining a precisely rendered target rendering image.

[0123] like Figure 8 FIG. 1 is a flowchart of another embodiment of a video synthesis method provided by the present disclosure. In the video synthesis method, a texture image can be set for a regional image. Figure 8 , rendering the background image, at least one second extended image, and at least one first extended image, and obtaining a target rendered image after the rendering is completed, the video synthesis method in the present disclosure may further include:

[0124] 801: Obtain the sticker image selected by the user for the target object.

[0125] 802: Determine a mapping position of the mapping image in the image frame based on the original position point of the regional image in the image frame.

[0126] 803: Render the texture image into the target rendering image according to the texture position to obtain the target texture image.

[0127] 804: Perform video synthesis on the target texture image corresponding to the image frame to obtain a texture target video corresponding to the video to be processed.

[0128] The sticker image can be specified by the user who provides the video to be processed, and can refer to various types of stickers such as various cartoon images, landscape images, and photo stickers. Of course, the sticker position can also be specified by the user. The sticker position point can be located according to the original position point.

[0129] Based on the original position point of the regional image in the image frame, the position point where the head vertex is located in the regional image can be determined, and the position point where the head vertex is located can be used as the mapping position.

[0130] In the disclosed embodiment, after obtaining the map image selected by the user for the target object, the map position of the map image in the image frame can be determined based on the original position point of the regional image in the image frame. By determining the map position, the map image can be rendered into the target rendered image according to the map position to obtain the target map image, and the target map image corresponding to the image frame is synthesized into a video to obtain the map target video corresponding to the video to be processed. Mapping can make the content of the target video more comprehensive and improve the video display efficiency.

[0131] As an embodiment, performing image segmentation on an image frame in a video to be processed to obtain a region image corresponding to a target object and a background image other than the region image includes:

[0132] Extract image frames from the video to be processed.

[0133] Identify object contour regions of target objects in image frames.

[0134] The image frame is segmented using the object contour region to obtain a region image corresponding to the object contour region and a background image excluding the region image.

[0135] Optionally, a contour extraction algorithm may be used to identify an object contour region of a target object in an image frame. The contour extraction algorithm may be a common contour extraction algorithm, and the present disclosure does not make too many restrictions on the specific type of contour extraction algorithm. Image segmentation may specifically refer to setting the pixel points surrounded by the object contour region as the object identifier, and setting other pixel points outside the object contour region as the background identifier. The pixel points of the regional image and the background image may be distinguished by the pixel point identifier.

[0136] In the disclosed embodiment, an image frame can be extracted from the video to be processed, and the object contour area of ​​the target object in the image frame can be identified. The object contour area can be used to segment the image frame to obtain an accurate regional image and background image of the object contour area. Image segmentation can achieve accurate image segmentation and accurate acquisition of regional images and background images.

[0137] In one possible design, the method may further include:

[0138] Based on the target video acquisition request triggered by the first user, a video to be processed is obtained.

[0139] Performing video synthesis on the target rendered image corresponding to the image frame to obtain the target video corresponding to the video to be processed, and then further comprising:

[0140] A target video is output for the first user.

[0141] The video to be processed can be collected in real time or read from a stored video.

[0142] Among them, outputting the target video for the first user may include outputting the target video for the first user using an output device or sending the target video to a user device corresponding to the first user, and the output device in the user device outputs the target video.

[0143] In the disclosed embodiment, the video to be processed provided by the first user can be obtained, and after the target video is obtained, the target video can be output to the first user. The acquisition of the video to be processed and the output of the target video can be achieved by interactive processing with the first user, so as to provide the target video to the first user in a more efficient and automated mode.

[0144] As an embodiment, after outputting the target video for the first user, the method further includes:

[0145] Detecting a publishing request triggered by a first user for a target video;

[0146] In response to the publishing request, the target video is published to the user video account of the first user, and the target video is read and played by the second user device of the second user.

[0147] Optionally, the electronic device can provide the first user with a publishing control for the target video. When the user device of the first user detects that the first user triggers the publishing control, the user device can send a publishing request to the electronic device. The electronic device can respond to the publishing request to store the target video in the user video account of the first user. Specifically, it can be to store the target video and establish a storage association between the video name or video identifier of the target video and the account name of the first user. The user video account can correspond to multiple videos, which can include the target video. When the second user device needs to play the target video, the identity video is determined from the user video account of the first user, and the target video is sent to the second user device. Among them, the target video is played once, and the number of times the target video is played increases accordingly.

[0148] In the disclosed embodiment, a publishing request triggered by a first user for a target video can be detected, and the target video can be published to the user video account of the first user, and the target video can be read and played by a second user device of a second user. Through the first user's control over the publishing of the target video, the target video can be effectively published.

[0149] In some embodiments, detecting a target video acquisition request triggered by a first user and obtaining a to-be-processed video includes:

[0150] Receiving a video to be processed sent by a first user device; the video to be processed is acquired by the user device of the first user when the user triggers a target video acquisition request;

[0151] Outputting a target video for a first user, including:

[0152] The target video is sent to the first user device, and the first user device is controlled to play the target video.

[0153] Of course, in some embodiments, the electronic device can directly interact with the first user, collect the video to be processed, and synthesize the target video corresponding to the video to be processed based on the video synthesis method disclosed in the present invention, so as to complete the video collection and processing in a single electronic device and improve the utilization rate of the electronic device.

[0154] In the disclosed embodiment, a video to be processed sent by a first user device can be received, and the video to be processed can be collected by the user device of the first user. By automatically generating a target video for a video frame of the video to be processed, the generated target video can be sent to the first user device, so that the target video can be fed back to the first user device. The generation and output of the target video are separated, the processing pressure of the first user device is reduced, and the normal operation of the first user device is guaranteed. At the same time, using an electronic device with stronger computing power to complete the generation of the target video can make the generation efficiency and stability of the target video higher, thereby ensuring the generation efficiency and success rate of the target video.

[0155] like Fig. 9 FIG. 1 is a schematic diagram of a structure of an embodiment of a video synthesis device provided by an embodiment of the present disclosure. The video synthesis device 900 may include:

[0156] Image segmentation unit 901: used to perform image segmentation on the image frames in the video to be processed, and obtain a region image corresponding to the target object and a background image other than the region image.

[0157] The mirror processing unit 902 is used to determine the mirror image corresponding to the regional image.

[0158] The first expansion unit 903 is configured to perform expansion processing on the regional image in a first expansion direction to obtain at least one first expanded image; the at least one first expanded image includes the regional image.

[0159] The second expansion unit 904 is used to expand the mirror image in the second expansion direction to obtain at least one second expanded image. The first expansion direction and the second expansion direction are different

[0160] Image rendering unit 905: used for rendering the background image, at least one second extended image and at least one first extended image to obtain a target rendered image when rendering is completed.

[0161] The video synthesis unit 906 is used to perform video synthesis on the target rendered image corresponding to the image frame to obtain a target video corresponding to the video to be processed.

[0162] As an embodiment, the first expansion unit may include:

[0163] A speed determination module, used to determine the displacement speed of the target object corresponding to the image frame;

[0164] A first expansion module is used to perform displacement processing based on the original position point of the regional image according to the displacement speed and the first expansion direction to obtain at least one positive displacement point including the original position point;

[0165] The first determining module is used to copy the regional image to the positive displacement point to obtain the first extended image corresponding to the regional image at at least one positive displacement point.

[0166] In some embodiments, the first expansion module may include:

[0167] A first determination submodule is used to determine the region width of the region image and the image width of the image frame;

[0168] A first calculation submodule is used to calculate the forward displacement amount and the forward offset step length of the regional image in the first expansion direction based on the regional width and the image width in combination with the displacement speed;

[0169] The forward displacement submodule is used to determine at least one forward position point according to the original position of the regional image, the forward offset step length and the forward displacement amount, and obtain at least one forward displacement point composed of the original position point and the at least one forward position point.

[0170] As an embodiment, the second expansion unit may include:

[0171] A speed determination module, used to determine the displacement speed of the target object corresponding to the image frame;

[0172] A second expansion module is used to perform displacement processing based on the displacement speed and the second expansion direction based on the original position point of the regional image to obtain at least one reverse displacement point corresponding to the original position point;

[0173] The second determining module is used to copy the mirror image to the reverse displacement point to obtain a second extended image corresponding to the mirror image at at least one reverse displacement point.

[0174] In some embodiments, the second expansion module includes:

[0175] A second determination submodule is used to determine the region width of the region image and the image width of the image frame;

[0176] A second calculation submodule is used to calculate the reverse displacement amount and the reverse offset step length of the regional image in the second expansion direction based on the regional width and the image width in combination with the displacement speed;

[0177] The reverse displacement submodule is used to determine at least one reverse position point according to the original position of the regional image, the reverse offset step length and the reverse offset quantity, and obtain at least one reverse displacement point composed of the at least one reverse position point.

[0178] In some embodiments, the speed determination module includes:

[0179] A third determination module is used to determine the previous image frame of the image frame corresponding to the video to be processed;

[0180] A fourth determination module, used to determine a first position of the target object in a previous image frame and a second position of the target object in the image frame;

[0181] A displacement calculation module, used to calculate the displacement corresponding to the target object according to the first position and the second position;

[0182] The speed calculation module is used to calculate the displacement speed of the target object corresponding to the image frame by using the image frame display time interval and displacement of the video to be processed.

[0183] In some embodiments, the image rendering unit may include:

[0184] A sequential rendering module is used to sequentially superimpose and render a background image, at least one second extended image, and at least one first extended image in a first rendering order, according to which the background image is in a first rendering order, at least one second extended image is in a second rendering order, and at least one first extended image is in a third rendering order, to obtain a target rendered image when rendering is completed.

[0185] In some embodiments, the sequential rendering module includes:

[0186] A first rendering module, used for rendering the background image at the bottom layer;

[0187] A first distance module, used to determine the reverse center distances corresponding to the center points of at least one second extended image and the background image respectively;

[0188] A second rendering module, used for rendering at least one second extended image in sequence according to the reverse center distance from large to small;

[0189] A second distance module, used to determine the forward center distances corresponding to the center points of at least one second extended image and the background image respectively;

[0190] The second rendering module is used to render at least one first extended image in order of forward center distance from large to small, until the rendering of the last first extended image is completed, so as to obtain a target rendered image.

[0191] As yet another embodiment, the device further comprises:

[0192] A sticker acquisition unit, used to acquire a sticker image selected by a user for a target object;

[0193] A position determination unit, used to determine a mapping position of the mapping image in the image frame based on an original position point of the regional image in the image frame;

[0194] An image mapping unit, used for rendering a mapping image into a target rendering image according to a mapping position to obtain a target mapping image;

[0195] The texture video unit is used to perform video synthesis on the target texture image corresponding to the image frame to obtain the texture target video corresponding to the video to be processed.

[0196] In some embodiments, the image segmentation unit comprises:

[0197] An image extraction module, used for extracting image frames from the video to be processed;

[0198] A region recognition module, used for recognizing an object contour region of a target object in an image frame;

[0199] The image segmentation module is used to segment the image frame using the object contour area to obtain a regional image corresponding to the object contour area and a background image other than the regional image.

[0200] In some embodiments, further comprising:

[0201] A video acquisition unit, configured to acquire a to-be-processed video based on a target video acquisition request triggered by a first user;

[0202] The video output unit is used to output the target video for the first user.

[0203] As yet another embodiment, the device may further include:

[0204] A publishing detection unit, used to detect a publishing request triggered by a first user for a target video;

[0205] The request response unit is used to publish the target video to the user video account of the first user in response to the publishing request, and the target video is read and played by the second user device of the second user.

[0206] In some embodiments, the video acquisition unit includes:

[0207] The video receiving module is used to receive the to-be-processed video sent by the first user device; the to-be-processed video is acquired by the user device of the first user when the user triggers a target video acquisition request;

[0208] Video output unit, including:

[0209] The video sending module is used to send the target video to the first user equipment and control the first user equipment to play the target video.

[0210] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0211] In order to implement the above embodiment, the embodiment of the present disclosure also provides an electronic device.

[0212] Reference 10 shows a schematic diagram of the structure of an electronic device 1000 suitable for implementing an embodiment of the present disclosure, and the electronic device 1000 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.10 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0213] like Fig.10 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 to a random access memory (RAM) 1003. Various programs and data required for the operation of the electronic device 1000 are also stored in the RAM 1003. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0214] Typically, the following devices may be connected to the I / O interface 1005: input devices 1006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1008 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 may allow the electronic device 1000 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.10 The electronic device 1000 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0215] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 1009, or installed from a storage device 1008, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0216] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0217] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0218] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0219] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0220] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0221] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0222] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0223] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0224] In a first aspect, according to one or more embodiments of the present disclosure, a method for synthesizing a target video is provided, comprising:

[0225] Perform image segmentation on the image frames in the video to be processed to obtain a region image corresponding to the target object and a background image other than the region image;

[0226] Determine the mirror image corresponding to the regional image;

[0227] Performing an expansion process on the regional image in a first expansion direction to obtain at least one first expanded image; the at least one first expanded image includes the regional image;

[0228] The mirror image is expanded in a second expansion direction to obtain at least one second expanded image; the first expansion direction and the second expansion direction are different.

[0229] Rendering the background image, at least one second extended image, and at least one first extended image to obtain a target rendered image when rendering is completed;

[0230] The target rendered image corresponding to the image frame is subjected to video synthesis to obtain a target video corresponding to the video to be processed.

[0231] According to one or more embodiments of the present disclosure, performing expansion processing on a regional image in a first expansion direction to obtain at least one first expanded image includes:

[0232] Determine the displacement speed of the target object corresponding to the image frame;

[0233] Based on the original position point of the regional image, displacement processing is performed according to the displacement speed and the first expansion direction to obtain at least one positive displacement point including the original position point;

[0234] The regional image is copied to the positive displacement point to obtain a first extended image corresponding to the regional image at at least one positive displacement point.

[0235] According to one or more embodiments of the present disclosure, based on the original position point of the regional image, displacement processing is performed according to the displacement speed and the first expansion direction, and obtaining at least one positive displacement point including the original position point includes:

[0236] Determine the region width of the region image and the image width of the image frame;

[0237] Based on the area width and the image width, combined with the displacement speed, the forward displacement amount and the forward offset step length of the area image in the first expansion direction are calculated;

[0238] According to the original position of the regional image, at least one positive position point is determined according to the positive offset step length and the positive displacement amount, and at least one positive displacement point consisting of the original position point and the at least one positive position point is obtained.

[0239] According to one or more embodiments of the present disclosure, performing expansion processing on the mirror image in a second expansion direction to obtain at least one second expanded image includes:

[0240] Determine the displacement speed of the target object corresponding to the image frame;

[0241] Based on the original position point of the regional image, displacement processing is performed based on the displacement speed and the second expansion direction to obtain at least one reverse displacement point corresponding to the original position point;

[0242] The mirror image is copied to the reverse displacement point to obtain a second extended image corresponding to the mirror image at at least one reverse displacement point.

[0243] According to one or more embodiments of the present disclosure, based on the original position point of the regional image, displacement processing is performed based on the displacement speed and the second expansion direction to obtain at least one reverse displacement point corresponding to the original position point, including:

[0244] Determine the region width of the region image and the image width of the image frame;

[0245] Based on the area width and the image width, combined with the displacement speed, the reverse displacement amount and the reverse displacement step length of the area image in the second expansion direction are calculated;

[0246] According to the original position of the regional image, at least one reverse position point is determined according to the reverse offset step length and the reverse offset amount, and at least one reverse displacement point formed by the at least one reverse position point is obtained.

[0247] According to one or more embodiments of the present disclosure, determining the displacement speed of the target object corresponding to the image frame includes:

[0248] Determine the previous image frame of the image frame corresponding to the video to be processed;

[0249] Determining a first position of the target object in a previous image frame and a second position of the target object in the image frame;

[0250] Calculating a displacement corresponding to the target object according to the first position and the second position;

[0251] The displacement speed of the target object in the image frame is calculated by using the image frame display time interval and displacement amount of the video to be processed.

[0252] According to one or more embodiments of the present disclosure, rendering a background image, at least one second extended image, and at least one first extended image to obtain a target rendered image at the end of rendering includes:

[0253] According to the first rendering order of the background image, the second rendering order of at least one second extended image and the third rendering order of at least one first extended image, the background image, at least one second extended image and at least one first extended image are superimposed and rendered in sequence to obtain the target rendered image when the rendering is completed.

[0254] According to one or more embodiments of the present disclosure, sequentially superimposing a rendering background image, at least one second extended image, and at least one first extended image to obtain a target rendered image at the end of rendering includes:

[0255] Render the background image at the bottom layer;

[0256] Determine reverse center distances corresponding to center points of at least one second extended image and the background image respectively;

[0257] Rendering at least one second extended image in sequence according to the reverse center distance from large to small;

[0258] Determine the forward center distances respectively corresponding to the center points of at least one second extended image and the background image;

[0259] At least one first extended image is rendered in sequence according to the order of forward center distance from large to small, until the rendering of the last first extended image is completed, so as to obtain a target rendered image.

[0260] According to one or more embodiments of the present disclosure, after rendering the background image, at least one second extended image, and at least one first extended image, and obtaining a target rendered image at the end of rendering, the method further includes:

[0261] Get the sticker image selected by the user for the target object;

[0262] Determine a mapping position of the mapping image in the image frame based on the original position point of the regional image in the image frame;

[0263] Rendering the texture image into the target rendering image according to the texture position to obtain the target texture image;

[0264] The target texture image corresponding to the image frame is synthesized into a video to obtain a texture target video corresponding to the video to be processed.

[0265] According to one or more embodiments of the present disclosure, image segmentation is performed on image frames in a video to be processed to obtain a region image corresponding to a target object and a background image other than the region image, including:

[0266] Extracting image frames from the video to be processed;

[0267] identifying an object contour region of a target object in an image frame;

[0268] The image frame is segmented using the object contour region to obtain a region image corresponding to the object contour region and a background image excluding the region image.

[0269] According to one or more embodiments of the present disclosure, the present invention further includes:

[0270] Acquire a to-be-processed video based on a target video acquisition request triggered by a first user;

[0271] Performing video synthesis on the target rendered image corresponding to the image frame to obtain the target video corresponding to the video to be processed, and then further comprising:

[0272] A target video is output for the first user.

[0273] According to one or more embodiments of the present disclosure, after outputting the target video for the first user, the method further includes:

[0274] Detecting a publishing request triggered by a first user for a target video;

[0275] In response to the publishing request, the target video is published to the user video account of the first user, and the target video is read and played by the second user device of the second user.

[0276] According to one or more embodiments of the present disclosure, based on a target video acquisition request triggered by a first user, obtaining a to-be-processed video includes:

[0277] Receiving a video to be processed sent by a first user device; the video to be processed is acquired by the user device of the first user when the user triggers a target video acquisition request;

[0278] Outputting a target video for a first user, including:

[0279] The target video is sent to the first user device, and the first user device is controlled to play the target video.

[0280] In a second aspect, according to one or more embodiments of the present disclosure, there is provided a device for video synthesis, comprising:

[0281] An image segmentation unit is used to perform image segmentation on the image frame in the video to be processed to obtain a region image corresponding to the target object and a background image other than the region image;

[0282] A mirror processing unit, used for determining a mirror image corresponding to the regional image;

[0283] A first expansion unit, configured to perform expansion processing on the regional image in a first expansion direction to obtain at least one first expanded image; the at least one first expanded image includes the regional image;

[0284] The second expansion unit is used to expand the mirror image in a second expansion direction to obtain at least one second expanded image; the first expansion direction and the second expansion direction are different.

[0285] An image rendering unit, used for rendering the background image, at least one second extended image and at least one first extended image to obtain a target rendered image when rendering is completed;

[0286] The video synthesis unit is used to perform video synthesis on the target rendered image corresponding to the image frame to obtain a target video corresponding to the video to be processed.

[0287] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0288] Memory stores computer-executable instructions;

[0289] At least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the video synthesis method of the first aspect and various possible designs of the first aspect as described above.

[0290] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the video synthesis method as described in the first aspect and various possible designs of the first aspect are implemented.

[0291] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the video synthesis method as described in the first aspect and various possible designs of the first aspect.

[0292] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0293] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0294] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A video synthesis method, characterized in that: include: Performing image segmentation on the image frames in the video to be processed to obtain a region image corresponding to the target object and a background image other than the region image; Determine a mirror image corresponding to the regional image; Performing expansion processing on the regional image in a first expansion direction to obtain at least one first expanded image; at least one first expanded image includes the regional image; Performing an expansion process on the mirror image in a second expansion direction to obtain at least one second expanded image; the first expansion direction and the second expansion direction are different; Rendering the background image, at least one of the second extended images, and at least one of the first extended images to obtain a target rendered image when rendering is completed; Performing video synthesis on the target rendered image corresponding to the image frame to obtain a target video corresponding to the video to be processed; The step of performing expansion processing on the regional image in the first expansion direction to obtain at least one first expanded image includes: Determine the displacement speed of the target object corresponding to the image frame; Based on the original position point of the regional image, displacement processing is performed according to the displacement speed and the first expansion direction to obtain at least one positive displacement point including the original position point; The regional image is copied to the positive displacement point to obtain a first extended image of the regional image corresponding to at least one of the positive displacement points.

2. The method according to claim 1, characterized in that The method of taking the original position point of the regional image as a basis, performing displacement processing according to the displacement speed and the first expansion direction, and obtaining at least one positive displacement point including the original position point, comprises: Determining a region width of the region image and an image width of the image frame; Based on the region width and the image width, and in combination with the displacement speed, calculating the forward displacement amount and the forward offset step length of the region image in the first expansion direction; According to the original position of the regional image, at least one forward position point is determined according to the forward offset step length and the forward displacement amount, and the at least one forward displacement point consisting of the original position point and the at least one forward position point is obtained.

3. The method according to claim 1, characterized in that: The step of performing an expansion process on the mirror image in the second expansion direction to obtain at least one second expanded image includes: Determine the displacement speed of the target object corresponding to the image frame; Taking the original position point of the regional image as a basis, performing displacement processing based on the displacement speed and the second expansion direction to obtain at least one reverse displacement point corresponding to the original position point; The mirror image is copied to the reverse displacement point to obtain a second extended image corresponding to the mirror image at at least one of the reverse displacement points.

4. The method according to claim 3, characterized in that The method of taking the original position point of the regional image as a basis and performing displacement processing based on the displacement speed and the second expansion direction to obtain at least one reverse displacement point corresponding to the original position point includes: Determining a region width of the region image and an image width of the image frame; Based on the region width and the image width, and in combination with the displacement speed, calculating the reverse displacement amount and the reverse offset step length of the region image in the second expansion direction; According to the original position of the regional image, at least one reverse position point is determined according to the reverse offset step length and the reverse offset amount, and at least one reverse displacement point formed by at least one reverse position point is obtained.

5. The method according to claim 1 or 2, characterized in that: Determining the displacement speed of the target object corresponding to the image frame includes: Determine the image frame that is a previous image frame corresponding to the video to be processed; Determine a first position of the target object in the previous image frame and a second position in the image frame; Calculating a displacement corresponding to the target object according to the first position and the second position; The displacement speed of the target object corresponding to the image frame is calculated by using the image frame display time interval of the video to be processed and the displacement amount.

6. The method according to claim 1, characterized in that The rendering of the background image, at least one of the second extended images, and at least one of the first extended images to obtain a target rendered image when rendering is completed includes: According to the first rendering order of the background image, the second rendering order of at least one of the second extended images and the third rendering order of at least one of the first extended images, the background image, at least one of the second extended images and at least one of the first extended images are superimposed and rendered in sequence to obtain the target rendered image at the end of rendering.

7. The method according to claim 6, characterized in that The sequentially superimposing and rendering the background image, at least one of the second extended images, and at least one of the first extended images to obtain the target rendered image when rendering is completed includes: Rendering the background image at the bottom layer; Determine the reverse center distances respectively corresponding to the center points of at least one of the second extended images and the background image; Rendering at least one of the second extended images in sequence according to the reverse center distance from large to small; Determine a positive center distance corresponding to a center point of at least one of the second extended images and the background image respectively; At least one of the first extended images is rendered in sequence according to the order of forward center distance from large to small, until the rendering of the last first extended image is completed, so as to obtain the target rendered image.

8. The method according to claim 1, characterized in that After rendering the background image, at least one of the second extended images and at least one of the first extended images to obtain a target rendered image when rendering is completed, the method further includes: Obtaining a sticker image selected by a user for the target object; Determining a mapping position of the mapping image in the image frame based on an original position point of the regional image in the image frame; Rendering the texture image into the target rendered image according to the texture position to obtain a target texture image; The target texture image corresponding to the image frame is subjected to video synthesis to obtain a texture target video corresponding to the video to be processed.

9. The method according to claim 1, characterized in that: The image frame in the video to be processed is segmented to obtain a region image corresponding to the target object and a background image other than the region image, including: Extracting image frames from the video to be processed; identifying an object contour region of a target object in the image frame; The image frame is segmented using the object contour region to obtain a region image corresponding to the object contour region and a background image excluding the region image.

10. The method according to claim 1, characterized in that Also includes: Based on the target video acquisition request triggered by the first user, acquiring the video to be processed; The target rendered image corresponding to the image frame is subjected to video synthesis to obtain a target video corresponding to the video to be processed, and then the process further includes: The target video is output for the first user.

11. The method according to claim 10, characterized in that After outputting the target video for the first user, the method further includes: Detecting a publishing request triggered by the first user for the target video; In response to the publishing request, the target video is published to the user video account of the first user, and the target video is read and played by a second user device of the second user.

12. The method according to claim 10, characterized in that The acquiring the to-be-processed video based on the target video acquisition request triggered by the first user comprises: receiving the to-be-processed video sent by the first user equipment; the to-be-processed video is acquired by the user equipment of the first user when the user triggers a target video acquisition request; Outputting the target video for the first user includes: The target video is sent to a first user device, and the first user device is controlled to play the target video.

13. A video synthesis device, characterized in that: include: An image segmentation unit is used to perform image segmentation on the image frame in the video to be processed to obtain a region image corresponding to the target object and a background image other than the region image; A mirror processing unit, used to determine a mirror image corresponding to the regional image; A first expansion unit, configured to perform expansion processing on the regional image in a first expansion direction to obtain at least one first expanded image; at least one first expanded image includes the regional image; A second expansion unit, configured to expand the mirror image in a second expansion direction to obtain at least one second expanded image; the first expanded image and the second expanded image are different; An image rendering unit, used for rendering the background image, at least one of the second extended images and at least one of the first extended images to obtain a target rendered image when rendering is completed; A video synthesis unit, used for performing video synthesis on the target rendered image corresponding to the image frame to obtain a target video corresponding to the video to be processed; A first expansion unit, specifically configured to determine a displacement speed of the target object corresponding to the image frame; Based on the original position point of the regional image, displacement processing is performed according to the displacement speed and the first expansion direction to obtain at least one positive displacement point including the original position point; The regional image is copied to the positive displacement point to obtain a first extended image of the regional image corresponding to at least one of the positive displacement points.

14. An electronic device, characterized in that: include: Processor, memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor is configured with the video synthesis method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the video synthesis method according to any one of claims 1 to 12 is implemented.

16. A computer program product comprising a computer program, characterized in that The computer program is executed by a processor to configure the video synthesis method according to any one of claims 1 to 12.

Images