Video clip display method and device, computer equipment and storage medium

Abstract

The invention relates to a video clip display method and device, computer equipment and a storage medium. A target tracking frame of a target object on a current video frame in a spherical panoramic video clip is acquired, a reference bounding box of the current video frame is acquired, and a real-time position relation of the target object on the current video frame is determined according to the target tracking frame and the reference bounding box; and adjusting the position of the target object on the current video frame according to the real-time position relationship, and determining a special effect image of the target object according to the identifier of the current video frame, thereby displaying a preview image corresponding to the current video frame based on the adjusted position and the special effect image. And generating a planar video clip corresponding to the spherical panoramic video clip according to each preview image. According to the invention, the position of the target object on the current video frame is adjusted according to the real-time position relationship of the target object on the current video frame, so that the position of the target object on the current video frame is dynamically changed without depending on the shooting view angle of the spherical panoramic video clip.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a video clip display method, device, computer equipment and storage medium. Background Art

[0002] Special effects images are added to images to generate images with special effects. Video clips generated based on the images with special effects can provide users with rich visual experiences. For example, photoelectric surround special effects are added to images of moving characters and dynamic special effects are added to the character subject. When generating video clips, a 3D photoelectric surround effect can be formed around the character subject.

[0003] However, the position of the target object in the currently generated video clip in the video frame depends on the shooting angle of the spherical panoramic video clip, and it is difficult to control the position of the target object in the video frame. Summary of the invention

[0004] Based on this, it is necessary to provide a video clip display method, device, computer equipment and storage medium that can control the position of a target object in a video frame in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for displaying a video clip, the method comprising:

[0006] Obtaining a target tracking frame of a target object on a current video frame in a spherical panoramic video clip, and obtaining a reference bounding box of the current video frame;

[0007] Determining a real-time position relationship of the target object on the current video frame according to the target tracking frame and the reference bounding box;

[0008] adjusting the position of the target object on the current video frame according to the real-time position relationship, and determining a special effect image of the target object according to an identifier of the current video frame;

[0009] The preview image corresponding to the current video frame is displayed based on the adjusted position and the special effect image, so as to generate a planar video segment corresponding to the spherical panoramic video segment according to each of the preview images.

[0010] In one of the embodiments, adjusting the position of the target object on the current video frame according to the real-time position relationship includes:

[0011] Acquire first coordinates of each first pixel point of the target tracking frame and second coordinates of each second pixel point of the reference bounding box;

[0012] For each of the first coordinates, if the distance between a preset number of the first coordinates and each of the second coordinates is less than a preset threshold, the position of the target object on the current video frame is adjusted.

[0013] In one embodiment, determining the special effect image of the target object according to the identifier of the current video frame includes:

[0014] Determine, according to the identifier of the current video frame, a sticker foreground and a sticker background corresponding to the current video frame;

[0015] The special effect image is generated according to the target image of the target object, the sticker foreground and the sticker background.

[0016] In one embodiment, determining the sticker foreground and sticker background corresponding to the current video frame according to the identifier of the current video frame includes:

[0017] Obtain a target sticker corresponding to the current video frame according to an identifier of the current video frame and a ratio of a first frame number to a second frame number; the first frame number is the number of video frames in the spherical panoramic video clip, and the second frame number is the number of stickers in the sticker sequence;

[0018] A sticker foreground and a sticker background corresponding to the current video frame are determined according to the target sticker.

[0019] In one embodiment, generating the special effect image according to the target image of the target object, the sticker foreground and the sticker background includes:

[0020] Determining a mask image according to the target image;

[0021] Determine a special effect foreground according to the target image and the sticker foreground;

[0022] Determine a special effect background according to the target image and the sticker background;

[0023] The special effect image is generated according to the mask image, the special effect foreground and the special effect background.

[0024] In one embodiment, determining the special effect foreground according to the target image and the sticker foreground includes:

[0025] Determine a first difference between a preset value and a first transparency parameter value of the sticker foreground;

[0026] Determine a first product result of a color pixel value of the target image and the first difference value;

[0027] Determine a second product result of the color pixel value of the sticker foreground and the first transparency parameter value;

[0028] The special effect foreground is determined according to a summation result of the first multiplication result and the second multiplication result.

[0029] In one embodiment, determining the special effect background according to the target image and the sticker background includes:

[0030] Determine a second difference between the preset value and a second transparency parameter value of the sticker background;

[0031] Determine a third product result of the color pixel value of the target image and the second difference value;

[0032] Determine a fourth product result of the color pixel value of the sticker background and the second transparency parameter value;

[0033] The special effect background is determined according to a sum result of the third multiplication result and the fourth multiplication result.

[0034] In one embodiment, generating the special effect image according to the mask image, the special effect foreground and the special effect background includes:

[0035] Determining a third difference between the preset value and the mask image;

[0036] Determine a fifth product result of the special effect prospect and the third difference;

[0037] Determine a sixth product result of the special effect background and the mask image;

[0038] The special effect image is determined according to a sum result of the fifth multiplication result and the sixth multiplication result.

[0039] In one embodiment, the method further comprises:

[0040] Tracking a target tracking frame of the target object in a previous video frame in the spherical panoramic video clip to obtain a target tracking frame of the current video frame;

[0041] According to the target tracking frame of the current video frame, a target object in the current video frame is segmented to obtain a target image of the target object.

[0042] In one embodiment, the method further comprises:

[0043] storing the mask image and the adjusted position of the target object on each of the video frames;

[0044] The special effect image is generated according to the stored mask image, and based on the special effect image and the stored adjusted position, the preview image corresponding to each of the video frames is played back to generate a planar video segment corresponding to the spherical panoramic video segment according to each of the preview images.

[0045] In a second aspect, the present application further provides a video clip display device, the device comprising:

[0046] An acquisition module, used to acquire a target tracking frame of a target object on a current video frame in a spherical panoramic video clip, and to acquire a reference bounding box of the current video frame;

[0047] A first determination module, configured to determine a real-time position relationship of the target object on the current video frame according to the target tracking frame and the reference bounding frame;

[0048] A second determination module, configured to adjust the position of the target object on the current video frame according to the real-time position relationship, and determine a special effect image of the target object according to an identifier of the current video frame;

[0049] The display module is used to display the preview image corresponding to the current video frame based on the adjusted position and the special effect image, so as to generate a planar video segment corresponding to the spherical panoramic video segment according to each of the preview images.

[0050] In a third aspect, the present application further provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0051] Obtaining a target tracking frame of a target object on a current video frame in a spherical panoramic video clip, and obtaining a reference bounding box of the current video frame;

[0052] Determining a real-time position relationship of the target object on the current video frame according to the target tracking frame and the reference bounding box;

[0053] adjusting the position of the target object on the current video frame according to the real-time position relationship, and determining a special effect image of the target object according to an identifier of the current video frame;

[0054] The preview image corresponding to the current video frame is displayed based on the adjusted position and the special effect image, so as to generate a planar video segment corresponding to the spherical panoramic video segment according to each of the preview images.

[0055] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0056] Obtaining a target tracking frame of a target object on a current video frame in a spherical panoramic video clip, and obtaining a reference bounding box of the current video frame;

[0057] Determining a real-time position relationship of the target object on the current video frame according to the target tracking frame and the reference bounding box;

[0058] adjusting the position of the target object on the current video frame according to the real-time position relationship, and determining a special effect image of the target object according to an identifier of the current video frame;

[0059] The preview image corresponding to the current video frame is displayed based on the adjusted position and the special effect image, so as to generate a planar video segment corresponding to the spherical panoramic video segment according to each of the preview images.

[0060] In a fifth aspect, the present application further provides a computer program product, the computer program product comprising a computer program, which implements the following steps when executed by a processor:

[0061] Obtaining a target tracking frame of a target object on a current video frame in a spherical panoramic video clip, and obtaining a reference bounding box of the current video frame;

[0062] Determining a real-time position relationship of the target object on the current video frame according to the target tracking frame and the reference bounding box;

[0063] adjusting the position of the target object on the current video frame according to the real-time position relationship, and determining a special effect image of the target object according to an identifier of the current video frame;

[0064] The preview image corresponding to the current video frame is displayed based on the adjusted position and the special effect image, so as to generate a planar video segment corresponding to the spherical panoramic video segment according to each of the preview images.

[0065] The above-mentioned video clip display method, device, computer equipment and storage medium obtain the target tracking frame of the target object on the current video frame in the spherical panoramic video clip, and obtain the reference bounding frame of the current video frame, determine the real-time position relationship of the target object on the current video frame according to the target tracking frame and the reference bounding frame, adjust the position of the target object on the current video frame according to the real-time position relationship, and determine the special effect image of the target object according to the identifier of the current video frame, so as to display the preview image corresponding to the current video frame based on the adjusted position and special effect image, so as to generate the plane video clip corresponding to the spherical panoramic video clip according to each preview image. In the embodiment of the present application, according to the real-time position relationship of the target object on the current video frame, the position of the target object on the current video frame is adjusted to reach the specified position, so that the position of the target object on the current video frame can be dynamically changed, forming a dynamic effect of looking ahead and suddenly behind, which is independent of the viewing angle of the spherical panoramic video clip taken. Moreover, each video frame in the spherical panoramic video clip is asynchronously processed to obtain a preview image, and the preview image with special effects is displayed, so as to generate the plane video clip corresponding to the spherical panoramic video clip according to each preview image, and maintain real-time preview. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A diagram showing an application environment of a method for displaying video clips in an embodiment;

[0067] Figure 2 is a schematic flow chart of a method for displaying video clips in one embodiment;

[0068] Figure 3 is a schematic diagram of a video frame in a spherical panoramic video clip in one embodiment;

[0069] Figure 4 is a schematic diagram of a video frame in a spherical panoramic video clip in another embodiment;

[0070] Figure 5 is a preview image corresponding to the current video frame in one embodiment;

[0071] Figure 6 A schematic diagram of a flow chart of adjusting the position of a target object on a current video frame in one embodiment;

[0072] Figure 7 A schematic diagram of a process of determining a special effect image of a target object in one embodiment;

[0073] Figure 8 A schematic diagram of a process of determining a sticker foreground and a sticker background corresponding to a current video frame in one embodiment;

[0074] Fig. 9 A schematic diagram of a process for generating a special effect image in one embodiment;

[0075] Fig.10 A schematic diagram of a process of determining a target image of a target object in one embodiment;

[0076] Fig.11 is a structural block diagram of a video clip display device in one embodiment;

[0077] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0079] The image display method provided in the embodiment of the present application can be applied to Figure 1 The application environment includes a computer device, which may be a server, and its internal structure diagram may be as shown in FIG. Figure 1 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video clip display related data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a video clip display method is implemented. The server can be implemented by an independent server or a server cluster consisting of multiple servers.

[0080] In one embodiment, Figure 2 As shown, a video clip display method is provided, which is applied to Figure 1 The computer device in the example is used to illustrate, including the following steps:

[0081] S201, obtaining a target tracking frame of a target object on a current video frame in a spherical panoramic video segment, and obtaining a reference bounding box of the current video frame.

[0082] The target object includes a human body or an animal. There may be only one object or multiple objects in each video frame.

[0083] In this embodiment, obtaining a target tracking frame of a target object on a current video frame in a spherical panoramic video clip may include: if the current video frame is a first video frame, an AI algorithm may be used to detect an object for the first video frame, the area where the detected object is located is represented by a tracking frame, and the tracking frame with the largest area among the tracking frames is used as the target tracking frame.

[0084] If the current video frame is a video frame other than the first video frame, a target tracking algorithm may be used to track the target tracking frame of the first video frame to obtain the target tracking frames of the target objects in the other video frames.

[0085] The current video frame can be identified to obtain a reference bounding box of the current video frame, or parameters can be set before shooting a video clip to preset the center coordinates, width and height of the current video frame to directly obtain a reference bounding box of the current video frame.

[0086] S202: Determine the real-time position relationship of the target object on the current video frame according to the target tracking frame and the reference bounding box.

[0087] In this embodiment, the real-time position relationship of the target object on the current video frame can be determined by comparing the first center coordinate of the target tracking frame with the second center coordinate of the reference bounding box. For example, if the x coordinate of the first center coordinate is less than the x coordinate of the second center coordinate, and the y coordinate of the first center coordinate is equal to the y coordinate of the second center coordinate, then it is determined that the target object is on the left side of the current video frame, and the specific position information of the target object on the current video frame is determined based on the specific difference between the x coordinate of the first center coordinate and the x coordinate of the second center coordinate, such as Figure 3 As shown, Figure 3 Schematic diagram of a video frame in a spherical panoramic video clip in an embodiment, in which a target object is located at the lower left of a reference bounding box.

[0088] In this embodiment, the real-time position relationship of the target object on the current video frame may also be determined according to the coordinates of the pixel points on the target tracking frame and the coordinates of the pixel points on the reference boundary frame.

[0089] S203, adjusting the position of the target object on the current video frame according to the real-time position relationship, and determining a special effect image of the target object according to the identifier of the current video frame.

[0090] Optionally, the identifier of the current video frame may be a frame number, or the time of the current video frame, etc.

[0091] In this embodiment, the real-time position relationship can be compared with the preset position relationship. When the real-time position relationship is inconsistent with the preset position relationship, the position of the target object is adjusted so that the target object is located at the preset position of the current video frame, such as Figure 4 As shown, Figure 4 is a schematic diagram of a video frame in a spherical panoramic video clip in another embodiment, Figure 4 It is through Figure 3 For example, the preset position relationship is a relationship in which the first center coordinates and the second center coordinates completely coincide with each other. If the real-time position relationship is that the x coordinate of the first center coordinates is smaller than the x coordinate of the second center coordinates, the target tracking frame is adjusted to the right until the x coordinate of the first center coordinates is consistent with the x coordinate of the second center coordinates.

[0092] Adjusting the position of the target object on the current video frame according to the real-time position relationship may also be to move the viewing angle of the current video frame to adjust the position of the target object.

[0093] Based on the identifier of the current video frame, a target sticker corresponding to the current video frame can be determined from the sticker sequence, the target sticker is added to the target object, and the special effect image of the target object is determined.

[0094] S204 , displaying a preview image corresponding to the current video frame based on the adjusted position and special effect image, so as to generate a planar video segment corresponding to the spherical panoramic video segment according to each preview image.

[0095] In this embodiment, according to the adjusted position of the target object, the target object in the current video frame is replaced with the special effect image to obtain a preview image with special effects. For each video frame in the spherical panoramic video clip, according to the above method, a preview image of the first video frame is first obtained, and after the preview image of the first video frame is obtained, it is displayed, and the iterative steps are continued to obtain the preview image of the second video frame and display the preview image, so as to obtain a real-time preview of the plane video clip, such as Figure 5 As shown, Figure 5 It is a preview image corresponding to the current video frame in an embodiment. The preview image is a plane image, so that a plane video segment corresponding to the spherical panoramic video segment is generated according to each preview image. This process does not generate an intermediate video, and directly adds a special effect image when playing the original spherical panoramic video segment. What you see is what you get.

[0096] In the above-mentioned video clip display method, by obtaining the target tracking frame of the target object on the current video frame in the spherical panoramic video clip, and obtaining the reference bounding frame of the current video frame, the real-time position relationship of the target object on the current video frame is determined according to the target tracking frame and the reference bounding frame, the position of the target object on the current video frame is adjusted according to the real-time position relationship, and the special effect image of the target object is determined according to the identifier of the current video frame, so that the preview image corresponding to the current video frame is displayed based on the adjusted position and the special effect image, so as to generate the plane video clip corresponding to the spherical panoramic video clip according to each preview image. In the embodiment of the present application, according to the real-time position relationship of the target object on the current video frame, the position of the target object on the current video frame is adjusted to reach the specified position, so that the position of the target object on the current video frame can be dynamically changed, forming a dynamic effect of looking ahead and behind, which is independent of the viewing angle of the spherical panoramic video clip taken. Moreover, each video frame in the spherical panoramic video clip is asynchronously processed to obtain a preview image, and the preview image with special effects is displayed, so as to generate the plane video clip corresponding to the spherical panoramic video clip according to each preview image, and maintain real-time preview.

[0097] Figure 6 FIG. 1 is a flow chart of adjusting the position of a target object on a current video frame in one embodiment. Figure 6 As shown, the embodiment of the present application relates to a possible implementation method of how to adjust the position of the target object on the current video frame according to the real-time position relationship, including the following steps:

[0098] S601, obtaining first coordinates of each first pixel point of the target tracking box and second coordinates of each second pixel point of the reference bounding box.

[0099] In this embodiment, based on the above embodiment, the first coordinates of each first pixel point of the target tracking frame can be obtained according to the first center coordinates, width and height of the target tracking frame, and the second coordinates of each second pixel point can be obtained according to the second center coordinates, width and height of the reference bounding box.

[0100] S602: For each first coordinate, if the distance between a preset number of first coordinates and each second coordinate is less than a preset threshold, adjust the position of the target object on the current video frame.

[0101] In this embodiment, for each first coordinate, the distance between the first coordinate and the second coordinate is calculated. For example, the first coordinate includes 10 coordinate points in total, and the second coordinate includes 60 coordinate points in total. The distance between the first coordinate a1 and each second coordinate is calculated to obtain 60 distance values, and then the distance between the first coordinate a2 and each second coordinate is calculated to obtain 60 distance values, and so on, to obtain 600 distance values. If there are 200 distance values ​​less than the preset threshold value among the 600 distance values, the position of the target object on the current video frame is adjusted.

[0102] In a possible implementation, the minimum distance value may be determined among the 60 distance values ​​between the first coordinate a1 and each second coordinate, and the minimum distance value may be determined among the 60 distance values ​​between the first coordinate a2 and each second coordinate, to obtain 10 minimum distance values. If 3 of the 10 minimum distance values ​​are less than a preset threshold, the position of the target object on the current video frame is adjusted.

[0103] In an embodiment of the present application, the first coordinates of each first pixel point of the target tracking frame and the second coordinates of each second pixel point of the reference boundary frame are obtained. For each first coordinate, if the distance between a preset number of first coordinates and each second coordinate is less than a preset threshold, the position of the target object on the current video frame is adjusted. It can be moved according to a specific control rule, so that the position of the target object in the current video frame changes dynamically, thereby enhancing the freshness of the preview image.

[0104] Figure 7 FIG. 1 is a flow chart of determining a special effect image of a target object in one embodiment. Figure 7 As shown, the embodiment of the present application relates to a possible implementation method of how to determine a special effect image of a target object according to an identifier of a current video frame, including the following steps:

[0105] S701, determining a sticker foreground and a sticker background corresponding to the current video frame according to an identifier of the current video frame.

[0106] In this embodiment, according to the identifier of the current video frame, the target sticker corresponding to the current video frame is first determined from the sticker sequence, and then the sticker foreground and sticker background corresponding to the current video frame are determined according to the target sticker.

[0107] In a possible implementation, the sticker foreground corresponding to the current video frame may be determined from the identification sticker foreground sequence of the current video frame, and the sticker background corresponding to the current video frame may be determined from the sticker foreground background sequence.

[0108] S702, generating a special effect image according to the target image of the target object, the sticker foreground and the sticker background.

[0109] In this embodiment, a special effects foreground is generated based on the target image of the target object and the sticker foreground, and a special effects background is generated based on the target image of the target object and the sticker background, so that the special effects foreground and the special effects background are fused according to the target image to obtain a special effects image.

[0110] In an embodiment of the present application, the sticker foreground and sticker background corresponding to the current video frame are determined according to the identification of the current video frame, and a special effects image is generated according to the target image, sticker foreground and sticker background of the target object, thereby laying the foundation for the subsequent display of a preview image based on the special effects image. It not only supports the specification of the target object for adding special effects, but also makes the gameplay more interesting and entertaining.

[0111] Figure 8 FIG. 1 is a flow chart of determining a sticker foreground and a sticker background corresponding to a current video frame in one embodiment. Figure 8 As shown, the embodiment of the present application relates to a possible implementation method of how to determine a sticker foreground and a sticker background corresponding to a current video frame according to an identifier of the current video frame, including the following steps:

[0112] S801, obtain the target sticker corresponding to the current video frame according to the identifier of the current video frame and the ratio of the first frame number to the second frame number; the first frame number is the number of video frames in the video clip, and the second frame number is the number of stickers in the sticker sequence.

[0113] In this embodiment, if the first number of video frames in the spherical panoramic video clip is 150 frames, and the second number of sticker frames in the sticker sequence is 30 frames, then the ratio of the first number to the second number is 5.

[0114] If the identifier of the current video frame is a frame number, the target sticker corresponding to the video frames of the 1st to 5th frames is the first frame sticker in the sticker sequence, and the target sticker corresponding to the video frames of the 6th to 10th frames is the second frame sticker in the sticker sequence. That is, when the video frames of the 1st to 5th frames are displayed, the first frame sticker is displayed in a loop, and when the video frames of the 5th frame are updated, one frame sticker is updated.

[0115] If the identifier of the current video frame is time, the frame number of the current video frame is calculated according to the time of the current video frame. For example, the formula "frame number = time / 1000*frame rate" is used, and then the target sticker corresponding to the current video frame is obtained according to the frame number and the ratio of the number of first frames to the number of second frames.

[0116] S802, determining a sticker foreground and a sticker background corresponding to the current video frame according to the target sticker.

[0117] In this embodiment, the image may be segmented, masked, and processed to obtain a sticker foreground and sticker background corresponding to the current video frame.

[0118] In the embodiment of the present application, the target sticker corresponding to the current video frame is obtained according to the identifier of the current video frame and the ratio of the number of first frames to the number of second frames, and the sticker foreground and sticker background corresponding to the current video frame are determined according to the target sticker. In the embodiment of the present application, in order to address the defect that the duration of stickers is fixed and extension does not take effect, the identifier of the current video frame and the ratio of the number of first frames to the number of second frames are used to obtain the target sticker corresponding to the current video frame, and the stickers are played in a loop to achieve a one-to-one correspondence between video frames and stickers.

[0119] Fig. 9 FIG. 1 is a flow chart of generating a special effect image in one embodiment. Fig. 9 As shown, the embodiment of the present application relates to a possible implementation method of how to generate a special effect image according to a target image, a sticker foreground, and a sticker background of a target object, including the following steps:

[0120] S901, determining a mask image according to a target image.

[0121] In this embodiment, the target image is first converted into a grayscale image, and then the grayscale image is converted into a three-channel image so that the R, G, and B values ​​of each pixel are equal. The pixel value of the background area in the target image is set to 0 and the pixel value of the target object is set to 255 using a binarization method to obtain a mask image of the target image.

[0122] S902, determining a special effect foreground according to the target image and the sticker foreground.

[0123] In this embodiment, the target image is a color image including three channels of R, G, and B, and the color pixel values ​​of R, G, and B are normalized. The sticker foreground includes not only the color pixel values ​​of R, G, and B, but also the alpha transparency parameter value. The special effect foreground can be determined according to the color pixel values ​​of the target image, the color pixel values ​​of the sticker foreground, and the transparency parameter value of the sticker foreground.

[0124] S903, determining a special effect background according to the target image and the sticker background.

[0125] In this embodiment, the specific implementation method can refer to the above S902, and this embodiment of the present application will not be elaborated on this.

[0126] S904, generating a special effect image according to the mask image, the special effect foreground and the special effect background.

[0127] In this embodiment, the product results of multiplying the mask image and the special effect foreground and the product results of multiplying the mask image and the special effect background are added together to achieve the fusion of the special effect foreground and special effect background images to obtain a special effect image.

[0128] In an embodiment of the present application, a mask image is determined based on a target image, a special effects foreground is determined based on the target image and a sticker foreground, and a special effects background is determined based on the target image and a sticker background, thereby generating a special effects image based on the mask image, the special effects foreground, and the special effects background, thereby replacing the target object in the video frame based on the special effects image, making the generated preview image information richer and bringing better visual effects to users.

[0129] In one embodiment, determining a special effect foreground based on a target image and a sticker foreground includes: determining a first difference between a preset value and a first transparency parameter value of the sticker foreground; determining a first product result of a color pixel value of the target image and the first difference; determining a second product result of a color pixel value of the sticker foreground and the first transparency parameter value; and determining the special effect foreground based on the sum of the first product result and the second product result.

[0130] In this embodiment, the special effect foreground can be calculated using the following formula: resFg(i,j).rgb=person_frame(i,j).rgb*(1.0-fg_sticker(i,j).a)+fg_sticker(i,j).rgb*fg_sticker(i,j).a.

[0131] Among them, resFg(i,j).rgb is the special effect foreground, person_frame(i,j).rgb is the color pixel value of the target image, fg_sticker(i,j).a is the first transparency parameter value of the sticker foreground, fg_sticker(i,j).rgb is the color pixel value of the sticker foreground, i is the width of the special effect foreground / target image / sticker foreground, and j is the height of the special effect foreground / target image / sticker foreground.

[0132] It should be noted that the color pixel value and the first transparency parameter value of the above-mentioned special effect foreground / target image / sticker foreground are normalized, so the maximum value of the color pixel value and the first transparency parameter value is 1.

[0133] In one embodiment, a special effects background is determined based on a target image and a sticker background, including: determining a second difference between a preset value and a second transparency parameter value of the sticker background; determining a third product result of a color pixel value of the target image and the second difference; determining a fourth product result of a color pixel value of the sticker background and the second transparency parameter value; and determining the special effects background based on the sum of the third product result and the fourth product result.

[0134] In this embodiment, the special effect background can be calculated using the following formula: reBg(i,j).rgb=person_frame(i,j).rgb*(1.0-bg_sticker(i,j).a)+bg_sticker(i,j).rgb*bg_sticker(i,j).a.

[0135] Among them, reBg(i,j).rgb is the special effect background, person_frame(i,j).rgb is the color pixel value of the target image, bg_sticker(i,j).a is the second transparency parameter value of the sticker background, bg_sticker(i,j).rgb is the color pixel value of the sticker background, i is the width of the special effect background / target image / sticker background, and j is the height of the special effect background / target image / sticker background.

[0136] It should be noted that the color pixel value and the second transparency parameter value of the above-mentioned special effect background / target image / sticker background are also normalized, so the maximum value of the color pixel value and the second transparency parameter value is 1.

[0137] In one embodiment, a special effect image is generated based on a mask image, a special effect foreground, and a special effect background, including: determining a third difference between a preset value and the mask image; determining a fifth multiplication result of the special effect foreground and the third difference; determining a sixth multiplication result of the special effect background and the mask image; and determining the special effect image based on a sum of the fifth multiplication result and the sixth multiplication result.

[0138] In this embodiment, the special effect image can be calculated using the following formula: result(i,j).rgb=resFg(i,j).rgb*[1.0-mask(i,j)]+resBg(i,j)*mask(i,j).

[0139] Among them, result(i,j).rgb is the special effect image, resFg(i,j).rgb is the special effect foreground, reBg(i,j).rgb is the special effect background, mask(i,j) is the mask image, i is the width of the special effect background / special effect foreground / mask image, and j is the height of the special effect background / special effect foreground / mask image.

[0140] Similarly, the maximum pixel value of the special effect background / special effect foreground / mask image is 1.

[0141] Fig.10 FIG. 1 is a flow chart of determining a target image of a target object in one embodiment. Fig.10 As shown, the following steps are included:

[0142] S1001, tracking a target tracking frame of a target object in a previous video frame in a spherical panoramic video segment to obtain a target tracking frame of a current video frame.

[0143] In this embodiment, for the target tracking frame of the target object in the previous video frame, the target tracking frame is tracked and filled using a target tracking algorithm to obtain a first tracking rectangular frame in the current video frame to prevent the first tracking frame from deviating from the target object when tracking the target tracking frame of the target object in the previous video frame. The position of the first tracking rectangular frame is updated on the current video frame according to the first tracking rectangular frame and the detection rectangular frames of each object in the current video frame to determine the target tracking frame of the current video frame.

[0144] In traditional technology, an image recognition algorithm is used to determine the detection rectangular frame of the detection object in each video frame for each video frame in a video clip. If there are multiple detection objects in the video frame, it is impossible to determine the target tracking frame from the multiple detection rectangular frames, or the target object of the target tracking frame in the first video frame is different from the target tracking frame of the video frame after the first video frame, resulting in the inability to specify the target object corresponding to the sticker, or an error in the target object corresponding to the sticker.

[0145] S1002, performing segmentation processing on the target object in the current video frame according to the target tracking frame of the current video frame to obtain a target image of the target object.

[0146] Among them, image segmentation is to divide the image into several non-overlapping areas according to grayscale, color, spatial texture, geometric shape and other features, so that these features show consistency or similarity in the same area, but show obvious differences between different areas. In this scheme, it specifically refers to distinguishing the target object and background area of ​​the current video frame.

[0147] In the embodiment of the present application, the target tracking frame of the target object in the previous video frame in the spherical panoramic video clip is tracked to obtain the target tracking frame of the current video frame, and the target object in the current video frame is segmented according to the target tracking frame of the current video frame to obtain the target image of the target object. In the embodiment of the present application, the target tracking frame of the current video frame is determined by tracking, which makes up for the situation in which the target object corresponding to the sticker cannot be specified in the traditional technology, or the target object corresponding to the sticker has an error.

[0148] In one embodiment, the video clip display method also includes: storing the adjusted position of the mask image and the target object on the current video frame; generating a special effect image according to the stored mask image, and playing back the preview image corresponding to the current video frame based on the special effect image and the stored adjusted position.

[0149] In this embodiment, during the process of tracking the target tracking frame, the target tracking frame of each video frame can be obtained, and the target image can be obtained according to the target tracking frame, thereby determining the mask image according to the target image, and storing the mask image during the tracking process. In addition, during the tracking process, the real-time position relationship of the target object on the video frame is obtained according to the tracked target tracking frame, thereby obtaining the adjusted position of the target object on the current video frame, and the adjusted position of the target object on the current video frame is also stored.

[0150] When playing back a planar video clip, a special effect image can be generated according to the stored mask image, and based on the special effect image and the stored adjusted position, the preview image corresponding to the current video frame is played back, thereby realizing the playback of the planar video clip corresponding to the spherical panoramic video clip.

[0151] In the embodiment of the present application, the adjusted positions of the mask image and the target object on the current video frame are stored; a special effect image is generated based on the stored mask image, and based on the special effect image and the stored adjusted position, a preview image corresponding to the current video frame is played back. Since the adjusted position of the target object of the mask image on the current video frame is stored during the tracking process, when the video clip is played back again, it is not necessary to track the target tracking frame again and perform segmentation processing based on the target tracking frame to obtain the mask image, which simplifies the operation steps of displaying the video clip.

[0152] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0153] Based on the same inventive concept, the embodiment of the present application also provides a video clip display device for implementing the video clip display method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations of one or more image display device embodiments provided below can refer to the limitations of the video clip display method above, and will not be repeated here.

[0154] In one embodiment, Fig.11As shown, a video clip display device is provided, comprising: an acquisition module 11, a first determination module 12, a second determination module 13 and a display module 14, wherein:

[0155] An acquisition module 11 is configured to acquire a target tracking frame of a target object on a current video frame in a spherical panoramic video clip, and acquire a reference bounding box of the current video frame;

[0156] A first determination module 12 is used to determine the real-time position relationship of the target object on the current video frame according to the target tracking frame and the reference bounding box;

[0157] A second determination module 13, used to adjust the position of the target object on the current video frame according to the real-time position relationship, and determine the special effect image of the target object according to the identifier of the current video frame;

[0158] The display module 14 is used to display the preview image corresponding to the current video frame based on the adjusted position and special effect image, so as to generate a planar video segment corresponding to the spherical panoramic video segment according to each preview image.

[0159] In one embodiment, the second determining module includes:

[0160] An acquisition unit, which acquires first coordinates of each first pixel point of the target tracking frame and second coordinates of each second pixel point of the reference boundary frame;

[0161] The adjustment unit is used to adjust the position of the target object on the current video frame for each first coordinate if the distance between a preset number of first coordinates and each second coordinate is less than a preset threshold.

[0162] In one embodiment, the second determining module further includes:

[0163] A first determining unit, configured to determine a sticker foreground and a sticker background corresponding to the current video frame according to an identifier of the current video frame;

[0164] The generating unit is used to generate a special effect image according to a target image of a target object, a sticker foreground and a sticker background.

[0165] In one embodiment, the first determination unit is also used to obtain the target stickers corresponding to the current video frame based on the identifier of the current video frame and the ratio of the first frame number to the second frame number; the first frame number is the number of video frames in the video clip, and the second frame number is the number of stickers in the sticker sequence; determine the sticker foreground and sticker background corresponding to the current video frame based on the target stickers.

[0166] In one embodiment, the generation unit is also used to determine a mask image based on the target image; determine a special effect foreground based on the target image and a sticker foreground; determine a special effect background based on the target image and a sticker background; and generate a special effect image based on the mask image, the special effect foreground and the special effect background.

[0167] In one embodiment, the generation unit is also used to determine a first difference between a preset value and a first transparency parameter value of the sticker foreground; determine a first product result of the color pixel value of the target image and the first difference; determine a second product result of the color pixel value of the sticker foreground and the first transparency parameter value; and determine the special effect foreground based on the sum of the first product result and the second product result.

[0168] In one embodiment, the generation unit is also used to determine a second difference between a preset value and a second transparency parameter value of the sticker background; determine a third product result of the color pixel value of the target image and the second difference; determine a fourth product result of the color pixel value of the sticker background and the second transparency parameter value; and determine the special effect background based on the sum of the third product result and the fourth product result.

[0169] In one embodiment, the generation unit is also used to determine a third difference between a preset value and a mask image; determine a fifth multiplication result of the special effect foreground and the third difference; determine a sixth multiplication result of the special effect background and the mask image; and determine the special effect image based on the sum of the fifth multiplication result and the sixth multiplication result.

[0170] In one embodiment, the second determining module further includes:

[0171] A second determining unit is used to track a target tracking frame of a target object in a previous video frame in the spherical panoramic video segment to obtain a target tracking frame of a current video frame;

[0172] The third determining unit is used to segment the target object in the current video frame according to the target tracking frame of the current video frame to obtain a target image of the target object.

[0173] In one embodiment, the video clip display device further comprises:

[0174] A storage module, used for storing the adjusted positions of the mask image and the target object on each video frame;

[0175] The playback module is used to generate a special effect image according to the stored mask image, and based on the special effect image and the stored adjusted position, play back the preview image corresponding to each video frame to generate a planar video segment corresponding to the spherical panoramic video segment according to each preview image.

[0176] Each module in the above video clip display device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0177] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.12 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a video clip display method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0178] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0179] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0180] Obtaining a target tracking frame of a target object on a current video frame in a spherical panoramic video clip, and obtaining a reference bounding box of the current video frame;

[0181] Determine the real-time position relationship of the target object on the current video frame based on the target tracking frame and the reference bounding box;

[0182] Adjusting the position of the target object on the current video frame according to the real-time position relationship, and determining the special effect image of the target object according to the identification of the current video frame;

[0183] The preview image corresponding to the current video frame is displayed based on the adjusted position and special effect image, so as to generate a planar video segment corresponding to the spherical panoramic video segment according to each preview image.

[0184] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0185] Obtaining first coordinates of each first pixel point of the target tracking box and second coordinates of each second pixel point of the reference bounding box;

[0186] For each first coordinate, if the distance between a preset number of first coordinates and each second coordinate is less than a preset threshold, the position of the target object on the current video frame is adjusted.

[0187] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0188] According to the identifier of the current video frame, determine the sticker foreground and sticker background corresponding to the current video frame;

[0189] A special effect image is generated according to a target image of a target object, a sticker foreground, and a sticker background.

[0190] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0191] Obtain the target sticker corresponding to the current video frame according to the identifier of the current video frame and the ratio of the number of first frames to the number of second frames; the number of first frames is the number of video frames in the video clip, and the number of second frames is the number of stickers in the sticker sequence;

[0192] Determine the sticker foreground and sticker background corresponding to the current video frame according to the target sticker.

[0193] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0194] Determine a mask image according to the target image;

[0195] Determine the special effect foreground according to the target image and the sticker foreground;

[0196] Determine the special effect background according to the target image and sticker background;

[0197] A special effect image is generated according to the mask image, the special effect foreground and the special effect background.

[0198] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0199] Determine a first difference between a preset value and a first transparency parameter value of the sticker foreground;

[0200] Determine a first product result of a color pixel value of a target image and a first difference value;

[0201] Determine a second product result of the color pixel value of the sticker foreground and the first transparency parameter value;

[0202] The special effect prospect is determined according to the sum of the first product result and the second product result.

[0203] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0204] Determine a second difference between the preset value and a second transparency parameter value of the sticker background;

[0205] Determine a third product result of the color pixel value of the target image and the second difference value;

[0206] Determine a fourth product result of the color pixel value of the sticker background and the second transparency parameter value;

[0207] The special effect background is determined according to the sum of the third product result and the fourth product result.

[0208] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0209] determining a third difference between the preset value and the mask image;

[0210] Determine a fifth product result of the special effect prospect and the third difference;

[0211] Determine the sixth product result of the special effect background and the mask image;

[0212] The special effect image is determined according to the sum of the fifth product result and the sixth product result.

[0213] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0214] Tracking the target tracking frame of the target object in the previous video frame in the video clip to obtain the target tracking frame of the current video frame;

[0215] According to the target tracking frame of the current video frame, the target object in the current video frame is segmented to obtain a target image of the target object.

[0216] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0217] storing the adjusted positions of the mask image and the target object on each video frame;

[0218] A special effect image is generated according to the stored mask image, and based on the special effect image and the stored adjusted position, the preview image corresponding to each video frame is played back to generate a planar video segment corresponding to the spherical panoramic video segment according to each preview image.

[0219] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0220] Obtaining a target tracking frame of a target object on a current video frame in a spherical panoramic video clip, and obtaining a reference bounding box of the current video frame;

[0221] Determine the real-time position relationship of the target object on the current video frame based on the target tracking frame and the reference bounding box;

[0222] Adjusting the position of the target object on the current video frame according to the real-time position relationship, and determining the special effect image of the target object according to the identification of the current video frame;

[0223] The preview image corresponding to the current video frame is displayed based on the adjusted position and special effect image, so as to generate a planar video segment corresponding to the spherical panoramic video segment according to each preview image.

[0224] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0225] Obtaining first coordinates of each first pixel point of the target tracking box and second coordinates of each second pixel point of the reference bounding box;

[0226] For each first coordinate, if the distance between a preset number of first coordinates and each second coordinate is less than a preset threshold, the position of the target object on the current video frame is adjusted.

[0227] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0228] According to the identifier of the current video frame, determine the sticker foreground and sticker background corresponding to the current video frame;

[0229] A special effect image is generated according to a target image of a target object, a sticker foreground, and a sticker background.

[0230] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0231] Obtain the target sticker corresponding to the current video frame according to the identifier of the current video frame and the ratio of the number of first frames to the number of second frames; the number of first frames is the number of video frames in the video clip, and the number of second frames is the number of stickers in the sticker sequence;

[0232] Determine the sticker foreground and sticker background corresponding to the current video frame according to the target sticker.

[0233] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0234] Determine a mask image according to the target image;

[0235] Determine the special effect foreground according to the target image and the sticker foreground;

[0236] Determine the special effect background according to the target image and sticker background;

[0237] A special effect image is generated according to the mask image, the special effect foreground and the special effect background.

[0238] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0239] Determine a first difference between a preset value and a first transparency parameter value of the sticker foreground;

[0240] Determine a first product result of a color pixel value of a target image and a first difference value;

[0241] Determine a second product result of the color pixel value of the sticker foreground and the first transparency parameter value;

[0242] The special effect prospect is determined according to the sum of the first product result and the second product result.

[0243] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0244] Determine a second difference between the preset value and a second transparency parameter value of the sticker background;

[0245] Determine a third product result of the color pixel value of the target image and the second difference value;

[0246] Determine a fourth product result of the color pixel value of the sticker background and the second transparency parameter value;

[0247] The special effect background is determined according to the sum of the third product result and the fourth product result.

[0248] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0249] determining a third difference between the preset value and the mask image;

[0250] Determine a fifth product result of the special effect prospect and the third difference;

[0251] Determine the sixth product result of the special effect background and the mask image;

[0252] The special effect image is determined according to the sum of the fifth product result and the sixth product result.

[0253] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0254] Tracking the target tracking frame of the target object in the previous video frame in the video clip to obtain the target tracking frame of the current video frame;

[0255] According to the target tracking frame of the current video frame, the target object in the current video frame is segmented to obtain a target image of the target object.

[0256] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0257] storing the adjusted positions of the mask image and the target object on each video frame;

[0258] A special effect image is generated according to the stored mask image, and based on the special effect image and the stored adjusted position, the preview image corresponding to each video frame is played back to generate a planar video segment corresponding to the spherical panoramic video segment according to each preview image.

[0259] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0260] Obtain a target tracking box of a target object on a current video frame of a video clip, and obtain a reference bounding box of the video frame;

[0261] Determine the real-time position relationship of the target object on the video frame based on the target tracking frame and the reference bounding box;

[0262] Adjust the position of the target object on the video frame according to the real-time position relationship, and determine the special effect image of the target object according to the identification of the current video frame;

[0263] A preview image corresponding to the video frame is displayed based on the adjusted position and special effect image.

[0264] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0265] If the real-time position relationship meets the preset conditions, the position of the target object on the video frame is adjusted.

[0266] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0267] Obtaining first coordinates of each first pixel point of the target tracking box and second coordinates of each second pixel point of the reference bounding box;

[0268] For each first coordinate, if the distance between a preset number of first coordinates and each second coordinate is less than a preset threshold, it is determined that the real-time position relationship meets the preset condition.

[0269] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0270] According to the identifier of the current video frame, determine the sticker foreground and sticker background corresponding to the current video frame;

[0271] A special effect image is generated according to a target image of a target object, a sticker foreground, and a sticker background.

[0272] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0273] Obtain the target sticker corresponding to the current video frame according to the identifier of the current video frame and the ratio of the number of first frames to the number of second frames; the number of first frames is the number of video frames in the video clip, and the number of second frames is the number of stickers in the sticker sequence;

[0274] Determine the sticker foreground and sticker background corresponding to the current video frame according to the target sticker.

[0275] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0276] Determine a mask image according to the target image;

[0277] Determine the special effect foreground according to the target image and the sticker foreground;

[0278] Determine the special effect background according to the target image and sticker background;

[0279] A special effect image is generated according to the mask image, the special effect foreground and the special effect background.

[0280] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0281] Determine a first difference between a preset value and a first transparency parameter value of the sticker foreground;

[0282] Determine a first product result of a color pixel value of a target image and a first difference value;

[0283] Determine a second product result of the color pixel value of the sticker foreground and the first transparency parameter value;

[0284] The special effect prospect is determined according to the sum of the first product result and the second product result.

[0285] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0286] Determine a second difference between the preset value and a second transparency parameter value of the sticker background;

[0287] Determine a third product result of the color pixel value of the target image and the second difference value;

[0288] Determine a fourth product result of the color pixel value of the sticker background and the second transparency parameter value;

[0289] The special effect background is determined according to the sum of the third product result and the fourth product result.

[0290] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0291] determining a third difference between the preset value and the mask image;

[0292] Determine a fifth product result of the special effect prospect and the third difference;

[0293] Determine the sixth product result of the special effect background and the mask image;

[0294] The special effect image is determined according to the sum of the fifth product result and the sixth product result.

[0295] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0296] Tracking the target tracking frame of the target object in the previous video frame in the video clip to obtain the target tracking frame of the current video frame;

[0297] According to the target tracking frame of the current video frame, the target object in the current video frame is segmented to obtain a target image of the target object.

[0298] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0299] storing the adjusted positions of the mask image and the target object on each video frame;

[0300] A special effect image is generated according to the stored mask image, and based on the special effect image and the stored adjusted position, the preview image corresponding to each video frame is played back to generate a planar video segment corresponding to the spherical panoramic video segment according to each preview image.

[0301] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0302] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0303] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0304] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A video clip display method, characterized in that: The method comprises: Obtaining a target tracking frame of a target object on a current video frame in a spherical panoramic video clip, and obtaining a reference bounding box of the current video frame; Determining a real-time position relationship of the target object on the current video frame according to the target tracking frame and the reference bounding box; adjusting the position of the target object on the current video frame according to the real-time position relationship, and determining a special effect image of the target object according to an identifier of the current video frame; The preview image corresponding to the current video frame is displayed based on the adjusted position and the special effect image, so as to generate a planar video segment corresponding to the spherical panoramic video segment according to each of the preview images.

2. The method according to claim 1, characterized in that The step of adjusting the position of the target object on the current video frame according to the real-time position relationship includes: Acquire first coordinates of each first pixel point of the target tracking frame and second coordinates of each second pixel point of the reference bounding box; For each of the first coordinates, if the distance between a preset number of the first coordinates and each of the second coordinates is less than a preset threshold, the position of the target object on the current video frame is adjusted.

3. The method according to claim 1, characterized in that The determining the special effect image of the target object according to the identifier of the current video frame includes: Determine, according to the identifier of the current video frame, a sticker foreground and a sticker background corresponding to the current video frame; The special effect image is generated according to the target image of the target object, the sticker foreground and the sticker background.

4. The method according to claim 3, characterized in that The step of determining a sticker foreground and a sticker background corresponding to the current video frame according to the identifier of the current video frame includes: Obtain a target sticker corresponding to the current video frame according to an identifier of the current video frame and a ratio of a first frame number to a second frame number; the first frame number is the number of video frames in the spherical panoramic video clip, and the second frame number is the number of stickers in the sticker sequence; A sticker foreground and a sticker background corresponding to the current video frame are determined according to the target sticker.

5. The method according to claim 3, characterized in that: The generating the special effect image according to the target image of the target object, the sticker foreground and the sticker background comprises: Determining a mask image according to the target image; Determine a special effect foreground according to the target image and the sticker foreground; Determine a special effect background according to the target image and the sticker background; The special effect image is generated according to the mask image, the special effect foreground and the special effect background.

6. The method according to claim 5, characterized in that The determining of the special effect foreground according to the target image and the sticker foreground includes: Determine a first difference between a preset value and a first transparency parameter value of the sticker foreground; Determine a first product result of a color pixel value of the target image and the first difference value; Determine a second product result of the color pixel value of the sticker foreground and the first transparency parameter value; The special effect foreground is determined according to a summation result of the first multiplication result and the second multiplication result.

7. The method according to claim 5, characterized in that The determining of the special effect background according to the target image and the sticker background comprises: Determine a second difference between the preset value and a second transparency parameter value of the sticker background; Determine a third product result of the color pixel value of the target image and the second difference value; Determine a fourth product result of the color pixel value of the sticker background and the second transparency parameter value; The special effect background is determined according to a sum result of the third multiplication result and the fourth multiplication result.

8. The method according to claim 5, characterized in that The generating the special effect image according to the mask image, the special effect foreground and the special effect background comprises: Determining a third difference between the preset value and the mask image; Determine a fifth product result of the special effect prospect and the third difference; Determine a sixth product result of the special effect background and the mask image; The special effect image is determined according to a sum result of the fifth multiplication result and the sixth multiplication result.

9. The method according to any one of claims 3 to 8, characterized in that: The method further comprises: Tracking a target tracking frame of the target object in a previous video frame in the spherical panoramic video clip to obtain a target tracking frame of the current video frame; According to the target tracking frame of the current video frame, a target object in the current video frame is segmented to obtain a target image of the target object.

10. The method according to claim 9, characterized in that The method further comprises: storing the mask image and the adjusted positions of the target object on each of the video frames; The special effect image is generated according to the stored mask image, and based on the special effect image and the stored adjusted position, the preview image corresponding to each of the video frames is played back to generate a planar video segment corresponding to the spherical panoramic video segment according to each of the preview images.

11. A video clip display device, characterized in that: The device comprises: An acquisition module, used to acquire a target tracking frame of a target object on a current video frame in a spherical panoramic video clip, and to acquire a reference bounding box of the current video frame; A first determination module, configured to determine a real-time position relationship of the target object on the current video frame according to the target tracking frame and the reference bounding frame; A second determination module, configured to adjust the position of the target object on the current video frame according to the real-time position relationship, and determine a special effect image of the target object according to an identifier of the current video frame; The display module is used to display the preview image corresponding to the current video frame based on the adjusted position and the special effect image, so as to generate a planar video segment corresponding to the spherical panoramic video segment according to each of the preview images.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images