Video processing method and apparatus, device, computer-readable storage medium, and product

Abstract

Embodiments of the present application provide a video processing method and apparatus, a device, a computer-readable storage medium, and a product. The method comprises: acquiring a video to be rendered, and determining target areas in a plurality of video frames corresponding to the video to be rendered; acquiring target three-dimensional data associated with the target areas; on the basis of set scanning information, determining scanning areas to be displayed in the target areas; and fusing the target three-dimensional data with the video frames on the basis of the scanning areas to obtain a target video. By means of special effect rendering based on target three-dimensional data, the special effect rendering effect can be made more stereoscopic. In addition, a display effect of scanning target areas by means of a light-emitting scanning object can be achieved, and the rendering effect of special effect rendering is improved.

Description

Video processing method, device, equipment, computer-readable storage medium and product

[0001] This application claims priority to Chinese patent application No. 202311199788.9 filed on September 15, 2023, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0002] The embodiments of the present disclosure relate to a video processing method, apparatus, device, computer-readable storage medium, and product. Background Art

[0003] With the continuous development of special effects technology, the variety of special effects has gradually increased. Users can now capture media content based on special effects. For example, users can select special effects that apply to specific areas for media content capture. This allows users to render virtual content associated with the special effects in the target area of ​​the media content, thereby improving the quality of the captured media content.

[0004] Current special effects are generally rendered based on collected two-dimensional images, but the results of special effects rendering are often relatively flat and lack three-dimensionality, which cannot meet the personalized needs of users.

[0005] Summary of the Invention

[0006] The embodiments of the present disclosure provide a video processing method, apparatus, device, computer-readable storage medium, and product, which are configured to solve the technical problem that existing special effects rendering effects lack stereoscopic effect.

[0007] The present disclosure provides a video processing method, including:

[0008] Obtaining a video to be rendered, and determining a target area in a plurality of video frames corresponding to the video to be rendered;

[0009] Acquiring target three-dimensional data associated with the target area;

[0010] Based on the set scanning information, determining a scanning area to be displayed in the target area;

[0011] Based on the scanning area, the target three-dimensional data is fused with the video frame to obtain a target video.

[0012] The present disclosure provides a video processing device, including:

[0013] A determination module is configured to obtain a video to be rendered and determine a target area in a plurality of video frames corresponding to the video to be rendered;

[0014] an acquisition module, configured to acquire target three-dimensional data associated with the target area;

[0015] a processing module configured to determine a scan area to be displayed in the target area based on the set scan information;

[0016] The fusion module is configured to fuse the target three-dimensional data with the video frame based on the scanning area to obtain the target video.

[0017] An embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0018] The memory stores computer-executable instructions;

[0019] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the video processing method as described above and in various possible designs.

[0020] An embodiment of the present disclosure provides a computer-readable storage medium having computer-executable instructions stored therein. When a processor executes the computer-executable instructions, the video processing method described above and in various possible designs is implemented.

[0021] An embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the video processing method as described above and in various possible designs. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] FIG1 is a flow chart of a video processing method according to an embodiment of the present disclosure;

[0024] FIG2 is a schematic diagram of optimization of three-dimensional data to be processed according to an embodiment of the present disclosure;

[0025] FIG3 is a flow chart of a video processing method provided by another embodiment of the present disclosure;

[0026] FIG4 is a schematic diagram of a scan object to be processed according to an embodiment of the present disclosure;

[0027] FIG5 is a schematic diagram of a scanning area provided by an embodiment of the present disclosure;

[0028] FIG6 is a flow chart of a video processing method provided by yet another embodiment of the present disclosure;

[0029] FIG7 is a flow chart of a video processing method provided by yet another embodiment of the present disclosure;

[0030] FIG8 is a schematic structural diagram of a video processing device provided by an embodiment of the present disclosure;

[0031] FIG9 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0033] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0034] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0035] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0036] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0037] Glossary:

[0038] A point cloud is a collection of points formed by obtaining the spatial coordinates of each sampled point on an object's surface. Point clouds used for 3D object detection are typically obtained from LiDAR scans and contain information such as the point's 3D coordinates and intensity.

[0039] To solve the technical problem that existing special effects rendering effects lack stereoscopic effect, the present disclosure provides a video processing method, device, equipment, computer-readable storage medium and product.

[0040] It should be noted that the video processing method, apparatus, device, computer-readable storage medium, and product provided by the present disclosure can be applied to any processing scenario of special effects rendering acting on a target area.

[0041] In related technologies, when capturing media content using special effects applied to a target area, rendering is typically performed based on the acquired two-dimensional image to produce a rendered result. However, rendering based on two-dimensional images often results in a flat, lacking three-dimensional rendering and a relatively monotonous display effect.

[0042] In the process of solving the above technical problems, the inventors discovered through research that, in order to make the special effects rendering results more three-dimensional and diverse, a preset 3D reconstruction algorithm can be used to process the data of the collected video frames in the video to be rendered to obtain target 3D data. Special effects rendering operations can be performed based on the target 3D data, thereby making the rendering results more three-dimensional.

[0043] Furthermore, the scanned area to be displayed within the target area can be determined based on the set scanning information. Based on the scanned area, the target 3D data is fused with the video frame to obtain the target video. This enables the display effect of scanning the target area using a luminous scanning object.

[0044] The execution subject of the embodiment of the present disclosure is a video processing device, which can be written in languages ​​such as C / C++, Java, Shell or Python.

[0045] Optionally, the video processing device may be coupled to a server. The server is communicatively connected to a terminal device, thereby acquiring a video to be rendered captured by the terminal device, determining the three-dimensional data to be processed and the target mask associated with the target area in multiple video frames corresponding to the video to be rendered, and constructing a luminous target scanning object based on the three-dimensional data to be processed and the target mask, thereby achieving a display effect in which the target area is scanned by the luminous scanning object.

[0046] Alternatively, the video processing device can be coupled to a terminal device, thereby determining, based on the acquired video to be rendered, the to-be-processed three-dimensional data and target mask associated with the target area in multiple video frames corresponding to the to-be-rendered video. Based on the to-be-processed three-dimensional data and target mask, a luminous target scanning object is constructed to achieve a display effect in which the luminous scanning object is used to scan the target area.

[0047] FIG1 is a flow chart of a video processing method provided by an embodiment of the present disclosure. As shown in FIG1 , the method includes:

[0048] Step 101: Obtain a video to be rendered, and determine a target area in multiple video frames corresponding to the video to be rendered.

[0049] In this embodiment, a video to be rendered may be obtained, wherein the video to be rendered includes multiple video frames, and the video frames include a target area. The target area may be a pre-set area or an area determined by the user according to actual needs, and this disclosure does not impose any restrictions on this.

[0050] Taking a practical application as an example, the target area may be a face area.

[0051] Optionally, the video to be rendered may be obtained from a terminal device. The video to be rendered may be captured in real time by a user through a video capture device preset in the terminal device, or the video to be rendered may be uploaded by the user on the terminal device according to actual needs, and this disclosure does not limit this.

[0052] Alternatively, the video processing device may be communicatively connected to a data acquisition device and / or a data server, so that the video to be rendered can be acquired based on the data acquisition device and / or the video to be rendered can be acquired from the data server.

[0053] In order to implement special effects rendering operations on the video to be rendered, a target area in each video frame corresponding to the video to be rendered can be determined. The target area can be determined based on a preset recognition algorithm or specified by the user according to actual needs, and this disclosure does not limit this.

[0054] Step 102: Acquire target three-dimensional data associated with the target area.

[0055] In this embodiment, after determining the target area in the video frame, the corresponding 3D data of the target area can be determined. Unlike existing special effects rendering solutions based on 2D images, performing special effects rendering based on the target 3D data of the target area can achieve a more three-dimensional and realistic display effect.

[0056] As an implementable method, the target three-dimensional data can be a three-dimensional point cloud, which is usually obtained by laser radar scanning and contains the three-dimensional coordinates, intensity and other information of the points. Therefore, special effects rendering based on the three-dimensional point cloud can more accurately express the three-dimensional spatial distribution of the target area and improve the three-dimensionality of the special effects rendering results.

[0057] Optionally, any three-dimensional reconstruction algorithm may be used to realize the recognition of the to-be-processed three-dimensional data of the target area, and the present disclosure does not impose any limitation on this.

[0058] Step 103: Determine a scan area to be displayed in the target area based on the set scan information.

[0059] In this embodiment, the scan area to be displayed in the target area may be determined, and during the rendering process, only the target three-dimensional data corresponding to the scan area may be rendered to present a display effect of scanning the target area.

[0060] Optionally, scanning information can be pre-set. Based on the pre-set scanning information, the scan area to be displayed in the target area can be determined. The scanning information includes, but is not limited to, a scan area to be processed with preset display parameters and its corresponding movement parameters. Display parameters include, but are not limited to, parameters such as display shape and display size. Movement parameters include, but are not limited to, one or more of movement direction, movement speed, and movement acceleration.

[0061] Step 104: Based on the scan area, the target three-dimensional data is fused with the video frame to obtain a target video.

[0062] In this embodiment, after determining the scanning area, the 3D data to be rendered and displayed in the target 3D data can be determined based on the scanning area. Then, the target 3D data can be fused with the video frame based on the scanning area to obtain the target video.

[0063] Optionally, if the video frame is L, the 3D data to be rendered P is determined based on the scan area and the target 3D data, and the final output image is O, the fusion processing can be implemented using the preset formula 1:

[0064] O=clamp(L+P,0,255) (1)

[0065] Here, 0 and 255 are the minimum and maximum values ​​allowed, respectively.

[0066] The video processing method provided in this embodiment, after acquiring the video to be rendered, determines, for each video frame in the video to be rendered, target 3D data corresponding to the target area in the video frame. This target 3D data may be 3D point cloud data. To achieve a rendering effect involving a scanning operation on the target area, the scanned area to be displayed within the target area is determined based on the preset scanning information. Based on the scanned area, the target 3D data is fused with the video frame to obtain the target video. By rendering special effects based on the target 3D data, the special effects rendering effect can be made more three-dimensional.

[0067] Further, based on any of the above embodiments, step 102 includes:

[0068] Acquire three-dimensional data to be processed corresponding to a target area in the video frame, wherein the three-dimensional data to be processed includes a plurality of triangular areas formed by connecting data points.

[0069] A merging operation is performed on two adjacent triangular areas in the three-dimensional data to be processed to form a rectangular area.

[0070] The connecting lines within the rectangular area are removed to obtain the target three-dimensional data.

[0071] In this embodiment, after obtaining the video to be rendered, the video frames in the video to be rendered can be processed based on a preset 3D reconstruction algorithm to obtain 3D data to be processed, wherein the 3D data to be processed can be 3D point cloud data to be processed.

[0072] Furthermore, after obtaining the 3D data to be processed, directly performing special effects rendering based on the 3D data to be processed may result in an unsightly rendering result. Therefore, in order to improve the rendering effect of special effects rendering, the 3D data to be processed can also be beautified based on a preset data optimization method to obtain the target 3D data.

[0073] Optionally, the three-dimensional data to be processed is composed of multiple data points. The multiple data points are connected by preset lines to form multiple triangular areas. Since there are many triangular areas in the three-dimensional data to be processed, the visual effect is poor.

[0074] Therefore, to optimize the processing of the 3D data, two adjacent triangular regions in the 3D data can be merged to form a rectangular region. After merging the two adjacent triangular regions, the rectangular region also includes the connecting lines corresponding to the triangular regions. To simplify the lines in the 3D data, the connecting lines within the rectangular region can be removed to obtain the target 3D data.

[0075] FIG2 is a schematic diagram illustrating optimization of unprocessed 3D data according to an embodiment of the present disclosure. As shown in FIG2 , unprocessed 3D data 21 includes multiple triangular regions 22 formed by connecting data points. Two adjacent triangular regions 22 in unprocessed 3D data 21 can be merged to form a rectangular region 23. The connecting lines within rectangular regions 23 are removed to obtain target 3D data 24.

[0076] The video processing method provided in this embodiment determines the 3D data to be processed corresponding to a target area in a video frame, then merges two adjacent triangular regions in the 3D data to form a rectangular region. Connecting lines within the rectangular region are removed to obtain the target 3D data, thereby enhancing the aesthetics of the 3D data and improving special effects rendering.

[0077] FIG3 is a flow chart of a video processing method according to another embodiment of the present disclosure. Based on any of the above embodiments, step 103 includes:

[0078] Step 301: Acquire a preset scanning area to be processed, and determine a target mask associated with the target area in the video frame.

[0079] Step 302: Determine the intersection between the scan area to be processed and the target mask, and determine the intersection as the scan area to be displayed.

[0080] In this embodiment, in order to implement the scanning operation on the target area, a scanning area to be processed can be pre-set, wherein the scanning area to be processed can be a rectangular area of ​​a preset size or other polygonal area.

[0081] After obtaining the video to be rendered, a target mask corresponding to a video frame in the video to be rendered can be determined based on a preset 3D reconstruction algorithm. The target mask matches the shape of the target area.

[0082] Furthermore, since the target area is generally an irregular area, for example, when the target area is a human face area, the human face area often has ups and downs, directly determining the scan area to be processed as the scan area may result in a mismatch between the scan area and the target area.

[0083] To make the scan area more closely match the shape of the target area, after determining the target mask corresponding to the target area, the scan area to be processed can be constrained based on the target mask. Since the target mask matches the shape of the target area, the intersection between the scan area to be processed and the target mask can be determined, and the intersection is determined as the scan area.

[0084] FIG4 is a schematic diagram of a scan object to be processed provided by an embodiment of the present disclosure. As shown in FIG4 , a preset scan area 41 to be processed can be obtained, and the intersection between the scan area 41 to be processed and the target mask 42 is taken, and the intersection is determined as a scan area 43.

[0085] The video processing method provided in this embodiment pre-sets a scan area to be processed and determines the intersection of the scan area to be processed and the target mask as the scan area. Because the target mask matches the shape of the target area, determining the intersection of the scan area to be processed and the target mask as the scan area allows the scan area to more closely match the target area, further improving the display quality of special effects rendering.

[0086] Further, based on any of the above embodiments, step 301 includes:

[0087] A preset first sub-region and a preset second sub-region are acquired, wherein the first sub-region and the second sub-region have different display sizes.

[0088] A merging operation is performed on the first sub-region and the second sub-region to obtain the scanning region.

[0089] In this embodiment, the scan area to be processed can be obtained by merging two sub-areas with different display sizes.

[0090] Optionally, a preset first sub-region and a second sub-region may be obtained, wherein the first sub-region and the second sub-region have different display sizes, and the first sub-region and the second sub-region are merged to obtain a scan region to be processed.

[0091] For example, the first sub-region may be a sub-region having the same display length as the second sub-region but a different display width.

[0092] Further, based on any of the above embodiments, step 104 includes:

[0093] Determining first three-dimensional data matching the first sub-region in the target three-dimensional data;

[0094] determining a second three-dimensional region in the target three-dimensional data that matches the second sub-region;

[0095] The first three-dimensional data and the second three-dimensional area are rendered onto the video frame using different rendering parameters to obtain the target video.

[0096] In this embodiment, in order to make the scan area present a richer display effect, different rendering parameters may be used to perform light effect rendering operations on the first sub-area and the second sub-area respectively.

[0097] To achieve precise rendering, first 3D data matching the first subregion in the target 3D data can be determined. A second 3D region matching the second subregion in the target 3D data can be determined. The first 3D data and the second 3D region can be rendered onto a video frame using different rendering parameters to obtain a target video.

[0098] For example, the display width of the second sub-area can be smaller than that of the first sub-area. The second sub-area can be superimposed on the center of the upper layer of the first sub-area to obtain the scan area to be processed. Optionally, the brightness and saturation of the second sub-area can be controlled to be higher than those of the first sub-area, thereby presenting a scan area with a higher central brightness than the surrounding brightness.

[0099] FIG5 is a schematic diagram of a scanning area provided by an embodiment of the present disclosure. As shown in FIG5 , a scanning area 51 to be processed may be composed of a first sub-area 52 and a second sub-area 53, wherein the display size of the first sub-area 52 is smaller than the display size of the second sub-area 53. By merging the first sub-area 52 and the second sub-area 53, the first 3D data corresponding to the first sub-area and the second 3D area corresponding to the second sub-area can be rendered onto a video frame using different rendering parameters, respectively, to obtain a target video.

[0100] The video processing method provided in this embodiment combines the first sub-area and the second sub-area to form a scanning area to be processed, so that different rendering parameters can be used to render the first three-dimensional data corresponding to the first sub-area and the second three-dimensional area corresponding to the second sub-area onto the video frame to obtain the target video, so that the scanning area presents a richer display effect.

[0101] Furthermore, in order to achieve accurate scanning operations, scanning operations can be performed within a preset range. Therefore, after obtaining the video to be rendered, the key point information associated with the target area in multiple video frames corresponding to the video to be rendered can be determined. The scanning start position is determined based on the key point information. Taking the target area as the face area as an example, the key point information may include the position information of the facial features. Among them, a preset three-dimensional reconstruction algorithm can be used to realize the recognition of key point information, and the present disclosure does not impose any restrictions on this.

[0102] Taking a practical application as an example, the target area may be a face area, and the scanning operation may be performed in the area from the forehead to the chin.

[0103] Optionally, the scanning start position can be determined based on preset filtering conditions and key point information. For example, the key point with the highest display position among the key points can be used as the starting position, and the key point with the lowest display position among the key points can be used as the ending position. Alternatively, the starting position can be determined based on a preset position interval and key point information, etc., and this disclosure does not impose any restrictions on this.

[0104] To implement the scanning operation, movement parameters can be pre-set for the scanning area, where the movement parameters include one or more of movement direction, movement speed, and movement acceleration. Therefore, after determining the starting position, the scanning area can be controlled to move between the scanning starting positions according to the preset movement parameters.

[0105] Further, based on any of the above embodiments, the scanning area moves in the target area according to preset movement parameters, wherein the movement parameters include one or more of movement direction, movement speed, and movement acceleration; step 603 includes:

[0106] Determine timestamp information corresponding to each video frame.

[0107] A scanning area to be displayed in the target area is determined based on the timestamp information and the movement parameter.

[0108] In this embodiment, the video to be rendered often consists of multiple video frames with timestamps. Therefore, in order to accurately determine the scanning area to be displayed in the target area, the timestamp information corresponding to each video frame in the video to be rendered can be first determined. In this way, the scanning area to be displayed in the target area can be determined based on the timestamp information and the preset movement parameters.

[0109] For example, the scanning area can move downward along the face area at a speed of 2 mm per second. That is, the preset movement parameters can be a downward movement direction and a movement speed of 2 mm / s. Determine the timestamp and display position of the first video frame. For each target video frame in the video to be rendered, the time interval between the target video frame and the first video frame can be determined, and the movement distance of the scanning area can be determined based on the time interval and the movement speed. The scanning area to be displayed in the target area can be determined based on the display position, movement distance and movement direction of the first video frame.

[0110] The video processing method provided in this embodiment determines the timestamp information corresponding to each video frame, thereby accurately determining the scanning area to be displayed in the target area based on the timestamp information and the movement parameters, thereby improving the accuracy of the scanning operation.

[0111] Further, based on any of the above embodiments, step 104 includes:

[0112] Determine the three-dimensional data to be rendered that matches the scan area in the target three-dimensional data.

[0113] The three-dimensional data to be rendered is rendered into the video frame to obtain a target video.

[0114] In this embodiment, after the scanning area is determined, a rendering operation may be performed on the target three-dimensional data at the position of the scanning area to achieve a visual effect of scanning the target area.

[0115] Therefore, the three-dimensional data to be rendered that matches the scan area in the target three-dimensional data can be determined, and the three-dimensional data to be rendered is rendered into the video frame to obtain the target video.

[0116] As an implementable method, a glow effect can be used to render the 3D data to be rendered. If the 3D data to be rendered is L, the output image is O, the preset blur operator is B, and the preset fusion operator is R, where B and R can be called multiple times, then the glow effect rendering operation can be implemented using Formula 2:

[0117] O=R(B(L),B(B(L))) (2)

[0118] It should be noted that any lighting effect rendering method can be used to implement lighting effect rendering of the three-dimensional data to be rendered, and this disclosure does not impose any limitation on this.

[0119] Figure 6 is a schematic diagram of scanning a target area according to an embodiment of the present disclosure. As shown in Figure 6, the target area can be a facial area 61. The scanning area can be moved according to preset movement parameters. For example, the scanning area can be scanned starting from the forehead and gradually downward. Based on the scanning area and the target 3D data, the 3D data to be rendered 62 can be determined. As the scanning area moves, the 3D data to be rendered 62 can be scanned from a position above the eyes to the position of the nose.

[0120] FIG7 is a flow chart of a video processing method provided by another embodiment of the present disclosure. Based on any of the above embodiments, as shown in FIG7 , the method further includes:

[0121] Step 701: Determine key point information associated with the target area.

[0122] Step 702: Determine target key points in the scanning area that meet preset adjustment conditions based on the key point information.

[0123] Step 703: Perform a halo rendering operation on the target key point based on preset rendering parameters.

[0124] In this embodiment, in order to further enrich the rendering effect of special effects, a halo rendering operation may be performed on a designated position within the scanning area.

[0125] Taking a practical application as an example, the target area may be a face area. When scanning the eye area, a halo rendering may be performed on the eye portion in the face area to make the eye portion appear to have a glowing effect.

[0126] Optionally, key point information associated with the target region in multiple video frames corresponding to the video to be rendered is determined. For example, if the target region is a face region, the key point information may include position information of facial features.

[0127] Furthermore, target key points within the scanning area that meet preset adjustment conditions are determined based on the key point information. For example, the preset adjustment conditions may include determining a key point at the center based on the key point information, determining a key point at the edge based on the key point information, determining a key point within a preset area based on the key point information, etc. This disclosure is not limited to this.

[0128] After determining the target key point, a halo rendering operation can be performed on the target key point based on preset rendering parameters. The preset rendering parameters can be preset or set by the user according to actual needs, and the present disclosure does not limit this.

[0129] The video processing method provided in this embodiment determines target key points that meet preset adjustment conditions based on key point information and performs halo rendering on the target key points based on preset rendering parameters, thereby enabling the target key points to present more diverse display effects and improving the effect of special effects rendering.

[0130] Figure 8 is a structural diagram of a video processing device provided by an embodiment of the present disclosure. As shown in Figure 8, the device includes: a determination module 81, an acquisition module 82, a processing module 83, and a fusion module 84. Among them, the determination module 81 is configured to acquire the video to be rendered and determine the target area in multiple video frames corresponding to the video to be rendered. The acquisition module 82 is configured to acquire target three-dimensional data associated with the target area. The processing module 83 is configured to determine the scan area to be displayed in the target area based on the set scan information. The fusion module 84 is configured to fuse the target three-dimensional data with the video frame based on the scan area to obtain the target video.

[0131] Furthermore, based on any of the above embodiments, the acquisition module is configured to: acquire unprocessed 3D data corresponding to a target area in the video frame, wherein the unprocessed 3D data includes a plurality of triangular areas formed by connecting data points; merge two adjacent triangular areas in the unprocessed 3D data to form a rectangular area; and remove connecting lines within the rectangular areas to obtain the target 3D data.

[0132] Furthermore, based on any of the above embodiments, the processing module is configured to: obtain a preset scan area to be processed, determine a target mask associated with the target area in the video frame, determine an intersection between the scan area to be processed and the target mask, and determine the intersection as the scan area to be displayed.

[0133] Furthermore, based on any of the above embodiments, the processing module is configured to: obtain a preset first sub-region and a second sub-region, wherein the first sub-region and the second sub-region have different display sizes, and merge the first sub-region and the second sub-region to obtain the scan region to be processed.

[0134] Further, based on any of the above embodiments, the fusion module is configured to: fuse the target 3D data with the video frame based on the scan area to obtain the target video, including: determining first 3D data matching the first sub-area in the target 3D data; determining a second 3D area matching the second sub-area in the target 3D data; and rendering the first 3D data and the second 3D area onto the video frame using different rendering parameters to obtain the target video.

[0135] Furthermore, based on any of the above embodiments, the scanning area moves within the target area according to preset movement parameters, wherein the movement parameters include one or more of movement direction, movement speed, and movement acceleration. The processing module is configured to determine timestamp information corresponding to the video frame, and determine a scanning area to be displayed in the target area based on the timestamp information and the movement parameters.

[0136] Furthermore, based on any of the above embodiments, the fusion module is configured to: determine the 3D data to be rendered that matches the scan area in the target 3D data, and render the 3D data to be rendered into the video frame to obtain the target video.

[0137] Furthermore, based on any of the above embodiments, the apparatus further includes: an identification module configured to determine key point information associated with the target area; a determination module configured to determine target key points within the scanning area that meet preset adjustment conditions based on the key point information; and a rendering module configured to perform a halo rendering operation on the target key points based on preset rendering parameters.

[0138] The device provided in this embodiment can be configured to execute the technical solution of the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0139] In order to implement the above embodiments, the embodiments of the present disclosure further provide a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the video processing method described in any of the above embodiments is implemented.

[0140] In order to implement the above embodiments, the embodiments of the present disclosure further provide a computer program product, including a computer program, which implements the video processing method as described in any of the above embodiments when executed by a processor.

[0141] In order to implement the above embodiment, the present disclosure further provides an electronic device, including: a processor and a memory;

[0142] The memory stores computer-executable instructions;

[0143] The processor executes the computer-executable instructions stored in the memory, so that the processor performs the video processing method as described in any of the above embodiments.

[0144] FIG9 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. As shown in FIG9 , the electronic device 900 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG9 is merely an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0145] As shown in Figure 9, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the electronic device 900 are also stored in the RAM 903. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0146] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although FIG9 shows an electronic device 900 with various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0147] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code configured to execute the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0148] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or convey a program configured for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be conveyed using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0149] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0150] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0151] Computer program code configured to perform the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0152] The flow charts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions configured to implement the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0153] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0154] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0155] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0156] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0157] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0158] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A video processing method, comprising: Obtain a video to be rendered, and determine a target area in a plurality of video frames corresponding to the video to be rendered; Acquiring target three-dimensional data associated with the target area; Based on the set scanning information, determining a scanning area to be displayed in the target area; Based on the scanning area, the target three-dimensional data is fused with the video frame to obtain a target video.

2. The method according to claim 1, wherein: The acquiring of target three-dimensional data associated with the target area includes: Acquire the to-be-processed three-dimensional data corresponding to the target area in the video frame, wherein the to-be-processed three-dimensional data includes a plurality of triangular areas formed by connecting data points; Merging two adjacent triangular regions in the three-dimensional data to be processed to form a rectangular region; The connecting lines within the rectangular area are removed to obtain the target three-dimensional data.

3. The method according to claim 1 or 2, wherein: The determining, based on the set scanning information, a scanning area to be displayed in the target area includes: Acquire a preset scanning area to be processed, and determine a target mask associated with the target area in the video frame; An intersection between the scan area to be processed and the target mask is determined, and the intersection is determined as the scan area to be displayed.

4. The method according to claim 3, wherein: The step of obtaining a preset scanning area to be processed includes: Acquire a preset first sub-region and a second sub-region, wherein the first sub-region and the second sub-region have different display sizes; A merging operation is performed on the first sub-region and the second sub-region to obtain the scan region to be processed.

5. The method according to claim 4, wherein: The step of fusing the target three-dimensional data with the video frame based on the scan area to obtain the target video includes: Determine first three-dimensional data matching the first sub-region in the target three-dimensional data; Determining a second three-dimensional region in the target three-dimensional data that matches the second sub-region; The first three-dimensional data and the second three-dimensional area are rendered onto the video frame using different rendering parameters respectively to obtain the target video.

6. The method according to claim 1 or 2, wherein: The scanning area moves in the target area according to preset movement parameters, wherein the movement parameters include one or more of a movement direction, a movement speed, and a movement acceleration; The determining, based on the set scanning information, a scanning area to be displayed in the target area includes: Determining timestamp information corresponding to the video frame; A scanning area to be displayed in the target area is determined based on the timestamp information and the movement parameter.

7. The method according to any one of claims 1 to 6, wherein: The step of fusing the target three-dimensional data with the video frame based on the scan area to obtain the target video includes: Determining three-dimensional data to be rendered that matches the scanning area in the target three-dimensional data; The three-dimensional data to be rendered is rendered into the video frame to obtain a target video.

8. The method according to any one of claims 1 to 6, wherein: The method further comprises: Determining key point information associated with the target area; Determine, according to the key point information, the target key point in the scanning area that meets the preset adjustment conditions; A halo rendering operation is performed on the target key point based on preset rendering parameters.

9. A video processing device, comprising: A determination module is configured to obtain a video to be rendered and determine a target area in a plurality of video frames corresponding to the video to be rendered; An acquisition module, configured to acquire target three-dimensional data associated with the target area; A processing module, configured to determine a scan area to be displayed in the target area based on the set scan information; The fusion module is configured to fuse the target three-dimensional data with the video frame based on the scanning area to obtain the target video.

10. An electronic device, comprising: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the video processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the video processing method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the video processing method according to any one of claims 1 to 8 is implemented.

Images