Video processing method and apparatus, computer equipment and storage medium

Abstract

This disclosure provides a video processing method and apparatus, a computer device and a storage medium, the method comprising: displaying instruction information including first instruction information for instructing a user to select a light and shadow rendering region and second instruction information for instructing a user to select lighting parameter information, in response to receiving a lighting processing request for a video to be processed; determining a target light and shadow rendering region selected by the user based on the first instruction information and target lighting parameter information selected based on the second instruction information for a target video frame in the video to be processed, wherein the target light and shadow rendering region and target lighting parameter information determined for the target video frame are used to perform lighting processing on a target video segment associated with the target video frame; and displaying a target video after performing lighting processing on the video to be processed based on the target light and shadow rendering region and target lighting parameter information.

Description

Technical Field

[0001] This application claims the priority of Chinese Patent Application No. 202310996106.0 filed on August 8, 2023, and hereby incorporates by reference in its entirety the content disclosed in the above-mentioned Chinese Patent Application as part of this application.

[0002] [Technical Field] This disclosure relates to a video processing method and apparatus, a computer device, and a storage medium.

Background Art

[0003] With the rapid development of Internet technology, when users use smart devices to display videos, there is a need for writing processing for the displayed videos. Here, intelligent writing aims to perform re-lighting on video frames in a video through three-dimensional stereo perception of the video, increase the beauty of the light and shadow in the video scene, emphasize the light and dark relationship of the video screen, and create an atmosphere. Therefore, in order to meet the needs of users, it is particularly important to propose a method for performing writing processing on videos.

Summary of the Invention

[0004] Embodiments of this disclosure provide at least a video processing method and apparatus, a computer device, and a storage medium.

[0005] According to a first aspect, embodiments of this disclosure provide a video processing method, the method comprising: displaying instruction information including first instruction information for instructing a user to select a light and shadow rendering area and second instruction information for instructing a user to select writing parameter information in response to receiving a writing processing request for a video to be processed; For the target video frame in the video to be processed, the target light and shadow rendering region selected by the user based on the first instruction information and the target lighting parameter information selected based on the second instruction information are determined, respectively, and the target light and shadow rendering region and target lighting parameter information determined for the target video frame are used to perform lighting processing on the target video segment associated with the target video frame. This includes displaying the target video after performing lighting processing on the video to be processed based on the target light and shadow rendering area and the target lighting parameter information.

[0006] In one optional embodiment, displaying the target video after performing lighting processing on the video to be processed based on the target light and shadow rendering region and the target lighting parameter information is: When displaying a target video segment in the target video that is related to the target video frame, in response to the selected target light and shadow rendering region for the target video frame containing the target region, the effect after lighting has been applied to the target region content in the video frame of the target video segment that matches the target region, according to the target lighting parameter information, is displayed. Here, the target region includes at least one of a background region, a foreground region, a target area region within the foreground region, and a global region. If the target region includes a background region, the target region content that matches the background region includes the background content of the background region and a portion of the foreground content adjacent to the background content.

[0007] In one optional embodiment, displaying the target video after performing lighting processing on the video to be processed based on the target light and shadow rendering region and the target lighting parameter information is: In response to the target lighting parameter information including light source movement trajectory information, when displaying a target video segment related to the target video frame in the target video, the system includes displaying a dynamic lighting effect in the target light and shadow rendering region that matches the light source movement trajectory information.

[0008] In one optional embodiment, lighting processing is performed on the video to be processed based on the target light and shadow rendering region and the target lighting parameter information to obtain the target video. For the video frames to be processed within the target video segment included in the video to be processed, detection is performed on the video frames to be processed, and an initial normal map and an initial segmentation map corresponding to the video frames to be processed are generated. The initial segmentation map is preprocessed to generate a target segmentation map corresponding to the video frame to be processed, and the initial normal map is preprocessed to generate a target normal map corresponding to the video frame to be processed. Based on the target normal map and target segmentation map corresponding to the video frame to be processed, lighting processing is performed on the target light and shadow rendering region of the video frame to be processed according to the target lighting parameter information of the video frame to be processed, and the processed video frame is generated. This includes generating the target video based on the processed video frames.

[0009] In one optional embodiment, preprocessing the initial segmentation map to generate a target segmentation map corresponding to the video frame to be processed is: This includes merging the target segmentation map of an adjacent video frame to be processed that precedes the video frame to be processed with the initial segmentation map of the video frame to be processed to generate a target segmentation map corresponding to the video frame to be processed.

[0010] In one optional embodiment, preprocessing the initial normal map to generate a target normal map corresponding to the video frame to be processed is: This includes fusing the target normal map of an adjacent video frame to be processed that precedes the video frame to be processed with the initial normal map of the video frame to be processed to generate a target normal map corresponding to the video frame to be processed.

[0011] In one optional embodiment, the method is: In response to the user turning on the tracking function, the system performs target identification on the video frame to be processed and determines initial detection frame information corresponding to the target area in the video frame to be processed. The process further includes performing a smoothing process on the initial detection frame information of the video frame to be processed based on the target detection frame information of the adjacent video frame to be processed preceding the video frame to be processed, thereby generating target detection frame information corresponding to the video frame to be processed. Based on the target normal map and target segmentation map corresponding to the video frame to be processed, and according to the target lighting parameter information of the video frame to be processed, lighting processing is performed on the target light and shadow rendering region of the video frame to be processed, and the processed video frame is generated. The process includes performing lighting processing on the target light and shadow rendering region of the video frame to be processed, according to the target lighting parameter information of the video frame to be processed, based on the target detection frame information, target normal map, and target segmentation map corresponding to the video frame to be processed, and generating a processed video frame.

[0012] In one optional embodiment, after generating the processed video frame, In response to receiving ambient light parameters, the system adjusts the ambient light of the processed video frame based on the ambient light parameters and generates the adjusted video frame. If it is confirmed that the adjusted video frame contains a target event including an overexposure event or an underexposure event, the process returns to the step of adjusting the ambient light parameter to obtain a new ambient light parameter, adjusting the ambient light on the processed video frame based on the new ambient light parameter, and generating the adjusted video frame. If it is confirmed that the target event does not exist in the adjusted video frame, the adjusted video frame is further determined to be the target video frame. Generating the target video based on the processed video frames includes generating the target video based on the target video frames.

[0013] According to a second aspect, embodiments of the present disclosure further provide a video processing apparatus, the apparatus is A first display module for displaying instruction information, which includes first instruction information for instructing the user to select a light and shadow rendering area and second instruction information for instructing the user to select lighting parameter information, in response to receiving a lighting processing request for a video to be processed. A determination module used to determine, for a target video frame in the video to be processed, a target light and shadow rendering region selected by the user based on the first instruction information, and target lighting parameter information selected based on the second instruction information, wherein the target light and shadow rendering region and target lighting parameter information determined for the target video frame are used to perform lighting processing on a target video segment associated with the target video frame. The system includes the target light and shadow rendering region and a second display module for displaying the target video after lighting processing has been performed on the video to be processed based on the target lighting parameter information.

[0014] According to a third aspect, embodiments of the present disclosure further provide a computer device comprising a processor, memory, and a bus, wherein the memory stores machine-readable instructions that the processor can execute, and when the computer device is in operation, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the first aspect or any one possible embodiment of the first aspect are performed.

[0015] According to a fourth aspect, embodiments of the present disclosure further provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the first aspect or any one possible embodiment of the first aspect are performed.

[0016] To make the above-mentioned objectives, features, and advantages of this disclosure clearer and easier to understand, preferred embodiments are given below and described in detail together with the drawings. [Brief explanation of the drawing]

[0017] In the following, to more clearly explain the technical solutions of the embodiments of the present disclosure, the drawings necessary to be used in the embodiments will be briefly described. Here, the drawings are incorporated into this specification and constitute a part of this specification. These drawings show embodiments consistent with the present disclosure and are used together with the specification to explain the technical solutions of the present disclosure. Since the following drawings only show some embodiments of the present disclosure, they should not be regarded as limiting the scope. It should be understood by those skilled in the art that other related drawings can also be obtained based on these drawings without creative labor [Figure 1] The flowchart of the video processing method according to an embodiment of the present disclosure is shown. [Figure 2] The interface schematic diagram of the display interface in the video processing method according to an embodiment of the present disclosure is shown. [Figure 3a] The interface schematic diagram of the display interface where the light source position is indicated in the video processing method according to an embodiment of the present disclosure is shown. [Figure 3b] The interface schematic diagram of the display interface where the color bar is indicated in the video processing method according to an embodiment of the present disclosure is shown. [Figure 3c] The interface schematic diagram of the display interface where the intensity marker bar is indicated in the video processing method according to an embodiment of the present disclosure is shown. [Figure 4] The flowchart of another video processing method according to an embodiment of the present disclosure is shown [Figure 5] The schematic diagram of the video processing apparatus according to an embodiment of the present disclosure is shown. [Figure 6] It is a schematic configuration diagram of a computer device according to an embodiment of the present disclosure.

Modes for Carrying Out the Invention

[0018] To clarify the purpose, technical concepts, and advantages of the embodiments of this disclosure, the technical concepts of the embodiments of this disclosure will be described clearly and completely below with reference to the drawings of the embodiments of this disclosure. Clearly, the embodiments described are only a selection of embodiments of this disclosure, not all of them. The components of the embodiments of this disclosure generally described and shown in the drawings can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the drawings is not intended to limit the scope of the disclosure for which protection is sought, but rather to show only selected embodiments of this disclosure. All other embodiments obtained based on the embodiments of this disclosure without creative effort by a person skilled in the art are all within the scope of protection of this disclosure.

[0019] With the rapid development of internet technology, users have a need for lighting processing for videos displayed on smart devices. Intelligent lighting aims to enhance the beauty of light and shadow in video scenes, emphasize the light and dark relationships of the video screen, and increase the sense of atmosphere by redistributing light to video frames through three-dimensional spatial sensing of the video.

[0020] According to this disclosure, the present disclosure provides a video processing method that, in response to receiving a lighting processing request for a video to be processed, displays instruction information including first instruction information for instructing the user to select a light and shadow rendering region and second instruction information for instructing the user to select lighting parameter information, thereby facilitating the user to determine the required target light and shadow rendering region and target lighting parameter information, and providing personalized data support for subsequent lighting processing. Furthermore, after determining the target light and shadow rendering region selected by the user based on the first instruction information and the target lighting parameter information selected based on the second instruction information for a target video frame in the video to be processed, lighting processing can be performed on the video to be processed based on the target light and shadow rendering region and target lighting parameter information, a target video can be obtained, and the target video can be displayed, thereby realizing video lighting processing, and different users can realize different lighting processing, providing high flexibility in lighting processing.

[0021] Note that similar symbols and letters represent similar terms in the following diagrams; therefore, if a term is defined in one diagram, it is not necessary to further define and interpret it in subsequent diagrams.

[0022] In this specification, the terms "and / or" describe only related relationships and indicate that three relationships may exist. For example, A and / or B may represent three cases: A alone, A and B as a combination, and B alone. In this specification, the terms "at least one" represent any combination of one or at least two of a plurality. For example, including at least one of A, B, and C may mean including one or more elements selected from the set consisting of A, B, and C.

[0023] To ensure understanding, before using any of the technical proposals disclosed in each embodiment of this disclosure, users should be informed in an appropriate manner, in accordance with applicable laws and regulations, of the type of personal information related to this disclosure, its scope of use, and usage scenarios, and their permission should be obtained.

[0024] For example, in response to receiving a voluntary request from the user, prompt information is sent to the user to clearly prompt the user that the operation requested to be performed requires the acquisition and use of the user's personal information. This allows the user to choose whether or not to provide personal information to software or hardware such as electronic devices, application programs, servers, or storage media that perform the operation of the proposed technical method of this disclosure based on the prompt information.

[0025] As one optional but non-restrictive implementation, a method for sending prompt information to a user in response to receiving a voluntary request from the user may be, for example, a pop-up window, and the prompt information may be presented in text format within the pop-up window. Furthermore, the pop-up window may include a selection control that allows the user to choose whether to "agree" or "disagree" to providing personal information to the electronic device.

[0026] To ensure clarity, the above notice and user permission process are illustrative and not limiting to any implementation of this disclosure. Other methods that comply with applicable laws and regulations may also be applicable to implementation of this disclosure.

[0027] To facilitate understanding of this embodiment, the video processing method disclosed in the embodiments of this disclosure will first be described in detail. The execution body of the video processing method according to the embodiments of this disclosure is generally a computer device having a certain degree of computing power, and such computer device includes, for example, a terminal device, which may be a user device (UE), mobile device, user terminal, personal digital assistant (PDA), handheld device, computing device, etc. In several possible implementations, the video processing method can be implemented by calling computer-readable instructions stored in memory by a processor.

[0028] The following describes a video processing method according to an embodiment of this disclosure, using the example that the execution device is a terminal device.

[0029] Referring to Figure 1, it is a flowchart of a video processing method according to an embodiment of the present disclosure, the method comprising S101 to S103, where, S101, in response to receiving a lighting processing request for a video to be processed, displays instruction information including first instruction information to instruct the user to select a light and shadow rendering area, and second instruction information to instruct the user to select lighting parameter information.

[0030] The video to be processed may be video collected in real time by the imaging device of the terminal device, or it may be video uploaded to the terminal device. After acquiring the video to be processed, a writing processing request is triggered for the video to be processed, and there may be multiple triggering methods for the writing processing request, and this disclosure does not specifically limit them. For example, a "Writing" function button may be displayed on the display interface, and the writing request may be triggered by clicking the "Writing" function button. Alternatively, the writing processing request may be triggered by long-pressing the video to be processed.

[0031] The instruction information includes first instruction information and second instruction information, the first instruction information is used to instruct the user to select a light and shadow rendering area, which includes, but is not limited to, a global area, a background area, a foreground area, a target area within the foreground area, a customized local area, etc. Exemplarily, multiple light and shadow rendering areas may be displayed in the form of a list, or exemplary cards may be constructed that match each light and shadow rendering area, each exemplary card displaying a diagram of a lighting example matched to the light and shadow rendering area, and these are displayed in card form for the user to select.

[0032] The second instruction information is used to instruct the user to select lighting parameter information, where the lighting parameter information may be set as needed, and may include, but is not limited to, the number of light sources, light source positions, light source colors, light source intensity, light source range, dark area intensity, global dimming intensity, light source movement trajectory, light source movement speed, light source rotation speed, and light source color change speed.

[0033] For example, the second instruction information may include a number input box, thereby facilitating the user to input the required number of light sources. Alternatively, the second instruction information may include "Install light sources on the display interface," thereby facilitating the user to trigger the generation of the required light sources on the interface based on the second instruction information, with one light source generated for each trigger, the trigger position determined as the light source position, and the construction of at least one light source. A color bar (i.e., the second instruction information matching the light source color) may be further displayed, thereby facilitating the user to perform trigger operations on the color bar and determine the light source color for each light source. There are multiple forms in which the second instruction information can be displayed, and this disclosure does not specifically limit them.

[0034] S102, for the target video frame in the video to be processed, the target light and shadow rendering region selected by the user based on the first instruction information and the target lighting parameter information selected based on the second instruction information are determined, and the target light and shadow rendering region and target lighting parameter information determined for the target video frame are used to perform lighting processing on the target video segment associated with the target video frame.

[0035] The target video frame may be a keyframe in the video to be processed, where the keyframe may be determined by performing a decoding operation on the video to be processed. During implementation, after obtaining the video to be processed, the video to be processed and the target video frame contained within the video to be processed can be displayed as shown in Figure 2. Displaying the target video frame contained within the video frame to be processed facilitates the user to determine the matched target light and shadow rendering region and target lighting parameter information for the target video frame, where the target light and shadow rendering region and / or target lighting parameter information corresponding to multiple target video frames may be the same or different, i.e., this disclosure supports determining the target light and shadow rendering region and target lighting parameter information for multiple target video frames, and also supports multiple target video frames sharing the same target light and shadow rendering region and target lighting parameter information.

[0036] A target video frame may be associated with a target video segment. After determining the target light and shadow rendering region and target lighting parameter information corresponding to the target video frame, the target light and shadow rendering region and target lighting parameter information corresponding to the target video frame can be determined as the target light and shadow rendering region and target lighting parameter information that matches the associated target video segment. This facilitates performing lighting processing on the video frame in the target video segment associated with the target video frame using the target light and shadow rendering region and target lighting parameter information determined by the target video frame.

[0037] During execution, the user can select a target video frame, determine a target light and shadow rendering area for the selected target video frame based on first instruction information, and determine target lighting parameter information based on second instruction information. The target lighting parameter information is the specific content of the lighting parameters, and may include, for example, light source color - yellow, light source position - coordinates (x, y), etc.

[0038] S103, Based on the target light and shadow rendering area and the target lighting parameter information, the target video is displayed after the lighting processing has been performed on the video to be processed.

[0039] After determining the target light and shadow rendering area and target lighting parameter information, lighting processing is performed on the video to be processed based on the target light and shadow rendering area and target lighting parameter information, the target video is obtained, and displayed.

[0040] In one optional embodiment, displaying a target video after performing lighting processing on the video to be processed based on the target light rendering region and the target lighting parameter information includes, in response to the selected target light rendering region for the target video frame containing the target region, displaying a target video segment in the target video related to the target video frame, and displaying the effect after lighting the target region content in the video frame of the target video segment that matches the target region, according to the target lighting parameter information. Here, the target region includes at least one of a background region, a foreground region, a target area region within the foreground region, and a global region. If the target region includes a background region, the target region content that matches the background region includes the background content of the background region and some foreground content adjacent to the background content.

[0041] When the target area includes a background area, and when displaying a target video segment related to the target video frame in the target video, the effect after lighting is applied to the background content of the video frame of the target video segment and some foreground content adjacent to the background content, according to the target lighting parameter information, is displayed. In this disclosure, when lighting is applied to the background area, lighting is applied to the background content of the video frame and some foreground content adjacent to the background content to increase the realism of the lighting effect, thereby achieving a visual effect in which the background and some foreground are illuminated.

[0042] When the target area includes a foreground area or a target area within the foreground area, and a target video segment related to the target video frame in the target video is displayed, the effect after lighting is applied to the foreground content located in the foreground area or target area area according to the target lighting parameter information is displayed. Here, the target area of ​​the foreground area may be determined based on the first instruction information; for example, if the foreground area is a human body area, the target area may be a person's face, head, etc.

[0043] When the target area includes the global area and displays a target video segment related to the target video frame in the target video, the lighting effect is displayed on the global content of the video frame of the target video segment according to the target lighting parameter information, i.e., lighting is performed on the entire video frame.

[0044] By setting multiple target areas, flexible lighting processing for the video to be processed is achieved, improving the versatility of video lighting and enhancing the user experience.

[0045] In one optional embodiment, displaying a target video after performing lighting processing on the video to be processed based on the target light rendering region and the target lighting parameter information includes, in response to the target lighting parameter information including light source movement trajectory information, displaying a target video segment in the target video that is related to the target video frame, and displaying a dynamic lighting effect in the target light rendering region that matches the light source movement trajectory information.

[0046] During implementation, a light source movement trajectory may be set, for example, a pre-set trajectory may be used as the light source movement trajectory. The pre-set trajectory may be, for example, a circular trajectory surrounding the top of the head area, a circular trajectory surrounding the face area, etc. The user may be prompted by the second instruction information to draw the light source movement trajectory on the display interface. If there are multiple light sources installed, the multiple light sources may correspond to different light source movement trajectories, or they may be matched to the same light source movement trajectory.

[0047] When displaying a target video segment related to a target video frame in a target video, a dynamic lighting effect matching the light source movement trajectory information can be displayed in the target light and shadow rendering area. For example, if the light source movement trajectory information is a circular trajectory surrounding the face area, the dynamic lighting effect may be a lighting effect that highlights different areas of the face.

[0048] By setting the light source movement trajectory information here, it is possible to increase the dynamic lighting effect in the video to be processed, thereby improving the video lighting effect.

[0049] The following exemplifies several lighting scenes that can be realized by the method proposed in this disclosure.

[0050] Scene 1, Basic Lighting - Panoramic Lighting. During execution, based on the first instruction information, it can be confirmed that the target light and shadow rendering area is a global area, and based on the second instruction information, the target lighting parameter information for global area lighting can be determined. For each target video frame in the video frames to be processed in the scene, one or more light sources can be added, and for each light source, one or more light source parameters such as light source position, light source color, light source intensity, light source range, dark area intensity, and global dimming degree can be determined. When performing lighting processing on the video to be processed in the scene, it is possible to illuminate the background and foreground simultaneously and achieve a shading effect in the background.

[0051] For example, a pre-set shape representing the light source position, such as a circular shape, can be set, and the light source position can be adjusted by moving the pre-set shape in the display interface. Referring to Figure 3a, Figure 3a shows the light source position. For example, a color bar may also be set up, containing color information that changes in a gradient, and the user can select the desired light source color from the color bar and adjust the light source color by sliding in the color gradient area. Referring to Figure 3b, Figure 3b shows the color bar. Alternatively, for example, an intensity marker bar may be set up, and the light source intensity can be adjusted by adjusting the position of the adjustment marker on the intensity marker bar. Refer to the intensity marker bar shown in Figure 3c. Here, the adjustment methods for the light source range, dark area intensity, and global dimming degree can be found in the light source intensity adjustment method, and are omitted from this explanation.

[0052] Scene 2, Basic Lighting - Foreground Lighting. During execution, based on the first instruction information, it can be confirmed that the target light and shadow rendering area is the foreground area or a target area on the foreground area, and based on the second instruction information, target lighting parameter information for foreground area lighting can be determined. For each target video frame in the video frames to be processed in the scene, one or more light sources can be added, and for each light source, one or more light source parameters such as light source position, light source color, light source intensity, light source range, dark area intensity, and global dimming degree can be determined. When performing lighting processing on the video to be processed in the scene, lighting effects can be achieved by lighting the front and sides of the foreground, the light source position can be fixed to achieve a fixed-position lighting effect, and the light source position can be set to move according to the foreground object to achieve a moving lighting effect.

[0053] Scene 3: Basic Lighting - Background Lighting. During execution, based on the first instruction information, it can be confirmed that the target light and shadow rendering area is the background area, and based on the second instruction information, the target lighting parameter information for background area lighting can be determined. For each target video frame in the video frames to be processed in the scene, one or more light sources can be added, and for each light source, one or more light source parameters can be determined, such as light source position, light source color, light source intensity, light source range, dark area intensity, and global dimming degree. When performing lighting processing on the video to be processed in the scene, the lighting effect can be improved by illuminating the background and some foreground areas at the same time, and the light source shape of the background light source can be specified, such as a sunset light or a straight light tube.

[0054] Scene 4 - Dynamic Lighting Effects. When implemented, the target light and shadow rendering area can be determined based on the first instruction information, and in this scene, the target light and shadow rendering area may be any one of the foreground area, background area, or global area. Based on the second instruction information, target lighting parameter information can be determined, and the target lighting parameter information includes light source movement trajectory information, and the start point, end point, trajectory points, etc. of the light source movement trajectory information can be adjusted. Here, the dynamic lighting effects may include multiple effects, and the light source movement trajectory information and light source parameters can be determined based on the effect that the dynamic lighting effects aim to achieve. Dynamic lighting effects may be set as needed, and for example, the dynamic lighting effects may include, but are not limited to, a rotating light effect, a rainbow light effect, a glowing butterfly light effect, a blinking light effect, a music light effect, etc.

[0055] For example, if a dynamic lighting effect includes a rotational lighting effect, the light source movement trajectory information can instruct the light source to rotate around the top of the foreground subject's head and to render around the face area. Light source parameters include the starting light source position, light source color, light source intensity, light source range, dynamic light source rotation speed, and dynamic lighting effect color change speed.

[0056] For example, if the dynamic lighting effect includes a rainbow lighting effect, then in the dynamic lighting effect, the light source moves randomly to illuminate a person's face, the light source movement trajectory information is random, and the light source parameters include the richness of the light source color, the light source intensity, the light source range, and the random movement speed of the dynamic light source.

[0057] For example, if the dynamic light effect is a luminous butterfly effect, the shape of the light source is specified as a butterfly shape, and the light source parameters, such as the light source color and light source intensity, are set. For example, if the dynamic light effect is a flashing light effect, the light source intensity can be changed according to a preset frequency to achieve a flashing effect. For example, in a music light effect, the beat of the music being played can be identified, and the movement speed and light source intensity of the light source can be adjusted based on the music beat.

[0058] The following describes the writing process in detail.

[0059] In one optional embodiment, lighting processing is performed on the video to be processed based on the target light and shadow rendering region and the target lighting parameter information to obtain the target video. Step a1 involves performing detection on the video frames to be processed within the target video segment included in the video to be processed, and generating an initial normal map and an initial segmentation map corresponding to the video frames to be processed. Step a2 involves preprocessing the initial segmentation map to generate a target segmentation map corresponding to the video frame to be processed, and preprocessing the initial normal map to generate a target normal map corresponding to the video frame to be processed. Step a3: Based on the target normal map and target segmentation map corresponding to the video to be processed, lighting processing is performed on the target light and shadow rendering region of the video frame to be processed according to the target lighting parameter information of the video frame to be processed, and a processed video frame is generated. The process includes step a4, which generates the target video based on the processed video frames.

[0060] In step a1, a multitask neural network model can be used to detect the video frames to be processed within the target video segment of the video to be processed, and an initial normal map and initial segmentation map corresponding to the video frames to be processed can be generated. Here, the network structure of the multitask neural network model may be determined as needed. The video frames to be processed may be each video frame contained in the target video segment, or at least some of the video frames within the target video segment.

[0061] Here, the size of the initial normal map and the initial segmentation map may be the same as the size of the video frame to be processed. The pixel information of each pixel point in the initial normal map represents the normal information of the pixel point at the same pixel position in the video frame to be processed, and the pixel information of each pixel point in the initial segmentation map represents the type of object to which the pixel point at the same pixel position in the video frame to be processed belongs. The type of object may include, for example, a foreground type and a background type, or a specific type of object and a background type, and the type of object may be, for example, a human body, an animal, a plant, etc.

[0062] In step a2, preprocessing may be performed on the initial segmentation map to improve the accuracy of the segmentation map. For example, preprocessing such as blurring may be performed on the initial segmentation map to obtain a target segmentation map. To improve the accuracy of the normal map, preprocessing such as guided filtering and noise reduction may be performed on the initial normal map to obtain a target normal map. When lighting processing is performed using the target normal map and target segmentation map obtained after processing, the resulting target video is smooth and the display effect is desirable. And / or, preprocessing may also be performed on the initial segmentation map and initial normal map using the target segmentation map and target normal map of the video frame to be processed in the history before the video frame to be processed.

[0063] In one optional embodiment, preprocessing the initial segmentation map to generate a target segmentation map corresponding to the video frame to be processed includes merging the target segmentation map of an adjacent video frame to be processed prior to the video frame to be processed with the initial segmentation map of the video frame to be processed to generate a target segmentation map corresponding to the video frame to be processed.

[0064] To mitigate the problem of abrupt segmentation changes between adjacent video frames and improve the video smoothing effect, the target segmentation map of the previous video frame to be processed and the initial segmentation map of the current video frame to be processed (or the processed segmentation map obtained after pre-processing such as blurring) can be merged to obtain a target segmentation map corresponding to the video to be processed. Here, the merging process may be performed according to a first predetermined ratio. For example, the pixel information at the same pixel position in the target segmentation map of the adjacent video frame to be processed before the video frame to be processed and the initial segmentation map of the video frame to be processed can be added according to the ratio to obtain fused pixel information at that pixel position, and the corresponding fused pixel information can be processed based on each pixel position to obtain a target segmentation map corresponding to the video frame to be processed.

[0065] In this disclosure, by fusing the initial segmentation map of the video frame to be processed with the target segmentation map of the previous video frame to be processed, fusing segmentation information at the timing between adjacent video frames to be processed in the video to be processed is achieved, thereby improving the accuracy of the target segmentation map and making it easier to subsequently obtain a target video with a favorable display effect based on the target segmentation map.

[0066] In one optional embodiment, preprocessing the initial normal map to generate a target normal map corresponding to the video frame to be processed includes merging the target normal map of an adjacent video frame to be processed prior to the video frame to be processed with the initial normal map of the video frame to be processed to generate a target normal map corresponding to the video frame to be processed.

[0067] During execution, the target normal map of the adjacent video frame to be processed before the video frame to be processed may be merged with the initial normal map of the video frame to be processed. Alternatively, the initial normal map of the video frame to be processed may first be processed, for example, by guided filtering or denoising, and then the processed normal map may be merged with the target normal map of the previous video frame to be processed.

[0068] Here, the fusion process may be a fusion according to a second predetermined ratio. For example, the target normal map of the adjacent video frame to be processed before the video frame to be processed and the pixel information at the same pixel position in the initial normal map (or processed normal map) of the video frame to be processed are added according to the ratio to obtain fused normal information at that pixel position, the fused normal information corresponding to each pixel position is processed based on each pixel position to obtain a target normal map corresponding to the video frame to be processed.

[0069] In this disclosure, by fusing the initial normal map of the video frame to be processed with the target normal map of the previous video frame to be processed, normal information fusing is achieved at the timing between adjacent video frames to be processed in the video to be processed, thereby improving the accuracy of the target normal map and making it easier to subsequently obtain a target video with a favorable display effect based on the target normal map.

[0070] In step a3, after obtaining the target normal map and target segmentation map corresponding to the video frame to be processed, the pixel values ​​of the pixel points in the target segmentation map are first adjusted based on the target light and shadow rendering region. For example, the pixel values ​​of target pixel points whose pixel positions in the target segmentation map match the target light and shadow rendering region are adjusted to a first value, for example, 1, and the pixel values ​​of other pixel points are adjusted to a second value, for example, 0, to obtain the adjusted segmentation map. Furthermore, the adjusted segmentation map and the target normal map are merged to obtain the merged normal map. Then, using a lighting model, for example, a Lambert lighting model, lighting processing is performed on the video frame to be processed in accordance with the target lighting parameter information of the video frame to be processed and the merged normal map, and the processed video frame is obtained.

[0071] For example, based on the merged normal map, the normal vector of each pixel point in the video frame to be processed can be determined, and based on the target lighting parameter information, the inverse vector of the incident ray can be determined. Based on the inverse vector, the normal vector, and the light intensity of the determined incident ray, the light intensity of the reflected ray corresponding to each pixel point in the video frame to be processed can be determined and obtained. Based on the light intensity of the reflected ray of each pixel point, pixel adjustments can be performed on the pixel information of each pixel point in the video frame to be processed to obtain the processed video frame.

[0072] In step a4, multiple processed video frames are coded and arranged to obtain the target video. Alternatively, if there are other video frames that have not undergone writing processing, the multiple processed video frames and the other video frames may be coded and arranged according to timing information to obtain the target video.

[0073] Here, the obtained target depth map and target segmentation map are used to perform lighting processing on the video frames to be processed, according to the target lighting parameter information, thereby making it easier to obtain the target video that meets the user's needs in subsequent processes.

[0074] In one optional embodiment, a tracking function may be pre-configured, and when the user turns on the tracking function, lighting processing can be performed on the target object being tracked using a light source. When specifically implemented, the method is as follows: The system further includes, in response to the user turning on the tracking function, performing target identification on the video frame to be processed and determining initial detection frame information corresponding to the target area in the video frame to be processed, and performing a smoothing process on the initial detection frame information of the video frame to be processed based on the target detection frame information of the adjacent video frame to be processed prior to the video frame to be processed, thereby generating target detection frame information corresponding to the video frame to be processed.

[0075] The user can turn on the tracking function in the display interface, for example by clicking the tracking function button. When turning on the tracking function, the user can also set the target object, which facilitates target object identification for the video frames to be processed. Here, the target object may be determined based on the video content, for example, a person's face, a pet's face, etc.

[0076] In response to the user turning on the tracking function, target identification is performed on the video frame to be processed, and initial detection frame information is obtained for the target region (i.e., the region where the target object is located) in the video frame to be processed. The initial detection frame information may include pixel coordinate information of the four vertices of the detection frame.

[0077] To mitigate the jitter problem caused by abrupt changes in the position of the target object in adjacent video frames to be processed, smoothing may be performed on the initial detection frame information of the video frame to be processed, based on the target detection frame information of the adjacent video frame to be processed preceding the video frame to be processed. For example, averaging can be performed on the pixel coordinate information of the top-left vertex indicated by the target detection frame information of the adjacent video frame to be processed and the pixel coordinate information of the top-left vertex indicated by the initial detection frame information of the video frame to be processed to obtain the processed pixel coordinate information of the top-left vertex, and similarly, processed pixel coordinate information of the four vertices can be obtained, and the processed pixel coordinate information of the four vertices constitutes the target detection frame information corresponding to the video frame to be processed.

[0078] Furthermore, based on the target detection frame information, target normal map, and target segmentation map corresponding to the video frame to be processed, lighting processing can be performed on the target light and shadow rendering region of the video frame to be processed according to the target lighting parameter information of the video frame to be processed, thereby generating the processed video frame.

[0079] Here, if the target light and shadow rendering region matches a target object included within the target detection frame, for example, if the target detection frame includes the area of ​​a person's face, then the target light and shadow rendering region is the area of ​​a person's face. During execution, adjustments are made to the pixel values ​​of the pixel points in the target segmentation map based on the target detection frame information. For example, within the target detection frame of the target segmentation map, the pixel values ​​of target pixel points whose pixel positions match the target light and shadow rendering region are adjusted to a first value, for example, 1, and the pixel values ​​of other pixel points other than the target pixel points are adjusted to a second value, for example, 0, thereby obtaining the adjusted segmentation map.

[0080] Furthermore, the adjusted segmentation map and target normal map are merged to obtain a merged normal map. Then, using a lighting model, such as the Lambert lighting model, lighting processing is performed on the video frame to be processed, according to the target lighting parameter information of the video frame to be processed and the merged normal map, thereby obtaining the processed video frame.

[0081] Here, by setting a tracking function, such as a human face tracking function, the lighting process utilizes the target detection frame information of the adjacent video frame to be processed before the video frame to be processed for the target of the tracking function. This process smooths the initial detection frame information of the video frame to be processed, obtaining target detection frame information corresponding to the video frame to be processed. Subsequently, lighting processing can be accurately performed on the target object based on the target detection frame information, mitigating the jitter problem of the target object and improving the video lighting effect.

[0082] After obtaining the processed video frame, ambient light can be added to the processed video frame to improve the video lighting effect.

[0083] In one optional embodiment, after generating a processed video frame, in response to receiving ambient light parameters, the process further includes: adjusting the ambient light on the processed video frame based on the ambient light parameters to generate an adjusted video frame; if it is confirmed that the adjusted video frame contains a target event including an overexposure or underexposure event, adjusting the ambient light parameters to obtain new ambient light parameters, and returning to the step of adjusting the ambient light on the processed video frame based on the ambient light parameters to generate an adjusted video frame; and if it is confirmed that the adjusted video frame does not contain a target event, determining the adjusted video frame as the target video frame. Generating the target video based on the processed video frames includes generating the target video based on the target video frames.

[0084] During execution, the user can input ambient light parameters, and in response to the received ambient light parameters, ambient light adjustment is performed on the processed video frame based on those parameters. That is, the processed video frame is occluded using the selected ambient light, and an adjusted video frame is generated.

[0085] The system determines whether a target event exists in the adjusted video frame, i.e., whether an overexposure event and / or underexposure event exists. If it does not exist, the adjusted video frame is used as the target video frame. If it does exist, the ambient light parameters are re-determined, and ambient light adjustments are performed on the processed video frame using the new ambient light parameters until an adjusted video frame without a target event is obtained. Here, the ambient light parameters can be re-determined based on the target event. For example, if the target event is an overexposure event, the re-determined ambient light parameters will be darker than the previous ambient light parameters. Conversely, if the target event is an underexposure event, the re-determined ambient light parameters will be brighter than the previous ambient light parameters.

[0086] During implementation, it is possible to determine whether or not a target event exists in the adjusted video frame based on the following process: Determine an event threshold corresponding to each pixel point in the adjusted video frame, the event threshold includes a first event threshold that matches the overexposure event and a second event threshold that matches the underexposure event, determine a first number of overexposure pixel points in the adjusted video frame whose pixel value is greater than the first event threshold, determine whether or not an overexposure event exists in the adjusted video frame based on the first number of overexposure pixel points in the adjusted video frame, determine a second number of underexposure pixel points in the adjusted video frame whose pixel value is less than the second event threshold, and determine whether or not an overexposure event exists in the adjusted video frame based on the second number of underexposure pixel points in the adjusted video frame.

[0087] During implementation, for each pixel point in the adjusted video frame, the pixel information at that pixel point in the video frame to be processed is determined based on the pixel position of the pixel point, and a first event threshold and a second event threshold corresponding to the pixel point can be determined based on the pixel information and the set overexposure ratio and underexposure ratio. For example, first, the pixel information and the overexposure ratio can be multiplied to obtain a first multiplicative value, and then the sum obtained by adding the first multiplicative value and the pixel information can be set as the first event threshold. Second, the pixel information and the underexposure ratio can be multiplied to obtain a second multiplicative value, and then the difference obtained by subtracting the second multiplicative value from the pixel information can be set as the second event threshold. Furthermore, an event threshold corresponding to each pixel point in the adjusted video frame can be obtained, and the event threshold includes the first event threshold and the second event threshold.

[0088] The first pixel point in the adjusted video frame whose pixel value is greater than a first event threshold is statistically analyzed, and this first pixel point is determined to be an overexposed pixel point. The first number of overexposed pixel points is then determined. If the first number is greater than a first preset number, it is determined that an overexposed event exists. Conversely, if the first number is greater than a first preset number, it is determined that no overexposed event exists. Alternatively, the overexposure ratio can be determined based on the first number and the total number of pixel points in the adjusted video frame. If the overexposure ratio is greater than a first preset ratio, it is determined that an overexposed event exists. Conversely, if the first number is greater than a preset ratio, no overexposure event exists.

[0089] The second pixel point in the adjusted video frame whose pixel value is smaller than the second event threshold can be statistically identified, and this second pixel point can be determined as an underexposed pixel point. The second number of underexposed pixel points can then be determined. If the second number is greater than a second preset number, it is determined that an underexposed event exists. Conversely, if the second number is greater than the second preset number, it is determined that no underexposed events exist. Alternatively, the underexposure ratio can be determined based on the second number and the total number of pixel points in the adjusted video frame. If the underexposure ratio is greater than a second preset ratio, it is determined that an underexposed event exists. Conversely, if the second number is greater than the preset ratio, no underexposure events exist.

[0090] Finally, multiple target video frames can be coded and arranged according to timing information to generate the target video.

[0091] This disclosure allows for the creation of target video frames to match scene requirements by adding ambient light to the processed video frames, and by performing underexposure detection or overexposure detection after adding ambient light, it is possible to mitigate the problem of poor display quality when the adjusted video frames are overexposed or underexposed, thereby ensuring the effectiveness of the video processing.

[0092] Referring to the flowchart shown in Figure 4, and while linking to Figure 4, the video processing method will be explained exemplarily, and the video to be processed may include a user. The method may also include the following:

[0093] Firstly, target video frame display. Specifically, after receiving the video to be processed, the system can determine and display the video frame that needs to be processed within that video.

[0094] Secondly, there are the normal mode and segmentation mode. Specifically, the video frames to be processed are input into the normal model and segmentation model, detection is performed, and initial normal maps and initial segmentation maps for the video frames to be processed are obtained.

[0095] Thirdly, there is face detection. If the user turns on the face tracking function, the system will proceed to face detection, for example, by using a face detection model to detect a human face and obtain initial detection frame information corresponding to the human face area. If the user does not turn on the face tracking function, there is no need to perform face detection.

[0096] Fourth, acceptance of the lighting mode. The lighting mode is used to specify the target light and shadow rendering area.

[0097] Fifth, structural information processing. Specifically, the initial normal map output from the normal model can be preprocessed, for example, by guided filtering and noise reduction, to obtain the processed normal map. The initial segmentation map output from the segmentation model can be preprocessed, for example, by blurring, to obtain the processed segmentation map.

[0098] Sixth, timing information fusion and parameter smoothing. Specifically, the timing information fusion process includes the following: Fusing the target segmentation map of the adjacent video frame to be processed that precedes the video frame to be processed with the segmentation map of the video frame to be processed to generate a target segmentation map corresponding to the video frame to be processed. Fusing the target normal map of the adjacent video frame to be processed that precedes the video frame to be processed with the normal map of the video frame to be processed to generate a target normal map corresponding to the video frame to be processed.

[0099] The parameter smoothing process includes performing a smoothing operation on the initial detection frame information of the video frame to be processed, based on the target detection frame information of the adjacent video frame to be processed that precedes the video frame to be processed, in order to generate target detection frame information corresponding to the video frame to be processed.

[0100] Here, the parameter smoothing process is performed when the human face tracking function is turned on, and conversely, if the human face tracking function is not turned on, the parameter smoothing process is not performed.

[0101] Seventh, light and shadow rendering. Specifically, target lighting parameter information is received, and lighting processing is performed based on the target lighting parameter information, target segmentation map, and target normal map to obtain the processed video frame.

[0102] When the human face tracking function is turned on, lighting processing is performed based on target lighting parameter information, target detection frame information, target segmentation map, and target normal map, and the processed video frame is obtained.

[0103] Eighth, ambient light adjustment. Specifically, ambient light parameters are received, ambient light adjustment is performed on the processed video frame based on these parameters, and the adjusted video frame is generated.

[0104] Ninth, overexposure event detection and underexposure event detection. Specifically, it is determined whether or not overexposure and underexposure events exist in the adjusted video frame. If none exist, step 10 is performed; if they exist, the ambient light parameters are re-determined and the process returns to step 8.

[0105] Tenth, the target video frame output. Here, the target video contains the target video frame.

[0106] Those skilled in the art will understand that in the above-described method of a specific embodiment, the order in which each step is created does not constitute a strict execution order or any limitation on the implementation process, but rather the specific execution order of each step should be determined by its function and possible inherent logic.

[0107] Based on the same concept of the invention, embodiments of this disclosure also provide a video processing apparatus corresponding to a video processing method, and since the principle by which the apparatus in embodiments of this disclosure solves the problem is similar to that of the video processing method in embodiments of this disclosure, the implementation of the apparatus can refer to the implementation of the method, and the overlaps will no longer be explained.

[0108] Referring to Figure 5, it is a schematic diagram of the architecture of a video processing device according to an embodiment of the present disclosure, the device comprising a first display module 501, a determination module 502, and a second display module 503, where, The first display module 501 is used to display instruction information, which includes first instruction information for instructing the user to select a light and shadow rendering area and second instruction information for instructing the user to select lighting parameter information, in response to receiving a lighting processing request for a video to be processed. The determination module 502 is used to determine, for a target video frame in the video to be processed, the target light and shadow rendering region selected by the user based on the first instruction information, and the target lighting parameter information selected based on the second instruction information, respectively. Here, the target light and shadow rendering region and target lighting parameter information determined for the target video frame are used to perform lighting processing on the target video segment associated with the target video frame. The second display module 503 is used to display the target video after lighting processing has been performed on the video to be processed based on the target light and shadow rendering area and the target lighting parameter information.

[0109] In one optional embodiment, when the second display module 503 displays the target video after performing lighting processing on the video to be processed based on the target light and shadow rendering area and the target lighting parameter information, In response to the selected target light and shadow rendering region for the target video frame containing the target region, when displaying a target video segment in the target video related to the target video frame, it is used to display the effect after lighting has been applied to the target region content in the video frame of the target video segment that matches the target region, according to the target lighting parameter information. Here, the target region includes at least one of a background region, a foreground region, a target area region within the foreground region, and a global region. If the target region includes a background region, the target region content that matches the background region includes the background content of the background region and some foreground content adjacent to the background content.

[0110] In one optional embodiment, when the second display module 503 displays the target video after performing lighting processing on the video to be processed based on the target light and shadow rendering area and the target lighting parameter information, In response to the fact that the target lighting parameter information includes light source movement trajectory information, when displaying a target video segment related to the target video frame in the target video, it is used to display a dynamic lighting effect that matches the light source movement trajectory information in the target light and shadow rendering region.

[0111] In one optional embodiment, the second display module 503 performs lighting processing on the video to be processed based on the target light and shadow rendering area and the target lighting parameter information to obtain the target video. For each video frame to be processed within the target video segment included in the video to be processed, detection is performed on the video frame to be processed, and an initial normal map and an initial segmentation map corresponding to the video frame to be processed are generated. The initial segmentation map is preprocessed to generate a target segmentation map corresponding to the video frame to be processed, the initial normal map is preprocessed to generate a target normal map corresponding to the video frame to be processed, Based on the target normal map and target segmentation map corresponding to the video frame to be processed, lighting processing is performed on the target light and shadow rendering region of the video frame to be processed according to the target lighting parameter information of the video frame to be processed, and the processed video frame is generated. The processed video frames are used to generate the target video.

[0112] In one optional embodiment, the second display module 503 preprocesses the initial segmentation map to generate a target segmentation map corresponding to the video frame to be processed, This method is used to merge the target segmentation map of the adjacent video frame to be processed that precedes the video frame to be processed with the initial segmentation map of the video frame to be processed, in order to generate a target segmentation map corresponding to the video frame to be processed.

[0113] In one optional embodiment, the second display module 503 preprocesses the initial normal map to generate a target normal map corresponding to the video frame to be processed, This is used to merge the target normal map of the adjacent video frame to be processed that precedes the video frame to be processed with the initial normal map of the video frame to be processed, in order to generate a target normal map corresponding to the video frame to be processed.

[0114] In one optional embodiment, the apparatus is In response to the user turning on the tracking function, target identification is performed on the video frame to be processed, and initial detection frame information corresponding to the target area in the video frame to be processed is determined. The system further includes a detection module 504 for generating target detection frame information corresponding to the video frame to be processed by performing a smoothing process on the initial detection frame information of the video frame to be processed based on the target detection frame information of the adjacent video frame to be processed that precedes the video frame to be processed. The second display module 503, based on the target normal map and target segmentation map corresponding to the video frame to be processed, performs lighting processing on the target light and shadow rendering region of the video frame to be processed according to the target lighting parameter information of the video frame to be processed, and generates the processed video frame, Based on the target detection frame information, target normal map, and target segmentation map corresponding to the video frame to be processed, lighting processing is performed on the target light and shadow rendering region of the video frame to be processed according to the target lighting parameter information of the video frame to be processed, and this is used to generate the processed video frame.

[0115] In one optional embodiment, after generating the processed video frame, In response to receiving ambient light parameters, ambient light adjustment is performed on the processed video frame based on the ambient light parameters, and the adjusted video frame is generated. If it is confirmed that the adjusted video frame contains a target event including an overexposure event or an underexposure event, the ambient light parameter is adjusted to obtain a new ambient light parameter, and the process returns to the step of adjusting the ambient light on the processed video frame based on the new ambient light parameter to generate the adjusted video frame. If it is confirmed that the target event does not exist in the adjusted video frame, the adjustment module 505 further includes a module for determining the adjusted video frame as the target video frame. Generating the target video based on the processed video frames includes generating the target video based on the target video frames.

[0116] The processing flow of each module within the device, and the interaction flow between each module, can be described by referring to the relevant explanations in the embodiments of the above method, and will not be explained in detail here.

[0117] Based on the same technical concept, embodiments of this disclosure further provide computer equipment. Referring to Figure 6, it is a schematic diagram of the configuration of computer equipment 600 according to embodiments of this disclosure. Computer equipment in embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable media players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers, or various portable servers, such as standalone servers or server clusters. The computer equipment shown in Figure 6 is merely an example and does not imply any limitation on the functions and scope of use of embodiments of this disclosure.

[0118] As shown in Figure 6, the computer device 600 may include a processing unit (e.g., a central processing unit, graphics processor, etc.) 601, which can perform various appropriate operations and processes depending on the program stored in the read-only memory (ROM) 602 or the program loaded from the storage device 608 into the random access memory (RAM) 603. The RAM 603 also stores various programs and data necessary for the operation of the computer device 600. The processing unit 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0119] Generally, devices such as input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, and gyroscopes; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, and vibrators; storage devices 608 including, for example, magnetic tape and hard disks; and communication devices 609 can be connected to the I / O interface 605. The communication device 609 can allow the computer equipment 600 and other devices to exchange data via wireless or wired communication. Figure 6 shows computer equipment 600 with various devices, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have all of the devices instead.

[0120] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowchart may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product which includes a computer program placed on a computer-readable medium, the computer program including program code for performing a video processing method. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-described functions, limited by the methods of embodiments of the present disclosure, are performed.

[0121] Embodiments of the present disclosure further provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the video processing method described in the embodiment of the above method are performed. Here, the storage medium may be a volatile or non-volatile computer-readable storage medium.

[0122] Embodiments of the present disclosure further provide a computer program product on which program code is embedded, and instructions contained in the program code can be used to perform the steps of the video processing method described in the embodiment of the above method, specifically referencing the embodiment of the above method, which is omitted here.

[0123] Here, the computer program product described above can be specifically implemented by hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically implemented as a computer storage medium, and in another optional embodiment, the computer program product is specifically implemented as a software product such as a software development kit (SDK).

[0124] As will be apparent to those skilled in the art, for the sake of clarity and conciseness, the specific operating processes of the systems and apparatus described above can be referenced to the corresponding processes in the embodiments of the methods described above and will not be repeated here. In some embodiments of this disclosure, it should be understood that the disclosed systems, apparatus, and methods can be implemented in other ways. The embodiments of the apparatus described above are merely illustrative, and for example, the division of the units is merely a logical functional division, and there may be other methods of division in actual implementation, for example, multiple units or components may be combined, or integrated into another system, or some features may be ignored or not implemented. Also, the combinations, direct combinations, or communication connections between them that are shown or discussed may be indirect combinations or communication connections via some communication interfaces, devices, or units, and may be in electrical, mechanical, or other forms.

[0125] The units shown as separate components may or may not be physically separated, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Depending on the actual needs, some or all of these units can be selected to achieve the objectives of the technical proposal of this embodiment.

[0126] Each functional unit in each embodiment of the present disclosure may be integrated into a single processing unit, each unit may exist physically independently, or two or more units may be integrated into a single unit.

[0127] When the aforementioned functions are implemented in the form of a software function unit and sold or used as an independent product, they can be stored in a processor-executable non-volatile computer-readable storage medium. With this understanding, the proposed techniques of the present disclosure may be substantially, or contribute to the prior art, or relate to the proposed techniques, in the form of a software product. This computer software product is stored in a storage medium and includes some instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present disclosure. The storage medium includes various media capable of storing program code, such as USB memory, removable hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0128] The embodiments described above are merely specific embodiments of the Disclosure and are used to illustrate the technical ideas of the Disclosure, not to limit them, and the scope of protection of the Disclosure is not limited thereto. However, although the Disclosure has been described in detail with reference to the embodiments, those skilled in the art should understand that within the scope of the art disclosed, they can still easily modify, change, or substitute some of the technical features described in the embodiments, and such modifications, changes, or substitutions should be included within the scope of protection of the Disclosure without causing the essence of the corresponding technical idea to deviate from the spirit and scope of the technical idea of ​​the embodiments of the Disclosure. Therefore, the scope of protection of the Disclosure should be in accordance with the claims.

Claims

1. A video processing method, In response to receiving a lighting processing request for a video to be processed, the system displays instruction information including first instruction information to instruct the user to select a light and shadow rendering area, and second instruction information to instruct the user to select lighting parameter information. For the target video frame in the video to be processed, the target light and shadow rendering region selected by the user based on the first instruction information and the target lighting parameter information selected based on the second instruction information are determined, and the target light and shadow rendering region and target lighting parameter information determined for the target video frame are used to perform lighting processing on the target video segment associated with the target video frame. A video processing method comprising displaying a target video after performing lighting processing on the video to be processed based on the target light and shadow rendering region and the target lighting parameter information.

2. Displaying the target video after performing lighting processing on the video to be processed based on the target light and shadow rendering area and the target lighting parameter information is, When displaying a target video segment in the target video that is related to the target video frame, in response to the selected target light and shadow rendering region for the target video frame containing the target region, the effect after lighting has been applied to the target region content in the video frame of the target video segment that matches the target region, according to the target lighting parameter information, is displayed. The method according to claim 1, wherein the target region includes a background region, a foreground region, and at least one of a target area region and a global region within the foreground region, and if the target region includes a background region, the target region content that matches the background region includes background content of the background region and a portion of foreground content adjacent to the background content.

3. Displaying the target video after performing lighting processing on the video to be processed based on the target light and shadow rendering area and the target lighting parameter information is, The method according to claim 1, wherein, in response to the target lighting parameter information including light source movement trajectory information, when displaying a target video segment related to the target video frame in the target video, a dynamic lighting effect that matches the light source movement trajectory information is displayed in the target light and shadow rendering region.

4. Performing lighting processing on the video to be processed based on the target light and shadow rendering region and the target lighting parameter information to obtain the target video is: For the video frames to be processed within the target video segment included in the video to be processed, detection is performed on the video frames to be processed, and an initial normal map and an initial segmentation map corresponding to the video frames to be processed are generated. The initial segmentation map is preprocessed to generate a target segmentation map corresponding to the video frame to be processed, and the initial normal map is preprocessed to generate a target normal map corresponding to the video frame to be processed. Based on the target normal map and target segmentation map corresponding to the video frame to be processed, lighting processing is performed on the target light and shadow rendering region of the video frame to be processed according to the target lighting parameter information of the video frame to be processed, and the processed video frame is generated. The method according to any one of claims 1 to 3, comprising generating the target video based on the processed video frames.

5. Performing preprocessing on the initial segmentation map and generating a target segmentation map corresponding to the video frame to be processed is, The method according to claim 4, comprising fusing the target segmentation map of an adjacent video frame to be processed prior to the video frame to be processed with the initial segmentation map of the video frame to be processed to generate a target segmentation map corresponding to the video frame to be processed.

6. Performing preprocessing on the initial normal map and generating a target normal map corresponding to the video frame to be processed is, The method according to claim 4 or 5, comprising fusing the target normal map of an adjacent video frame to be processed prior to the video frame to be processed with the initial normal map of the video frame to be processed to generate a target normal map corresponding to the video frame to be processed.

7. In response to the user turning on the tracking function, the system performs target identification on the video frame to be processed and determines initial detection frame information corresponding to the target area in the video frame to be processed. The process further includes performing a smoothing process on the initial detection frame information of the video frame to be processed based on the target detection frame information of the adjacent video frame to be processed preceding the video frame to be processed, thereby generating target detection frame information corresponding to the video frame to be processed. Based on the target normal map and target segmentation map corresponding to the video frame to be processed, and according to the target lighting parameter information of the video frame to be processed, lighting processing is performed on the target light and shadow rendering region of the video frame to be processed, and the processed video frame is generated. The method according to any one of claims 4 to 6, comprising performing lighting processing on the target light and shadow rendering region of the video frame to be processed according to the target lighting parameter information of the video frame to be processed, based on the target detection frame information, target normal map, and target segmentation map corresponding to the video frame to be processed, and generating a processed video frame.

8. After generating the processed video frame, In response to receiving ambient light parameters, the system adjusts the ambient light of the processed video frame based on the ambient light parameters and generates the adjusted video frame. If it is confirmed that the adjusted video frame contains a target event including an overexposure event or an underexposure event, the process returns to the step of adjusting the ambient light parameter to obtain a new ambient light parameter, adjusting the ambient light on the processed video frame based on the new ambient light parameter, and generating the adjusted video frame. If it is confirmed that the target event does not exist in the adjusted video frame, the adjusted video frame is further determined to be the target video frame. Generating the target video based on the processed video frames as described above is: The method according to any one of claims 4 to 7, comprising generating the target video based on the target video frame.

9. A video processing device, A first display module is configured to display instruction information including first instruction information for instructing the user to select a light and shadow rendering area and second instruction information for instructing the user to select lighting parameter information, in response to receiving a lighting processing request for a video to be processed. A determination module configured to determine, for a target video frame in the video to be processed, a target light and shadow rendering region selected by the user based on the first instruction information, and target lighting parameter information selected based on the second instruction information, wherein the target light and shadow rendering region and target lighting parameter information determined for the target video frame are used to perform lighting processing on a target video segment associated with the target video frame. A video processing device comprising the target light and shadow rendering region and a second display module configured to display a target video after performing lighting processing on the video to be processed based on the target lighting parameter information.

10. A computer device comprising a processor, memory, and a bus, wherein the memory stores machine-readable instructions that the processor can execute, and when the computer device is in operation, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the computer device performs the steps of the video processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the video processing method described in any one of claims 1 to 8 are performed.

Images