Real-time rendering method for specified video frame in real-time recorded video and related device

By rendering video frames in real time during real-time video recording and then rendering transitional videos after each frame is displayed, the problem of screen stuttering and jumps during real-time rendering is solved, resulting in smooth video display.

CN119011941APending Publication Date: 2024-11-22BAIDU COM TIMES TECH (BEIJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411131996.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-16
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Existing technologies suffer from stuttering and abrupt transitions when rendering video frames in real-time during real-time video recording.

Method used

By judging in real time whether the video frames meet the preset selection requirements during the recording process, the video frames that meet the requirements are selected as the video frames to be rendered. After the previous frame is displayed, the rendering transition video is displayed continuously until the rendered video frame is completed, and then the display is switched to avoid screen stuttering.

Benefits of technology

It achieves smooth display of video frames, avoiding screen stuttering and abrupt transitions, and improving the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119011941A_ABST
    Figure CN119011941A_ABST
Patent Text Reader

Abstract

The invention provides a real-time rendering method for a specified video frame in a real-time recorded video and a related device, and relates to the technical field of artificial intelligence such as frame rendering, generative large language models, graph-to-graph models and the like. The method comprises the following steps: firstly, judging whether each video frame of a video generated in real time in a recording process meets a preset selection requirement or not, selecting the current video frame meeting the preset selection requirement as a to-be-rendered video frame, rendering the to-be-rendered video frame in real time, and after the previous frame of the to-be-rendered video frame is displayed, displaying the previous frame of the to-be-rendered video frame in real time. And connecting to realize rendering of the transition video until rendering is completed and a rendered video frame is obtained, and switching to display the rendered video frame. According to the scheme, the rendering transition video is connected and displayed at the appropriate time point to avoid image lagging caused by rendering of the video frame to be rendered, and the rendering transition video of the last round is completely displayed when the rendering transition video is switched back to be displayed after rendering, so that smooth connection display is guaranteed, and abrupt image jumping does not occur.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, specifically to the field of artificial intelligence technology such as frame rendering, generative large language models, and graph-to-graph models, and in particular to a real-time rendering method, device, electronic device, computer-readable storage medium, and computer program product for specifying a video frame in a real-time recorded video. Background Art

[0002] With the advancement of image processing technology and the enhancement of equipment performance, video image rendering has gradually shifted from pure post-production work to real-time work, that is, the video images captured by the shooting device in real time are rendered in real time and presented to the user, rather than post-rendering some of the video frames after the complete video is obtained after the shooting is completed.

[0003] However, there are many challenges in the actual operation of real-time rendering of certain video frames in real-time recorded video. Summary of the invention

[0004] The embodiments of the present disclosure provide a method, device, electronic device, computer-readable storage medium, and computer program product for real-time rendering of a specified video frame in a real-time recorded video.

[0005] In the first aspect, an embodiment of the present disclosure proposes a real-time rendering method for a specified video frame in a real-time recorded video, comprising: obtaining a frame sequence of a video in the recording process; selecting a current video frame in the frame sequence that meets a preset selection requirement as a video frame to be rendered, and rendering the video frame to be rendered in real time; after the previous video frame of the video frame to be rendered is displayed, displaying a rendered transition video determined based on a preset rendering transition method; in response to the completion of real-time rendering of the video frame to be rendered to obtain a rendered video frame, displaying the rendered video frame after the current rendered transition video is displayed.

[0006] In the second aspect, the embodiment of the present disclosure proposes a real-time rendering device for specifying video frames in a real-time recorded video, comprising: a real-time recorded video frame sequence acquisition unit, configured to acquire a frame sequence of a video in a recording process; a to-be-rendered video frame selection and real-time rendering unit, configured to select a current video frame in the frame sequence that meets a preset selection requirement as a to-be-rendered video frame, and to perform real-time rendering on the to-be-rendered video frame; a rendered transition video connection and display unit, configured to connect and display a rendered transition video determined based on a preset rendering transition method after the previous video frame of the to-be-rendered video frame is displayed; a rendered video frame connection and display unit, configured to obtain a rendered video frame in response to the completion of real-time rendering of the to-be-rendered video frame, and to connect and display the rendered video frame after the current rendered transition video is displayed.

[0007] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes, the method for real-time rendering of a specified video frame in a real-time recorded video as described in the first aspect can be implemented.

[0008] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, which are used to enable a computer to implement the real-time rendering method of a specified video frame in a real-time recorded video as described in the first aspect when executed.

[0009] In a fifth aspect, an embodiment of the present disclosure provides a computer program product comprising a computer program, which, when executed by a processor, can implement the steps of the real-time rendering method of a specified video frame in a real-time recorded video as described in the first aspect.

[0010] The real-time rendering scheme for specifying video frames in real-time recorded videos provided by the present disclosure first determines whether each video frame of the video generated in real time during the recording process meets the preset selection requirements, and selects the current video frame that meets the preset selection requirements as the video frame to be rendered, and renders it in real time, and after the previous frame of the video frame to be rendered is displayed, the rendered transition video determined based on the preset rendering transition method is connected until the rendered video frame is rendered, and then the rendered video frame is switched to be displayed. That is, this scheme avoids screen freezes caused by rendering the video frame to be rendered by connecting the rendered transition video at an appropriate time point, and completely displays the last round of rendered transition video when switching back to display the rendered video frame, thereby ensuring smooth connection display and no abrupt screen jumps.

[0011] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0013] Figure 1 is an exemplary system architecture in which the present disclosure may be applied;

[0014] Figure 2 A flowchart of a method for real-time rendering of a specified video frame in a real-time recorded video provided by an embodiment of the present disclosure;

[0015] Figure 3 A branch diagram of two different implementation methods for determining a video frame to be rendered provided in an embodiment of the present disclosure;

[0016] Figure 4 A branch schematic diagram of two different implementation methods for determining rendering transition videos provided by an embodiment of the present disclosure;

[0017] Figure 5 A flowchart of a method for completing real-time rendering of a video frame to be rendered by using a large graph-generated graph model provided in an embodiment of the present disclosure;

[0018] Figure 6 A flowchart of a latest video frame connection display mode and a method for processing undisplayed video frames provided by an embodiment of the present disclosure;

[0019] Figure 7 A structural block diagram of a real-time rendering device for a specified video frame in a real-time recorded video provided by an embodiment of the present disclosure;

[0020] Figure 8 A schematic diagram of the structure of an electronic device suitable for executing a real-time rendering method for a specified video frame in a real-time recorded video provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments in the present disclosure and the features in the embodiments may be combined with each other without conflict.

[0022] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0023] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the method, apparatus, electronic device, and computer-readable storage medium for real-time rendering of a specified video frame in a real-time recorded video of the present disclosure can be applied.

[0024] like Figure 1As shown, the system architecture 100 may include terminal devices 101, 102, 103 with camera functions, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0025] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. The terminal devices 101, 102, 103 and the server 105 can be installed with various applications for realizing information communication between the two, such as real-time rendering applications, live broadcast applications, instant messaging applications, etc.

[0026] The terminal devices 101, 102, 103 and the server 105 can be hardware or software. When the terminal devices 101, 102, 103 are hardware, they can be various electronic devices with at least a camera function, including but not limited to cameras, smart phones, tablet computers, laptop portable computers, desktop computers, etc.; when the terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above, which can be implemented as multiple software or software modules, or as a single software or software module, and no specific limitation is made here. When the server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server; when the server is software, it can be implemented as multiple software or software modules, or as a single software or software module, and no specific limitation is made here.

[0027] The server 105 can provide various services through various built-in applications. Taking a live broadcast application that can provide a real-time rendering service as an example, the server 105 can achieve the following effects when running the live broadcast application: first, a frame sequence of a video captured and recorded in real time by the terminal devices 101, 102, and 103 is received through the network 104; then, the current video frame in the frame sequence that meets the preset selection requirements is selected as the video frame to be rendered, and the video frame to be rendered is rendered in real time; next, after the previous video frame of the video frame to be rendered is displayed, a rendering transition video determined based on a preset rendering transition method is displayed in connection; finally, in response to the completion of the real-time rendering of the video frame to be rendered to obtain a rendered video frame, the rendered video frame is displayed in connection with the current rendering transition video after the display of the rendered video is completed.

[0028] Since the rendering of the video frames of the real-time recorded video needs to ensure real-time performance, it needs to occupy more computing resources and stronger computing power. Therefore, the real-time rendering method of the specified video frame in the real-time recorded video provided in the subsequent embodiments of the present disclosure is generally performed by the server 105 with stronger computing power and more computing resources. Correspondingly, the real-time rendering device of the specified video frame in the real-time recorded video is generally also set in the server 105. However, it should also be pointed out that when the terminal devices 101, 102, and 103 also have computing power and computing resources that meet the requirements, the terminal devices 101, 102, and 103 can also complete the above-mentioned various operations handed over to the server 105 through the live broadcast application installed thereon, and then output the same results as the server 105. Especially when there are multiple terminal devices with different computing capabilities at the same time, but the live broadcast application determines that the terminal device has stronger computing capabilities and more remaining computing resources, it can let the terminal device perform the above-mentioned operations, thereby appropriately reducing the computing pressure of the server 105. Correspondingly, the real-time rendering device of the specified video frame in the real-time recorded video can also be set in the terminal devices 101, 102, and 103. In this case, the exemplary system architecture 100 may also not include the server 105 and the network 104.

[0029] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0030] Please refer to Figure 2 , Figure 2 This is a flowchart of a method for real-time rendering of a specified video frame in a real-time recorded video provided by an embodiment of the present disclosure, wherein process 200 includes the following steps:

[0031] Step 201: Acquire a frame sequence of a video in the recording process;

[0032] This step is intended to be performed by the execution subject (e.g. Figure 1 The server 105 shown in FIG. 1 obtains the image capturing device (eg Figure 1 The terminal device shown, such as a camera or an intelligent terminal with a camera function, is a frame sequence of video generated during real-time shooting.

[0033] It should be understood that a video is composed of a series of continuous image frames, which are played at a certain frame rate (such as 24 frames or 30 frames per second) to form a continuous dynamic picture; and a video in the recording process is a video frame obtained by a technology that obtains a frame sequence while the video recording is in progress. The video recording process includes receiving a data stream from a camera or other video capture device and storing it as a video file. During this process, the above-mentioned execution entity needs to process and extract each video frame of the frame sequence in real time.

[0034] Specifically, the extraction of each video frame can be completed in the following manner:

[0035] Video stream parsing: The system needs to parse the video stream to determine the frame boundaries. The video stream can be in compressed format (such as H.264) or uncompressed format;

[0036] Decoding: If the video stream is compressed, decoding is required to restore the original image frame. The decoding process converts the compressed data back into a frame image;

[0037] Frame extraction: Extract specific frames from the decoded data. You may need to determine the extracted interval or frame at a specific time point.

[0038] At the specific technical implementation level, you can use a programming interface (API) to access the video stream and extract frames (such as OpenCV library or FFmpeg), use video processing software tools or functions provided by a development kit (SDK) to process and extract frames, etc.

[0039] Step 202: selecting a current video frame that meets a preset selection requirement in a frame sequence as a video frame to be rendered, and performing real-time rendering on the video frame to be rendered;

[0040] Based on step 201, this step aims to select the current video frame that meets the preset selection requirement in the frame sequence as the video frame to be rendered by the above-mentioned execution subject, and perform real-time rendering on the video frame to be rendered. The preset selection requirement is used to reflect the requirement information of which video frames are the target video frames when the above-mentioned execution subject expects to render, and the requirement can be expressed in various forms. Figure 3 The branch diagram provided exemplifies the two forms of selection requirements:

[0041] Method 1: The current video frame in the frame sequence that has the same actual recording time as the specified recording time recorded in the preset recording script is selected as the video frame to be rendered, wherein the preset recording script records at least one specified recording time. For example, the preset recording script records that the video frames at the 5th second, the 8th second and the 24th second in the actual recording process are to be selected as the video frames to be rendered, or it may also record that a video frame every 5 seconds is selected as the video frame to be rendered, or other video frame time interval forms.

[0042] Method 2: Select the current video frame in the frame sequence that has the picture elements required by the preset specified frame selection rule as the video frame to be rendered; wherein the preset specified frame selection rule includes a selection requirement for at least one picture element; the picture element may include at least one of the following: objects of a specified type (such as human body, animal, building, sculpture, etc.), objects in a specified position (animals in a lake, two buildings symmetrically distributed on the left and right, etc.), objects in a specified state (birds in flying state, people in crawling state, etc.), objects with a specified number (3 or more people, 5 or more ducks, etc.).

[0043] That is, different from the method one of pre-indicating which second's video frame will be selected as the video frame (that is, the method of pre-setting the number of seconds, most likely because the real-time recorded video is carried out in accordance with the pre-set video content script, so it is roughly possible to calculate or infer what state it should be in and what picture content can be presented at how many seconds), the method two is completely based on the pre-set selection rules, that is, it is highly unlikely to predict at which second or at which moment a video frame that meets the selection rules will appear. Therefore, by setting the selection rules according to expectations, as long as the rule is met, it means that the video frame has picture elements that meet the expectations, and therefore the video frame is selected as the video frame to be rendered in real time.

[0044] Of course, except Figure 3 In addition to the two selection methods shown, there may be other situations in different actual scenarios, which can be flexibly set according to actual conditions and are not listed here one by one.

[0045] Step 203: after the video frame preceding the video frame to be rendered is displayed, a rendering transition video determined based on a preset rendering transition mode is displayed in succession.

[0046] On the basis of step 202, this step aims at enabling the execution subject to display the rendering transition video determined based on the preset rendering transition mode after the previous video frame of the video frame to be rendered is displayed.

[0047] Figure 4The branch diagram provided exemplarily shows the presentation forms of the rendering transition video determined by the two rendering transition modes:

[0048] Method 1: Based on the preset material-adding rendering transition method, a static image or dynamic video consisting of preset materials representing the rendering transition intention is added to the video frame to be rendered, and determined as the rendering transition video.

[0049] That is, the rendering transition video provided by method 1 is a static image or dynamic video in the form of adding materials to the video frame to be rendered, and the added materials are used to represent the rendering transition intention, such as some artistic words or stickers that represent that rendering is in progress and that the rendering results can be expected. This method uses the video frame to be rendered as the base image, so the connection between the rendered video frame and the rendered video frame will be smoother and less variable.

[0050] Method 2: Based on a preset universal rendering transition method, a static image or dynamic video composed of a preset universal rendering transition material is determined as a rendering transition video.

[0051] Different from the first method, the second method pre-configures a universal rendering transition material for all rendering transition deadlines. It does not use the actual video frame to be rendered as the background image, but is only used to indicate that the current rendering waiting process is in progress, so that users can know the actual situation and have expectations for the rendering results.

[0052] Of course, except Figure 4 In addition to the two selection methods shown, there may be other situations in different actual scenarios, which can be flexibly set according to actual conditions and are not listed here one by one.

[0053] S204: In response to the completion of real-time rendering of the video frame to be rendered to obtain a rendered video frame, the rendered video frame is displayed after the current rendered transition video is displayed.

[0054] On the basis of step 203, this step is based on the fact that the video frame to be rendered has completed real-time rendering and obtained the rendered video frame, and is intended to display the rendered video frame in conjunction with the current rendered transition video after it is displayed.

[0055] Since the rendered transition video can be a single static image or a dynamic video with a certain duration and different pictures at different times, any time can be regarded as the display is completed when it is a single static image, and when it is a dynamic video with a certain duration and different pictures at different times, if the video frame is switched to display immediately after the rendering is obtained (that is, the rendered video frame is switched to display immediately when all the picture contents of the rendered transition video have not been played), the viewer may be presented with an abrupt picture change due to the immediate interruption of the rendered transition video, which reduces the viewing experience. Therefore, even if the rendered transition video still has a certain length of time to play after the rendering is obtained at the moment of rendering the rendered video frame, it can also be selected to wait for all the picture contents of the rendered transition video to be displayed. Taking the rendered transition video as a dynamic video lasting 3 seconds as an example, if the entire rendering process takes 5 seconds, the rendered transition video will be played twice in a loop, that is, it will be switched to display the rendered video frame 1 second after the rendering is completed. As for which implementation method to choose, it can be flexibly selected according to different situations in the actual application scenario, and no specific limitation is made here.

[0056] The real-time rendering method of a specified video frame in a real-time recorded video provided by the disclosed embodiment first determines whether each video frame of the video generated in real time during the recording process meets the preset selection requirements, and selects the current video frame that meets the preset selection requirements as the video frame to be rendered, and renders it in real time, and after the previous frame of the video frame to be rendered is displayed, the rendering transition video determined based on the preset rendering transition method is connected until the rendered video frame is obtained after the rendering is completed, and then the rendered video frame is switched to be displayed. That is, this solution avoids the screen freeze caused by the rendering of the video frame to be rendered by connecting the rendering transition video at an appropriate time point, and completely displays the last round of rendering transition video when switching back to display the rendered video frame, thereby ensuring smooth connection display and no abrupt screen jumps.

[0057] In order to deepen the understanding of how to perform real-time rendering of the video frame to be rendered, this embodiment also Figure 5 A solution is provided for implementing real-time rendering of a video frame to be rendered by using a large graph-generated graph model, wherein process 500 includes the following steps:

[0058] Step 501: Determine a target rendering mode for a video frame to be rendered;

[0059] An implementation method including but not limited to:

[0060] When there are multiple preset alternative rendering methods, an alternative interface for each alternative rendering method is presented, and then the position selection information for the alternative interface is received from the user, and finally the alternative rendering method corresponding to the position selection information is determined as the target rendering method. That is, this implementation method gives the right of choice to the user who needs to reflect his or her expectations, and of course it can also be given to other objects that can make biased choices, such as a generative large language model or digital human with relevant capabilities, etc.

[0061] Step 502: Generate rendering hint words corresponding to the target rendering mode;

[0062] On the basis of step 501, this step aims to generate a rendering prompt word (ie, prompt) corresponding to the target rendering mode by the above-mentioned execution subject, so as to inform the large model user of the rendering mode desired.

[0063] Step 503: calling a preset image generation model to perform real-time rendering on the input video frame to be rendered under the constraints of the rendering prompt words.

[0064] On the basis of step 502, this step aims to cause the above-mentioned execution subject to call the preset graph-generated graph model to perform real-time rendering on the input video frame to be rendered under the constraints of the rendering prompt words.

[0065] Considering that video editing and special effects production often require real-time rendering of video frames to apply special effects, filters or other image processing operations, these rendering operations usually need to be combined with pre-designed models and algorithms to achieve high-quality results.

[0066] The image generation model refers to a pre-trained image generation model, such as a deep learning model or a neural network model, which is used to process the input video frames and generate corresponding rendering effects; the video frames to be rendered represent the input video frames that need to be processed and rendered; the constraints of the rendering prompt words: may be user-defined rendering requirements, special effect parameters or guidance words, which are used to control the details and style of the rendering process.

[0067] The real-time rendering process is that the above-mentioned execution entity calls the pre-trained image generation model through the interface, and passes the video frames to be rendered and the rendering prompt words as input information to the image generation model for processing. That is, the image generation model performs real-time rendering processing on the video frames under constraints, optimizes the rendering output according to the prompt words, and ensures that the rendering effect meets the user's requirements.

[0068] Based on the above-mentioned description of any embodiment of how to determine the video frame to be rendered and how to use the rendered transition video for connection display before the rendering of the video frame to be rendered is completed, Figure 6It further shows a latest video frame connection display method and a processing solution for undisplayed video frames provided by the applicant, wherein the process 600 includes the following steps:

[0069] Step 601: After the rendered video frame is displayed, the latest video frame of the frame sequence is displayed;

[0070] That is, from the moment the video frame to be rendered is determined to the moment the rendered video frame is rendered, the newly recorded video frame will no longer be displayed because the rendering transition video is displayed in succession. Only the latest video frame of the frame sequence will be displayed in succession after the rendered video frame is displayed.

[0071] Step 602: Obtain a sequence of undisplayed video frames newly recorded during the display rendering transition video;

[0072] Step 603: In response to the presence of undisplayed video frames that meet the preset selection requirement in the undisplayed video frame sequence, determining the undisplayed video frames that meet the preset selection requirement as new video frames to be rendered;

[0073] Steps 602 and 603 are intended to enable the above-mentioned execution subject to obtain a sequence of undisplayed video frames newly recorded during the display rendering transition video, and to determine whether the sequence of undisplayed video frames contains undisplayed video frames that also meet the preset selection requirements, and if there are undisplayed video frames that meet the preset selection requirements in the sequence of undisplayed video frames, the above-mentioned execution subject will determine the undisplayed video frames that meet the preset selection requirements as new video frames to be rendered.

[0074] Step 604: Render the new video frame to be rendered, and display the rendered new video frame in an interstitial form.

[0075] Based on step 603, this step aims to have the above-mentioned execution subject render the new video frame to be rendered, and because the actual display time of the undisplayed video frame has passed and the rendering result of the previous video frame to be displayed has been displayed first, the rendering result of the new video frame to be rendered can only be displayed in the form of an interlude later.

[0076] An implementation method including but not limited to:

[0077] First, the new video frame to be rendered is rendered to obtain a new rendered video frame; then, the target insertion time is determined based on a preset insertion timing selection method; finally, the new rendered video frame is displayed at the target insertion time in the form of an additional insertion prompt, and the insertion prompt includes the actual recording time of the new video to be rendered and / or a playback reminder.

[0078] In order to meet the need to render some video frames in real time during the real-time recording of the video and present a loading (waiting) loop state during the rendering process, this embodiment also provides a specific implementation solution in this scenario:

[0079] 1. Before recording: read the configuration file information, including the start time and duration of the overlay loading, and the afx of the overlay screen effects (usually including the sound effects directly produced by the hand and foot movements of the characters in the screen, such as opening and closing doors, punching and kicking, pressing buttons, etc.) material video files;

[0080] 2. Recording: When the user clicks to start recording, a time starting point start is recorded. When rendering each frame, it is determined whether the current time current-start reaches the loading start time. If it reaches the loading start time, the loading state is prepared and the loading start time loadingStart is recorded:

[0081] a) The camera layer image is fixed. The operation of fixing the frame is to stop updating the camera data to the current canvas and keep the texture of the last drawing result;

[0082] b) When rendering each frame, determine whether the current time current-loadingStart reaches the duration of loading. If so, seek the afx video file with superimposed screen effects to the loading start time and reset loadingStart to achieve the effect of looping for a specific duration;

[0083] c) Simultaneously start parallel tasks to transmit fixed-frame images to the server, and request in turn to obtain the results of large-model raw images;

[0084] d) Until the raw image result is obtained, the loading state is sent to stop;

[0085] e) After the rendering thread receives the stop loading status, the mark stops looping. When the loading is finished, it stops loading and records the last loading time, loadingEnd.

[0086] f) Update the raw image to the screen and automatically stop recording.

[0087] 3. After recording: Split the recorded video by the earliest loadingStart and loadingEnd and splice them into a video result that only contains one loading screen.

[0088] Further references Figure 7As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a real-time rendering device for a specified video frame in a real-time recorded video. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0089] like Figure 7 As shown, the real-time rendering device 700 for specifying a video frame in a real-time recorded video of this embodiment may include: a real-time recorded video frame sequence acquisition unit 701, a to-be-rendered video frame selection and real-time rendering unit 702, a rendered transition video connection display unit 703, and a rendered video frame connection display unit 704. The real-time recorded video frame sequence acquisition unit 701 is configured to acquire a frame sequence of a video in the recording process; the to-be-rendered video frame selection and real-time rendering unit 702 is configured to select a current video frame in the frame sequence that meets a preset selection requirement as a to-be-rendered video frame, and perform real-time rendering on the to-be-rendered video frame; the rendered transition video connection display unit 703 is configured to connect and display a rendered transition video determined based on a preset rendering transition mode after the previous video frame of the to-be-rendered video frame is displayed; the rendered video frame connection display unit 704 is configured to obtain a rendered video frame in response to the to-be-rendered video frame completing real-time rendering, and connect and display the rendered video frame after the current rendered transition video is displayed.

[0090] In this embodiment, in the real-time rendering device 700 for specifying a video frame in a real-time recorded video, the specific processing and technical effects brought by the real-time recorded video frame sequence acquisition unit 701, the video frame selection and real-time rendering unit 702, the rendering transition video connection display unit 703, and the rendered video frame connection display unit 704 can be referred to in detail. Figure 2 The relevant descriptions of steps 201 - 204 in the corresponding embodiment are not repeated here.

[0091] In some optional implementations of this embodiment, the to-be-rendered video frame selection and real-time rendering unit 702 may include a to-be-rendered video frame selection subunit configured to select a current video frame that meets a preset selection requirement in a frame sequence as a to-be-rendered video frame, and the to-be-rendered video frame selection subunit may be further configured to:

[0092] A current video frame in a frame sequence having an actual recording time that is the same as a specified recording time recorded in a preset recording script is selected as a video frame to be rendered; wherein the preset recording script records at least one specified recording time.

[0093] In some optional implementations of this embodiment, the to-be-rendered video frame selection and real-time rendering unit 702 may include a to-be-rendered video frame selection subunit configured to select a current video frame that meets a preset selection requirement in a frame sequence as a to-be-rendered video frame, and the to-be-rendered video frame selection subunit may be further configured to:

[0094] A current video frame having a screen element that meets the requirements of a preset specified frame selection rule in a frame sequence is selected as a video frame to be rendered; wherein the preset specified frame selection rule includes a selection requirement for at least one screen element.

[0095] In some optional implementations of this embodiment, the screen element includes at least one of the following:

[0096] Objects of a specified type, objects in a specified location, objects in a specified state, objects in a specified number.

[0097] In some optional implementations of this embodiment, the to-be-rendered video frame selection and real-time rendering unit 702 may include a real-time rendering subunit configured to perform real-time rendering on the to-be-rendered video frame, and the real-time rendering subunit may include:

[0098] A target rendering mode determination module is configured to determine a target rendering mode for a video frame to be rendered;

[0099] A rendering hint word generation module is configured to generate rendering hint words corresponding to a target rendering mode;

[0100] The large model rendering module is configured to call a preset large model of the image generation to perform real-time rendering on the input video frame to be rendered under the constraints of the rendering prompt words.

[0101] In some optional implementations of this embodiment, the target rendering mode determination module may be further configured to:

[0102] In response to the presence of multiple preset alternative rendering modes, presenting an alternative interface for each alternative rendering mode;

[0103] Receiving location selection information for an alternative interface;

[0104] The alternative rendering mode corresponding to the position selection information is determined as the target rendering mode.

[0105] In some optional implementations of this embodiment, the real-time rendering apparatus 700 for a specified video frame in a real-time recorded video may further include: a rendering transition video determining unit configured to determine a rendering transition video based on a preset rendering transition mode, and the rendering transition video determining unit may be further configured to:

[0106] Based on the preset material-adding rendering transition method, a static image or dynamic video consisting of preset materials representing the rendering transition intention is added to the video frame to be rendered and determined as the rendering transition video.

[0107] In some optional implementations of this embodiment, the real-time rendering apparatus 700 for a specified video frame in a real-time recorded video may further include: a rendering transition video determining unit configured to determine a rendering transition video based on a preset rendering transition mode, and the rendering transition video determining unit may be further configured to:

[0108] Based on the preset universal rendering transition mode, a static image or dynamic video composed of the preset universal rendering transition material is determined as a rendering transition video.

[0109] In some optional implementations of this embodiment, the real-time rendering device 700 for a specified video frame in a real-time recorded video may further include:

[0110] The cyclic repetition display unit is configured to cyclically and repeatedly display the rendered transition video before a rendered video frame is obtained in response to the rendered transition video being a non-static image with a preset duration.

[0111] In some optional implementations of this embodiment, the real-time rendering device 700 for a specified video frame in a real-time recorded video may further include:

[0112] The latest video frame connection display unit is configured to connect and display the latest video frame of the frame sequence after the rendered video frame is displayed.

[0113] In some optional implementations of this embodiment, the real-time rendering device 700 for a specified video frame in a real-time recorded video may further include:

[0114] The non-displayed video frame sequence acquisition unit is configured to acquire the non-displayed video frame sequence newly recorded during the display rendering transition video;

[0115] A new to-be-rendered video frame determining unit, configured to, in response to the presence of an undisplayed video frame that meets a preset selection requirement in the undisplayed video frame sequence, determine the undisplayed video frame that meets the preset selection requirement as a new to-be-rendered video frame;

[0116] The newly rendered video frame interleaving display unit is configured to render the new video frame to be rendered, and display the rendered newly rendered video frame in an interleaving form.

[0117] In some optional implementations of this embodiment, the newly rendered video frame insertion display unit may be further configured to:

[0118] Rendering the new video frame to be rendered to obtain a new rendered video frame after rendering is completed;

[0119] Determine the target insertion time based on a preset insertion timing selection method;

[0120] The newly rendered video frame is displayed at the target insertion time in the form of an additional insertion prompt; wherein the insertion prompt includes the actual recording time and / or playback reminder of the new video to be rendered.

[0121] This embodiment exists as a device embodiment corresponding to the above method embodiment. The real-time rendering device for specifying video frames in real-time recorded video provided by this embodiment first determines whether each video frame of the video generated in real time during the recording process meets the preset selection requirements, and selects the current video frame that meets the preset selection requirements as the video frame to be rendered, and renders it in real time, and after the previous frame of the video frame to be rendered is displayed, the rendering transition video determined based on the preset rendering transition method is connected until the rendering is completed to obtain the rendered video frame, and then the rendered video frame is switched to be displayed. That is, this scheme avoids the screen freeze caused by the rendering of the video frame to be rendered by connecting the rendering transition video at an appropriate time point, and completely displays the last round of rendering transition video when switching back to display the rendered video frame, thereby ensuring smooth connection display and no abrupt screen jumps.

[0122] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes, it can implement the real-time rendering method of a specified video frame in a real-time recorded video described in any of the above embodiments.

[0123] According to an embodiment of the present disclosure, the present disclosure further provides a readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the real-time rendering method of a specified video frame in a real-time recorded video described in any of the above embodiments when executed.

[0124] According to an embodiment of the present disclosure, the present disclosure further provides a computer program product, which, when executed by a processor, can implement the real-time rendering method of a specified video frame in a real-time recorded video described in any of the above embodiments.

[0125] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0126] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0127] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0128] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as a real-time rendering method for a specified video frame in a real-time recorded video. For example, in some embodiments, a real-time rendering method for a specified video frame in a real-time recorded video may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the real-time rendering method for a specified video frame in the real-time recorded video described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured in any other appropriate manner (eg, by means of firmware) to execute a real-time rendering method for a specified video frame in a real-time recorded video.

[0129] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0130] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0131] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0132] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0133] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0134] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0135] According to the technical solution of the disclosed embodiment, for each video frame of the video generated in real time during the recording process, a judgment is first made as to whether it meets the preset selection requirements, and the current video frame that meets the preset selection requirements is selected as the video frame to be rendered, and is rendered in real time, and after the previous frame of the video frame to be rendered is displayed, the rendering transition video determined based on the preset rendering transition method is connected until the rendered video frame is obtained after the rendering is completed, and then the rendered video frame is switched to be displayed. That is, this solution avoids the screen freeze caused by the rendering of the video frame to be rendered by connecting the rendering transition video at an appropriate time point, and completely displays the last round of rendering transition video when switching back to display the rendered video frame, thereby ensuring smooth connection display and no abrupt screen jumps.

[0136] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0137] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A real-time rendering method for a specified video frame in a real-time recorded video, comprising: Get the frame sequence of the video in the process of recording; Selecting a current video frame in the frame sequence that meets a preset selection requirement as a video frame to be rendered, and performing real-time rendering on the video frame to be rendered; After the video frame preceding the to-be-rendered video frame is displayed, a rendering transition video determined based on a preset rendering transition mode is displayed in succession; In response to the completion of real-time rendering of the to-be-rendered video frame, a rendered video frame is obtained, and the rendered video frame is displayed after the current rendered transition video is displayed.

2. The method according to claim 1, wherein: The step of selecting the current video frame in the frame sequence that meets the preset selection requirement as the video frame to be rendered includes: The current video frame in the frame sequence having the same actual recording time as the specified recording time recorded in the preset recording script is selected as the video frame to be rendered; wherein the preset recording script records at least one of the specified recording times.

3. The method according to claim 1, wherein: The step of selecting the current video frame in the frame sequence that meets the preset selection requirement as the video frame to be rendered includes: The current video frame in the frame sequence that has the screen element required by the preset specified frame selection rule is selected as the video frame to be rendered; wherein the preset specified frame selection rule includes a selection requirement for at least one screen element.

4. The method according to claim 3, wherein: The screen elements include at least one of the following: Objects of a specified type, objects in a specified location, objects in a specified state, objects in a specified number.

5. The method according to claim 1, wherein: The real-time rendering of the to-be-rendered video frame comprises: Determining a target rendering mode for the video frame to be rendered; Generate a rendering hint word corresponding to the target rendering mode; The preset graph generation model is called to perform real-time rendering on the input video frame to be rendered under the constraint of the rendering prompt word.

6. The method according to claim 5, wherein: The determining of a target rendering mode for the video frame to be rendered includes: In response to the presence of multiple preset alternative rendering modes, presenting an alternative interface for each of the alternative rendering modes; Receiving location selection information for the alternative interface; The candidate rendering mode corresponding to the position selection information is determined as the target rendering mode.

7. The method according to claim 1, wherein: The rendering transition video is determined based on the preset rendering transition method, including: Based on the preset material-adding rendering transition method, a static image or dynamic video consisting of preset materials representing the rendering transition intention is added to the video frame to be rendered, and is determined as the rendering transition video.

8. The method according to claim 1, wherein: The rendering transition video is determined based on the preset rendering transition method, including: Based on a preset universal rendering transition mode, a static image or dynamic video composed of preset universal rendering transition materials is determined as the rendering transition video.

9. The method according to claim 1, further comprising: In response to the rendered transition video being a non-static image with a preset duration, the rendered transition video is repeatedly displayed in a loop before the rendered video frame is obtained.

10. The method according to any one of claims 1 to 9, further comprising: After the rendered video frame is displayed, the latest video frame of the frame sequence is displayed continuously.

11. The method according to claim 10, further comprising: Acquire a sequence of undisplayed video frames newly recorded during display of the rendered transition video; In response to the presence of an undisplayed video frame that meets the preset selection requirement in the undisplayed video frame sequence, determining the undisplayed video frame that meets the preset selection requirement as a new video frame to be rendered; The new video frame to be rendered is rendered, and the rendered new video frame is displayed in an interstitial form.

12. The method according to claim 11, wherein: The step of rendering the new video frame to be rendered and displaying the rendered new video frame in an interstitial form includes: Rendering the new video frame to be rendered to obtain a new rendered video frame; Determine the target insertion time based on a preset insertion timing selection method; The newly rendered video frame is displayed at the target insertion time in the form of an additional insertion prompt; wherein the insertion prompt includes the actual recording time of the new video to be rendered and / or a playback reminder.

13. A real-time rendering device for a specified video frame in a real-time recorded video, comprising: A real-time recording video frame sequence acquisition unit is configured to acquire a frame sequence of a video in the recording process; The unit for selecting and rendering video frames to be rendered in real time is configured to select the current video frames in the frame sequence that meet the preset selection requirements as the video frames to be rendered, and perform real-time rendering on the video frames to be rendered; A rendering transition video connection display unit is configured to connect and display a rendering transition video determined based on a preset rendering transition mode after a video frame preceding the video frame to be rendered is displayed; The rendered video frame connection display unit is configured to obtain a rendered video frame in response to the completion of real-time rendering of the to-be-rendered video frame, and to connect and display the rendered video frame after the current rendered transition video is displayed.

14. The device according to claim 13, wherein: The to-be-rendered video frame selection and real-time rendering unit includes a to-be-rendered video frame selection subunit configured to select a current video frame that meets a preset selection requirement in the frame sequence as a to-be-rendered video frame, and the to-be-rendered video frame selection subunit is further configured to: The current video frame in the frame sequence having the same actual recording time as the specified recording time recorded in the preset recording script is selected as the video frame to be rendered; wherein the preset recording script records at least one of the specified recording times.

15. The device according to claim 13, wherein: The to-be-rendered video frame selection and real-time rendering unit includes a to-be-rendered video frame selection subunit configured to select a current video frame that meets a preset selection requirement in the frame sequence as a to-be-rendered video frame, and the to-be-rendered video frame selection subunit is further configured to: The current video frame in the frame sequence that has the screen element required by the preset specified frame selection rule is selected as the video frame to be rendered; wherein the preset specified frame selection rule includes a selection requirement for at least one screen element.

16. The device according to claim 15, wherein: The screen elements include at least one of the following: Objects of a specified type, objects in a specified location, objects in a specified state, objects in a specified number.

17. The device according to claim 13, wherein: The to-be-rendered video frame selection and real-time rendering unit includes a real-time rendering subunit configured to perform real-time rendering on the to-be-rendered video frame, and the real-time rendering subunit includes: A target rendering mode determination module is configured to determine a target rendering mode for the video frame to be rendered; A rendering prompt word generation module, configured to generate a rendering prompt word corresponding to the target rendering mode; The large model rendering module is configured to call a preset large model of the image generation to perform real-time rendering on the input video frame to be rendered under the constraint of the rendering prompt word.

18. The device according to claim 17, wherein: The target rendering mode determination module is further configured to: In response to the presence of multiple preset alternative rendering modes, presenting an alternative interface for each of the alternative rendering modes; Receiving location selection information for the alternative interface; The candidate rendering mode corresponding to the position selection information is determined as the target rendering mode.

19. The apparatus according to claim 13, further comprising: A rendering transition video determining unit configured to determine a rendering transition video based on a preset rendering transition mode, wherein the rendering transition video determining unit is further configured to: Based on the preset material-adding rendering transition method, a static image or dynamic video consisting of preset materials representing the rendering transition intention is added to the video frame to be rendered, and is determined as the rendering transition video.

20. The apparatus of claim 13, further comprising: A rendering transition video determining unit configured to determine a rendering transition video based on a preset rendering transition mode, wherein the rendering transition video determining unit is further configured to: Based on a preset universal rendering transition mode, a static image or dynamic video composed of preset universal rendering transition materials is determined as the rendering transition video.

21. The method of claim 13, further comprising: The cyclic repetition display unit is configured to cyclically and repeatedly display the rendered transition video before the rendered video frame is obtained in response to the rendered transition video being a non-static image with a preset duration.

22. The apparatus according to any one of claims 13 to 21, further comprising: The latest video frame connection display unit is configured to connect and display the latest video frame of the frame sequence after the rendered video frame is displayed.

23. The apparatus according to claim 22, further comprising: a non-displayed video frame sequence acquisition unit, configured to acquire a non-displayed video frame sequence newly recorded during display of the rendered transition video; A new to-be-rendered video frame determining unit, configured to, in response to the presence of an undisplayed video frame that meets the preset selection requirement in the undisplayed video frame sequence, determine the undisplayed video frame that meets the preset selection requirement as a new to-be-rendered video frame; The newly rendered video frame interleaving display unit is configured to render the new video frame to be rendered, and display the rendered newly rendered video frame in an interleaving manner.

24. The device according to claim 23, wherein: The newly rendered video frame insertion display unit is further configured to: Rendering the new video frame to be rendered to obtain a new rendered video frame; Determine the target insertion time based on a preset insertion timing selection method; The newly rendered video frame is displayed at the target insertion time in the form of an additional insertion prompt; wherein the insertion prompt includes the actual recording time of the new video to be rendered and / or a playback reminder.

25. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the real-time rendering method of a specified video frame in a real-time recorded video described in any one of claims 1-12.

26. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the real-time rendering method of a specified video frame in a real-time recorded video according to any one of claims 1 to 12.

27. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method for real-time rendering of a specified video frame in a real-time recorded video according to any one of claims 1 to 12.