Video shooting real-time filming method and device

Abstract

The invention discloses a video shooting real-time filming method and device, and the method comprises the steps: dynamically obtaining a plurality of video frames which are shot in real time according to a preset time length, and recognizing the plurality of video frames to obtain a plurality of pieces of label information; obtaining cue word input information and scene information, and generating a plurality of cue words with the plurality of pieces of label information; according to the plurality of cue words, utilizing a preset large language model to generate a plurality of pieces of corresponding copywriting information according to a preset duration, and determining a target material based on a preset material library; and sequentially laying the copywriting information and the target material into the video to obtain a rendered video. According to a video shot in real time, a video frame is identified to determine label information, scene information and prompt word input information are combined, a preset large language model is called in real time to dynamically generate copywriting information fitting the video, a target material is recommended to render the video, and a user is helped to make and improve the video.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a method and device for real-time video shooting. Background Art

[0002] As filming equipment becomes more popular, intelligent and convenient, more and more users are recording the beautiful moments in their lives by shooting videos. For most users, it is difficult to shoot a high-quality and creative video. They lack professional video editing skills and creative inspiration, which results in the videos being monotonous, lacking coherence and appeal.

[0003] Existing optimizations for shooting products are often just optimizations on shooting photos, resulting in the release of photos that are mostly pictures or simple videos transformed from pictures. Therefore, there is an urgent need for a method that can help users easily shoot and produce videos in real time. Summary of the invention

[0004] In view of the above problems, embodiments of the present application are proposed to provide a method and device for real-time video shooting that overcomes the above problems or at least partially solves the above problems.

[0005] According to a first aspect of an embodiment of the present application, a method for real-time video shooting is provided, which includes:

[0006] Dynamically obtain multiple video frames shot in real time according to a preset duration, and identify the multiple video frames to obtain multiple label information;

[0007] Based on the voice information during shooting, the corresponding emotional information and text information are obtained;

[0008] Obtain prompt word input information and scene information, and generate multiple prompt words with multiple tag information;

[0009] According to the plurality of prompt words, a preset large language model is used to generate a plurality of corresponding copywriting information according to a preset duration, and a target material is determined based on a preset material library;

[0010] The copy information and target materials are laid into the video in sequence to obtain a rendered video.

[0011] Optionally, dynamically acquiring multiple video frames shot in real time according to a preset duration, and identifying the multiple video frames to obtain tag information further includes:

[0012] Based on the real-time captured video, multiple video frames are dynamically acquired according to a preset duration;

[0013] Use the preset labeling model to identify multiple video frames and obtain the recognition results and recognition weights;

[0014] According to the sorting from high to low according to the recognition weight, a preset number of recognition results that are sorted first are determined as label information; the label information includes picture description information of the video frame.

[0015] Optionally, obtaining prompt word input information and scene information, and generating prompt words with tag information further includes:

[0016] Obtaining prompt word input information based on input interface;

[0017] Identify the scene and obtain scene information including location information and / or weather information;

[0018] The prompt word input information, scene information and label information are generated according to the preset template, and prompt words are generated every preset time interval until the shooting is completed.

[0019] Optionally, according to the plurality of prompt words, using a preset large language model to generate a plurality of corresponding copywriting information according to a preset duration, and determining the target material based on a preset material library further includes:

[0020] According to the prompt words generated at each preset time interval, the preset large language model is used to generate corresponding copy information according to the preset time until the shooting is completed;

[0021] Using the preset large model, the material tags in the preset material library are matched with the copy information to obtain the target material; the target material includes transitions, filters, subtitle templates and / or music.

[0022] Optionally, laying the copy information and the target material into the video in sequence to obtain the rendered video further includes:

[0023] Lay out the captured video streams in sequence, use the filters in the target material to render the video streams, and add transitions in the target material based on the split-screen points;

[0024] Use the subtitle template to lay the generated text information to the video stream in sequence, and add the music in the target material to obtain the rendered video.

[0025] Optionally, before laying the copy information and the target material into the video in sequence to obtain the rendered video, the method further includes:

[0026] Identify the video stream shot in real time and determine the video stream to be adjusted; the video stream to be adjusted includes a video stream with jittery images and / or a video stream with incoherent scenes;

[0027] Perform cut and transition processing on the video stream to be adjusted.

[0028] Optionally, the method further comprises:

[0029] Based on the voice information during shooting, the corresponding emotional information and text information are obtained;

[0030] Acquiring prompt word input information and scene information, and generating multiple prompt words with multiple tag information further includes:

[0031] Obtain prompt word input information and scene information, and generate multiple prompt words with emotion information, text information and / or multiple tag information.

[0032] Optionally, obtaining corresponding emotion information and text information based on the voice information during shooting further includes:

[0033] Acquire voice information during shooting, use a preset emotion recognition model to recognize the voice information to obtain emotion information; and use a preset speech-to-text model to convert the voice information into text information.

[0034] Optionally, the method further comprises:

[0035] Display the rendered video for users to preview;

[0036] Edit the rendered video based on the editing interface.

[0037] According to a second aspect of an embodiment of the present application, a video shooting and real-time filming device is provided, comprising:

[0038] The label module is adapted to dynamically acquire multiple video frames shot in real time according to a preset duration, and identify the multiple video frames to obtain multiple label information;

[0039] A prompt word module, adapted to obtain prompt word input information and scene information, and generate multiple prompt words with multiple tag information;

[0040] The copywriting material module is adapted to generate corresponding multiple copywriting information according to a preset duration based on a plurality of prompt words using a preset large language model, and to determine a target material based on a preset material library;

[0041] The rendering module is suitable for laying the copy information and target materials into the video in sequence to obtain a rendered video.

[0042] According to a third aspect of an embodiment of the present application, there is provided a computing device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0043] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the above-mentioned video shooting real-time filming method.

[0044] According to a fourth aspect of an embodiment of the present application, a computer storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned video shooting and real-time filming method.

[0045] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising at least one executable instruction, wherein the executable instruction enables a processor to perform operations corresponding to the above-mentioned video shooting and real-time filming method.

[0046] According to the video shooting real-time filming method and device provided by the present application, it is possible to identify the video frame and determine the label information based on the real-time video, combine the scene information and the prompt word input information input by the user, call the preset large language model in real time, dynamically generate the copywriting information that fits the video, and intelligently recommend suitable target materials for rendering the video, thereby helping users to easily produce a complete video with creativity and coherence. Furthermore, the prompt word input information can be customized by the user to meet the user's personalized creation needs and generate a video that better meets the user's needs.

[0047] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0049] Figure 1 A flowchart of a method for real-time video recording according to an embodiment of the present application is shown;

[0050] Figure 2 A flowchart of a method for real-time video recording according to another embodiment of the present application is shown;

[0051] Figure 3 A schematic diagram of the structure of a video shooting and real-time filming device according to an embodiment of the present application is shown;

[0052] Figure 4 A schematic diagram of the structure of a computing device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0053] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0054] First, the terms involved in one or more embodiments of the present application are explained.

[0055] Video frame labeling: refers to the process of obtaining a certain frame of the video, calling the end model (or remote model) to identify the main body of the picture, and obtaining the picture description label;

[0056] Big Language Model: A model built on machine learning and natural language processing techniques that can be used to generate text;

[0057] Prompt: refers to the demand description input to the big language model. The big language model will generate copywriting based on the prompt word;

[0058] TTS: Text To Speech, from text to speech, uses convolutional neural networks to train feature models and convert speech into text;

[0059] Speech emotion recognition: refers to the technology that uses the acoustic features of a speech to identify the speaker's emotional state.

[0060] Figure 1 FIG. 4 shows a flow chart of a method for real-time video recording according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0061] Step S101 , dynamically acquiring multiple video frames shot in real time according to a preset duration, and identifying the multiple video frames to obtain multiple tag information.

[0062] Most of the existing shooting optimizations are aimed at optimizing photos, or optimizing videos using professional video optimization tools after shooting. This has high technical requirements on users and cannot help users optimize videos conveniently and quickly in real time.

[0063] Considering the above problems, this embodiment dynamically acquires multiple video frames in the real-time video according to a preset duration, such as dynamically acquiring a video frame every 5 seconds, and identifies each acquired video frame. When identifying, a picture recognition technology can be used to identify each object information contained in the video frame, and each object information is combined to describe the whole video frame. Starting from the whole video frame, the label information of each video frame is determined. Alternatively, when identifying the video frame, combined with the front and back order of each video frame, after acquiring the label information of the previous video frame, the label information of the previous video frame can be used to identify the current video frame. For example, if the label information obtained by the previous video frame recognition is the seaside, and a sand object appears in the current video frame, the label information of the current video frame can be determined to be the beach; if the label information obtained by the previous video frame recognition is the Gobi, and a sand object appears in the current video frame, the label information of the current video frame can be determined to be the desert, etc., which not only takes into account the correlation between the video frames before and after, but also ensures the accuracy of label information recognition.

[0064] Furthermore, the tag information finally obtained may be one or more, such as 2 or 3. When there are multiple recognition results obtained from the video frame, one or more recognition results with the highest recognition weight may be selected as tag information according to the recognition weight, so as to subsequently generate accurate prompt words based on accurate tag information, optimize the video, etc.

[0065] Step S102, obtaining prompt word input information and scene information, and generating multiple prompt words with multiple tag information.

[0066] An input interface can be provided to the user before or during shooting. The user can use the input interface to input prompt word input information, so that the user can customize a personalized video. The input interface can facilitate text input, voice input, gesture input, etc. for the information input into the input interface. For example, when inputting text, the corresponding text can be directly obtained; when inputting voice, the input voice can be converted into text through technologies such as TTS; when inputting gestures, the association between gestures and text can be pre-set. When receiving gestures, the gestures are converted into corresponding texts through the pre-set associations. When inputting prompt word input information, guidance information can also be provided to help users input. For example, if the guidance information is "Set video theme", users can enter personalized video themes according to the guidance information, so as to generate personalized prompt words later. The above is an example, which is set according to the implementation situation and is not limited here.

[0067] Scene information may include location information, weather information, etc., which can be obtained through geographic location positioning, obtaining weather information based on geographic location, obtaining real-time weather information through sensors, etc.

[0068] According to the obtained prompt word input information and scene information, it is combined with the multiple label information obtained by identifying multiple video frames to generate multiple prompt words, that is, corresponding prompt words are generated based on each dynamically acquired video frame. Prompt words can be used to preset a large language model so as to generate multiple copy information for the video based on the prompt words. Here, the label information of different video frames may be different, and the generated prompt words will also be different. Different copy information can be generated according to different prompt words. When generating prompt words, a preset prompt word template can be used to combine the prompt word input information, scene information, label information and prompt word template to obtain prompt words. The prompt word template can use different templates. For example, when generating the first prompt word, the prompt word template used can be: in xxx (location), weather xxx, help me generate a xxx (emotion word), including label 1, label 2, and the beginning of the story of xxx (prompt word input information entered by the user), with no more than xx words and no more than x sentences; when generating subsequent prompt words, the prompt word template used can be: help me continue to expand the story script along this theme, and the expanded keywords include label n+1, label n+2, with no more than xx words and no more than x sentences, until the end of the shooting. The above is an example, which is set according to the implementation situation and is not limited here. Furthermore, the use of prompt word templates can ensure that the length of the prompt word is not too long or too short, avoiding the prompt word being too long and affecting the output efficiency of the preset large language model, and also avoiding the prompt word being too short, the content being too simple, and the generated copy information being too simple.

[0069] Step S103 : generating corresponding multiple copywriting information according to the preset durations using a preset large language model based on the multiple prompt words, and determining target materials based on a preset material library.

[0070] After obtaining multiple prompt words, each prompt word can be input into a preset large language model. The preset large language model can adopt any large language model, which is not limited here. The preset large language model generates corresponding copy information according to the prompt word according to the preset duration. Here, the preset duration can be the same as the preset duration when the video frame is dynamically acquired. For example, every 5 seconds, a piece of copy information is generated according to the prompt word composed of the prompt word input information, scene information and label information. After an interval of 5 seconds, a piece of copy information is continued to be generated according to the new prompt word composed of the prompt word input information, scene information and new label information. The new copy information can be a continuation of the previous copy information, etc. By generating the corresponding copy information according to the preset duration until the shooting is completed, multiple copy information is generated for the video so as to add copy explanation to the video.

[0071] In addition to adding copy information, you can also determine the target material for the video based on the preset large language model based on the prompt words, combined with the various materials already stored in the preset material library. The target material may include transitions, filters, subtitle templates, music, etc. The target material can be used for video rendering to help users beautify and enrich the video content. Each material in the preset material library may include materials and corresponding material tags. When determining the target material, the preset large language model can select and determine the target material suitable for the video based on the prompt words and material tags. You can also select multiple target materials of different types, and use different target materials in different positions of the video to present different visual effects. This is not limited here.

[0072] Step S104, laying the copy information and target materials into the video in sequence to obtain a rendered video.

[0073] The video shot by the user can be cut and storyboarded, and the video track can be laid out. While laying the video track, the video track can be rendered using the text information and the target material. Specifically, based on the text information generated according to the preset duration, a subtitle track of the corresponding text can be generated, and the subtitle track can be laid out to the corresponding video position. The transitions in the target material can be inserted into the storyboard points of the video track, filters can be rendered in the video, and music can be added to the video as background music to increase the video atmosphere, etc. This helps users get real-time rendered videos after completing real-time shooting. It is convenient and fast, and does not require users to master professional video optimization techniques.

[0074] According to the real-time video shooting method provided by this application, the video frame can be identified to determine the label information based on the real-time video, and the preset large language model can be called in real time in combination with the scene information and the prompt word input information input by the user, and the copywriting information that fits the video can be dynamically generated. Suitable target materials are intelligently recommended for rendering the video, thereby helping users to easily produce complete videos with creativity and coherence. Furthermore, the prompt word input information can be customized by the user to meet the user's personalized creation needs and generate videos that better meet user needs.

[0075] Figure 2 FIG. 4 shows a flow chart of a method for real-time video recording according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0076] Step S201, based on the real-time captured video, dynamically obtain multiple video frames according to a preset duration.

[0077] This embodiment processes the video shot by the user in real time and converts the real-time video into a film in real time. Therefore, when the user shoots in real time, based on the real-time video, multiple video frames can be dynamically acquired according to a preset time length, such as dynamically acquiring one video frame every 5 seconds, so as to determine the user's shooting content and optimize the video based on the dynamically acquired video frames.

[0078] The video frames are dynamically acquired from the beginning to the end of shooting, and one video frame is acquired at each preset interval. The video optimization information can be determined more specifically according to the real-time changing video frame content.

[0079] Step S202, using a preset labeling model to identify multiple video frames, obtain recognition results and recognition weights, sort them from high to low according to the recognition weights, and determine a preset number of recognition results that are sorted first as label information.

[0080] After obtaining the video frame, the preset labeling model can be used to identify the obtained video frame. During the identification, the object information contained in the video frame can be identified, such as the sky, ocean, animals and plants, people, objects, fields and other object information, to obtain the recognition result. Among them, the recognition result reflects the picture description information of the video frame, which is a comprehensive description of the content contained in the picture of the video frame. The recognition result is also set with a corresponding recognition weight, which can be determined according to the proportion of the object in the video frame, the position in the video frame, the recognition accuracy, etc. The recognition weights are sorted from high to low, and the preset number of recognition results that are sorted first are selected as label information. The preset number can be 2, that is, the two recognition results with the highest recognition weights are selected as label information. The label information contains the picture description information of the video frame, reflecting the important object information in the video frame.

[0081] For example, if the first video frame of the acquired video is a seaside scene, the video frame is identified by using a picture tagging model (which can be obtained based on neural network pre-training). The input is the video frame, and the output result is the label information: such as the seaside (weight 90%), coconut grove (weight 80%). The two with the highest recognition weights are selected as the label information. Five seconds later, the second video frame is acquired from the video. The video frame is a yacht on the sea. The picture tagging model is used for feature recognition to obtain label information: yacht (weight 92%), sea (weight 77%). The video frame is acquired every 5 seconds for recognition to obtain the corresponding label information until the video shooting is completed. The obtained label information can be temporarily stored for subsequent prompt word generation.

[0082] Step S203, obtaining corresponding emotion information and text information based on the voice information during shooting.

[0083] When shooting, you can also use devices such as microphones to collect the user's voice information when shooting. For the voice information, you can use a preset emotion recognition model to recognize the voice information and obtain the emotional information contained in the voice information, such as happiness, sadness, etc. The preset emotion recognition model can be obtained by pre-training acoustic features based on a neural network, which is not limited here. The voice information is input into the preset emotion recognition model, and the corresponding emotional information is output. The emotional information reflects the different emotions of the user when shooting, and different renderings can be performed for the user based on different emotional information, such as enhancing the emotional expression in the video, making it easier for viewers to empathize and experience the emotions of the shooting user.

[0084] Furthermore, the preset speech-to-text model can be used to convert voice information into text information, such as using the TTS speech-to-text model to convert the voice into corresponding text information. The text information can be used for subsequent prompt word generation and subsequent subtitle track laying.

[0085] Step S204, obtaining prompt word input information and scene information, and generating multiple prompt words with emotion information, text information and / or multiple tag information.

[0086] Before or during shooting, an input interface can be set up, such as providing an input interface through a floating small window in the shooting interface, so that the shooting user can input prompt word input information based on the input interface. The prompt word input information can be the user's shooting theme or user-defined text, such as the prompt word input information can be "go to the beach with family", etc., to provide users with a personalized input interface.

[0087] When identifying a scene, the location information may be obtained through positioning, or through scene recognition, etc., without limitation. Based on the location information, the corresponding weather information may be obtained, or the current weather information may be determined by identifying the weather information contained in the video, thereby obtaining scene information including the location information and the weather information.

[0088] The prompt word input information, scene information and the emotional information and text information obtained by voice information recognition, as well as the label information obtained by video frame recognition can be used according to a preset template to generate prompt words at preset time intervals until the shooting is completed. That is, after dynamically obtaining the label information by video frame recognition at each preset time interval, the label information is temporarily stored, and the label information is combined with the prompt word input information, scene information, emotional information, text information, etc. to generate prompt words at each preset time interval. The above information can be used when generating, or all the information can be used to generate prompt words, which is not limited here. One or more preset templates can be pre-set. When generating prompt words, the same preset template can be used, or multiple different preset templates can be used to generate different prompt words, so as to obtain more colorful copy information later. The preset template provides a template for splicing various information, such as the preset template: in xxx (location information), weather xxx (weather information), help me generate a xxx (emotional information), including tag information 1, tag information 2, xxx (prompt word input information) story beginning, no more than xx words, no more than x sentences, or the preset template is: continue to expand the story script along this theme, expand the keywords including tag information n+1, tag information n+2, no more than xx words, no more than x sentences, etc. The above is an example, which is set according to the implementation situation and is not limited here.

[0089] The number of prompt words generated can be the same as the number of dynamically acquired video frames. Different prompt words can be generated for different video frames, which is convenient for writing different copy information based on different prompt words based on the integrity of the video, making the copy information more holistic and relevant.

[0090] Step S205 , based on the plurality of prompt words, a plurality of corresponding copywriting information is generated according to the preset duration using a preset large language model, and a target material is determined based on a preset material library.

[0091] After obtaining each prompt word, the preset large language model can be called in real time to generate corresponding copy information for each prompt word generated at a preset time interval until the shooting is completed, and multiple copy information corresponding to multiple prompt words is obtained. In this embodiment, after dynamically acquiring the video frame according to the preset time, the label information of the video frame is identified, and the prompt word is generated based on the label information, and then the preset large language model is used to generate the corresponding copy information according to the prompt word. All of the above can be generated according to the same preset time, which can ensure the continuity of the association between the video frame-label information-prompt word-copy information, etc.

[0092] In addition to using the preset big model to generate prompt words, the preset big model can also match each material label in the preset material library with the copy information to obtain the target material, so that the addition of the target material is more consistent with the copy information and the video itself.

[0093] The target material may include various materials such as transitions, filters, subtitle templates, music, etc. The specific type is not limited and can be set according to the implementation situation.

[0094] Step S206, identifying the video stream shot in real time, determining the video stream to be adjusted, and performing a cutting and transition process on the video stream to be adjusted.

[0095] Before rendering the video, considering that real-time shooting may have video streams such as jittery video streams and incoherent scene video streams that affect the overall video playback effect, they need to be adjusted. Therefore, the real-time video streams can be identified first to identify the video streams to be adjusted, including jittery video streams and incoherent scene video streams. The video streams to be adjusted are cut and transitioned, that is, the video streams are cut and polished by adding transitions to improve the coherence and rhythm of the video.

[0096] Specifically, the video stream can be checked in real time, and a certain frame can be used as a reference frame to analyze the motion vectors of each frame. When the motion trajectory offset is greater than a first preset value such as x1, it indicates that jitter has occurred, and the jitter start point is recorded. If it is greater than a second preset value such as x2, it indicates that a large screen transition has occurred, and the split screen points are recorded. The identification continues, and the video frames in which the jitter point interval is less than 1s are removed, the video frames in which the split screen point interval is less than 1s are removed, and transition special effects are inserted into the split screen points. The above is an example for explanation, and the specific setting can be based on the implementation situation and is not limited here.

[0097] Step S207, laying the text information and target material into the video in sequence to obtain a rendered video.

[0098] When rendering, the video stream obtained by shooting can be laid out as a video track, the filter in the target material can be used to render the video stream, and the filter track can be laid out. The transition in the target material can be added based on the split screen point, and the transition track can be laid out. The subtitle template obtained from the target material can be used to lay out the generated multiple text information in the video stream in sequence to complete the laying of the subtitle track. In addition to laying out the text information as a subtitle track, the text information converted from the voice information can also be laid out as a subtitle track to add user-personalized text information. The music in the target material can be laid out as a music track to finally obtain the rendered video. The rendered video is a rendering of the video shot by the user in real time. After the user finishes shooting, the rendered video can be provided to the user in real time, without the user having to process the shot video, which is convenient and fast.

[0099] Step S208, displaying the rendered video for user preview, and editing the rendered video based on the editing interface.

[0100] After shooting, the rendered video can be displayed to the user, so that the user can directly preview and view the rendering effect. Furthermore, based on the rendered video, considering that the user may have the need for further editing, an editing interface can be provided to the user, so that the user can further edit and adjust the rendered video based on the editing interface, such as modifying the subtitle style, adjusting the background music volume, etc. The user can make personalized settings according to the needs. The user can save the edited video or directly publish it to social media, video platforms, etc. for video display, which is not limited here.

[0101] It should be noted that the user information (including but not limited to user terminal information, user account information, etc.) and user data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0102] According to the real-time video shooting method provided by the present application, the video frames shot in real time are dynamically acquired, and the label information corresponding to the video frames is determined based on the video frames. The label information is combined with the current location, weather and other scene information, the user's customized prompt word input information, and the text information obtained by the user's real-time voice information recognition and the emotional information contained therein to generate prompt words that match the video, so that the preset large language model can be called in real time to generate matching text information for the video based on the prompt words. Compared with the existing fixed prompt words, the generated text information is more compatible with the video, and different text information can be presented according to the different video frames, avoiding adding a single board of text information to the video. The preset large language model can also match the materials in the preset material library using the generated text information, so that the target material matching the video can be recommended to render the video, greatly improving the rendering effect of the video.

[0103] Figure 3 FIG. 1 is a schematic diagram showing the structure of a real-time video recording device provided by an embodiment of the present application. Figure 3 As shown, the device comprises:

[0104] The tag module 310 is adapted to dynamically acquire multiple video frames shot in real time according to a preset duration, and identify the multiple video frames to obtain multiple tag information;

[0105] A prompt word module 320, adapted to obtain prompt word input information and scene information, and generate multiple prompt words with multiple tag information;

[0106] The copy material module 330 is adapted to generate corresponding multiple copy information according to a preset duration using a preset large language model according to a plurality of prompt words, and to determine a target material based on a preset material library;

[0107] The rendering module 340 is suitable for laying the copy information and the target material into the video in sequence to obtain a rendered video.

[0108] Optionally, the tag module 310 is further adapted to:

[0109] Based on the real-time captured video, multiple video frames are dynamically acquired according to a preset duration;

[0110] Use the preset labeling model to identify multiple video frames and obtain the recognition results and recognition weights;

[0111] According to the sorting from high to low according to the recognition weight, a preset number of recognition results that are sorted first are determined as label information; the label information includes picture description information of the video frame.

[0112] Optionally, the prompt word module 320 is further adapted to:

[0113] Obtaining prompt word input information based on input interface;

[0114] Identify the scene and obtain scene information including location information and / or weather information;

[0115] The prompt word input information, scene information and label information are generated according to the preset template, and prompt words are generated every preset time interval until the shooting is completed.

[0116] Optionally, the copywriting material module 330 is further adapted to:

[0117] According to the prompt words generated at each preset time interval, the preset large language model is used to generate corresponding copy information according to the preset time until the shooting is completed;

[0118] Using the preset large model, the material tags in the preset material library are matched with the copy information to obtain the target material; the target material includes transitions, filters, subtitle templates and / or music.

[0119] Optionally, the rendering module 340 is further adapted to:

[0120] Lay out the captured video streams in sequence, use the filters in the target material to render the video streams, and add transitions in the target material based on the split-screen points;

[0121] Use the subtitle template to lay the generated text information to the video stream in sequence, and add the music in the target material to obtain the rendered video.

[0122] Optionally, the device also includes: a video cutting module 350, suitable for identifying the video stream shot in real time and determining the video stream to be adjusted; the video stream to be adjusted includes a video stream with picture jitter and / or an incoherent scene video stream; and cutting and transitioning the video stream to be adjusted.

[0123] Optionally, the device further comprises: a voice recognition module 360 ​​adapted to obtain corresponding emotion information and text information based on the voice information during shooting;

[0124] The prompt word module 320 is further adapted to:

[0125] Obtain prompt word input information and scene information, and generate multiple prompt words with emotion information, text information and / or multiple tag information.

[0126] Optionally, the speech recognition module 360 ​​is further adapted to:

[0127] Acquire voice information during shooting, use a preset emotion recognition model to recognize the voice information to obtain emotion information; and use a preset speech-to-text model to convert the voice information into text information.

[0128] Optionally, the device further includes: a display preview module 370, adapted to display the rendered video for preview by the user; and to edit the rendered video based on an editing interface.

[0129] The description of each module above refers to the corresponding description in the method embodiment and will not be repeated here.

[0130] According to the real-time video shooting device provided by the present application, the video frame can be identified to determine the label information based on the real-time video, and the preset large language model can be called in real time in combination with the scene information and the prompt word input information input by the user, and the copywriting information that fits the video can be dynamically generated, and suitable target materials can be intelligently recommended for rendering the video, thereby helping users to easily produce a complete video with creativity and coherence. Furthermore, the prompt word input information can be customized by the user to meet the user's personalized creation needs and generate a video that better meets the user's needs.

[0131] The present application also provides a non-volatile computer storage medium, which stores at least one executable instruction, and the executable instruction can execute operations corresponding to the video shooting and real-time filming method in any of the above method embodiments.

[0132] The present application also provides a computer program product, which includes at least one executable instruction or computer program, and the executable instruction or computer program can enable a processor to execute operations corresponding to the video shooting and real-time filming method in any of the above method embodiments.

[0133] Figure 4 A schematic diagram of the structure of a computing device according to an embodiment of the present application is shown. The specific embodiment of the present application does not limit the specific implementation of the computing device.

[0134] like Figure 4 As shown, the computing device may include: a processor (processor) 402 , a communications interface (Communications Interface) 404 , a memory (memory) 406 , and a communication bus 408 .

[0135] in:

[0136] The processor 402 , the communication interface 404 , and the memory 406 communicate with each other via a communication bus 408 .

[0137] The communication interface 404 is used to communicate with other devices such as clients or other servers.

[0138] The processor 402 is used to execute the program 410, and specifically can execute the relevant steps in the above-mentioned video shooting real-time filming method embodiment.

[0139] Specifically, the program 410 may include program codes, which include computer operation instructions.

[0140] The processor 402 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the present application. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0141] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0142] The program 410 can be specifically used to enable the processor 402 to execute the video shooting real-time filming method in any of the above method embodiments. The specific implementation of each step in the program 410 can refer to the corresponding descriptions in the corresponding steps and units in the above video shooting real-time filming embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above-mentioned method embodiment, which will not be repeated here.

[0143] The algorithm or display provided here is not inherently related to any specific computer, virtual system or other device. Various general systems can also be used together with the teaching based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the application is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the application described here, and the above description of specific languages ​​is to disclose the preferred embodiment of the application.

[0144] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0145] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following intention: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the claims below, the inventive aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.

[0146] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0147] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0148] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components of the present application. The present application can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing a part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0149] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be constructed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present application may be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A method for real-time video shooting, comprising: Dynamically acquiring multiple video frames shot in real time according to a preset duration, and identifying the multiple video frames to obtain multiple tag information; Acquire prompt word input information and scene information, and generate multiple prompt words with the multiple tag information; According to the plurality of prompt words, a preset large language model is used to generate a plurality of corresponding copywriting information according to a preset duration, and a target material is determined based on a preset material library; The text information and the target material are sequentially laid into the video to obtain a rendered video.

2. The method according to claim 1, wherein: The dynamically acquiring multiple video frames shot in real time according to a preset duration, and identifying the multiple video frames to obtain tag information further includes: Based on the real-time captured video, multiple video frames are dynamically acquired according to a preset duration; Using a preset marking model to identify the multiple video frames, and obtaining a recognition result and a recognition weight; According to the sorting from high to low recognition weights, a preset number of recognition results that are sorted first are determined as label information; the label information includes picture description information of the video frame.

3. The method according to claim 1, wherein: The acquiring of prompt word input information and scene information and generating prompt words with the tag information further includes: Obtaining prompt word input information based on input interface; Identify the scene and obtain scene information including location information and / or weather information; The prompt word input information, the scene information and the tag information are generated according to a preset template at preset time intervals until the shooting is completed.

4. The method according to any one of claims 1 to 3, wherein: The step of generating corresponding multiple copywriting information according to the preset duration using a preset large language model based on the multiple prompt words, and determining the target material based on the preset material library further includes: According to the prompt words generated at each preset time interval, the preset large language model is used to generate corresponding copy information according to the preset time until the shooting is completed; Using the preset macro model, each material tag in the preset material library is matched with the text information to obtain the target material; the target material includes transitions, filters, subtitle templates and / or music.

5. The method according to any one of claims 1 to 4, wherein: The step of sequentially laying the text information and the target material into the video to obtain a rendered video further comprises: Laying out the captured video streams in sequence, rendering the video streams using filters in the target material, and adding transitions in the target material based on storyboard points; The generated text information is sequentially applied to the video stream using a subtitle template, and the music in the target material is added to obtain a rendered video.

6. The method according to any one of claims 1 to 5, wherein: Before sequentially laying the text information and the target material into the video to obtain a rendered video, the method further includes: Identifying a video stream shot in real time to determine a video stream to be adjusted; the video stream to be adjusted includes a video stream with jittery images and / or a video stream with incoherent scenes; The video stream to be adjusted is cut and transitioned.

7. The method according to claim 1, wherein: The method further comprises: Based on the voice information during shooting, the corresponding emotional information and text information are obtained; The acquiring of prompt word input information and scene information and generating multiple prompt words with the multiple tag information further includes: Obtain prompt word input information and scene information, and generate multiple prompt words with the emotion information, text information and / or the multiple tag information.

8. The method according to claim 7, wherein: The obtaining of corresponding emotion information and text information based on the voice information during shooting further includes: Acquire voice information during shooting, use a preset emotion recognition model to recognize the voice information to obtain emotion information; and use a preset speech-to-text model to convert the voice information into text information.

9. The method according to any one of claims 1 to 8, wherein: The method further comprises: Display the rendered video for users to preview; The rendered video is edited based on an editing interface.

10. A video shooting and real-time filming device, comprising: A label module, adapted to dynamically acquire a plurality of video frames shot in real time according to a preset duration, and identify the plurality of video frames to obtain a plurality of label information; A prompt word module, adapted to obtain prompt word input information and scene information, and generate multiple prompt words with the multiple tag information; A copywriting material module, adapted to generate corresponding multiple copywriting information according to the plurality of prompt words using a preset large language model and a preset duration, and to determine target materials based on a preset material library; The rendering module is suitable for laying the text information and the target material into the video in sequence to obtain a rendered video.

11. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the video shooting and real-time filming method according to any one of claims 1-9.

12. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and wherein the executable instruction enables a processor to execute operations corresponding to the video shooting and real-time filming method according to any one of claims 1 to 9.

13. A computer program product, comprising at least one executable instruction, wherein the executable instruction enables a processor to execute operations corresponding to the video shooting and real-time filming method according to any one of claims 1 to 9.

Images