Video recording method and device, storage medium and program product

Abstract

The embodiment of the invention provides a video recording method and device, a storage medium and a program product. In the embodiment of the invention, the face state information of the current video frame is recognized and judged in real time in the recording process, and the image storage event is not executed and recording is performed again under the condition that the requirement is not met. Different video types correspond to different prompt information and re-recording schemes, so that re-recording is facilitated according to the adaptive prompt information and recording scheme, if the target video clip is a speaking type video, the whole target video clip needs to be re-recorded, and if the target video clip is a silent type video, the current video frame needs to be re-recorded. And when the current frame image meets the recording requirement, executing an image storage event to store the current video frame. The clear visual feedback is provided in the recording process, the user is guided to adaptively adjust the face state in the recording process so as to meet the video recording requirement, whether the image storage event is executed or not is automatically controlled, and the recording effect of the video frame sequence is improved.

Description

Technical Field

[0001] The present application relates to the field of computer processing technology, and in particular to a video recording method, device, storage medium and program product. Background Art

[0002] With the rapid development of computer technology, the functions of APPs have become increasingly powerful, and recording videos through smart APPs instead of traditional shooting equipment has become mainstream, such as recording videos of people.

[0003] However, clear visual feedback is usually not provided during the recording of a person's video, such as whether the person's face is located in the center, whether there is a side face, and whether the face size meets the standard. This will make it difficult for the recorded video to meet the expected shooting requirements in terms of composition and person's posture, and the visual effect of the recorded video is poor. Summary of the invention

[0004] Multiple aspects of the present application provide a video recording method, device, storage medium and program product, which are used to provide clear visual feedback during the video recording process, guide the user to make adaptive adjustments in terms of facial status and other aspects during the recording process to meet the recording requirements, and improve the overall visual effect and recording efficiency of the recorded video frames.

[0005] An embodiment of the present application provides a video recording method, the method comprising: in response to a video recording operation initiated for a target user, starting to record a video frame sequence containing at least a face area performed by the target user based on script content, the video frame sequence comprising multiple video frames; in the process of recording the video frame sequence, performing face state recognition for the current video frame being recorded and judging whether the recognized face state information meets the recording requirement information; if the face recognition information of the current video frame does not meet the recording requirement information, judging whether the target video segment to which the current video frame belongs is a speech-type video segment, the target video segment comprising the current video frame and a specified number of video frames recorded before the current video frame; if the target video segment is a speech-type video segment, outputting a first prompt information, the first prompt information being used to prompt the target user that the target video segment needs to be re-recorded and to guide the target user to adjust the face recording state to re-record the target video segment; and The first deduction process after the prompt information adjusts the face recording state is re-recorded until a plurality of first re-recorded video frames that meet the recording requirement information corresponding to the target video clip are obtained, and an image storage event is executed for each first re-recorded video frame to store the plurality of first re-recorded video frames in the target video file; if the target video clip is not a speech-type video clip, a second prompt information is output, and the second prompt information is used to prompt the target user to re-record the current video frame and guide the target user to adjust the face recording state to re-record the current video frame; and the second deduction process after the target user adjusts the face recording state according to the second prompt information is re-recorded until a second re-recorded video frame that meets the recording requirement information corresponding to the current video frame is obtained, and an image storage event is executed for the second re-recorded video frame to store the second re-recorded video frame in the target video file; in response to the recording end operation, the execution of the image storage event is terminated to obtain a complete video frame sequence.

[0006] An embodiment of the present application also provides an electronic device, including: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement each step in the above method.

[0007] An embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor implements each step in the above method.

[0008] In the embodiment of the present application, the technical solution provided by the above embodiment of the present application, in the process of recording a video frame sequence, for the current video frame being recorded, that is, when recording each video frame, face status recognition is performed and it is determined whether the recognized face status information meets the recording requirement information, and re-recording is performed if the requirement is not met. By performing status recognition while recording the current video frame, it is possible to timely identify whether the current video frame meets the recording requirement, and timely re-record if the recording requirement information is not met, which not only improves the visual effect of each video frame, but also improves the overall visual effect and recording efficiency of the video frame sequence. Furthermore, before re-recording, it is also necessary to determine the type of target video clip to which the current video frame belongs. Different video types correspond to different prompt information and re-recording schemes, so as to facilitate re-recording according to the adapted prompt information and recording scheme. For example, if the target video clip to which the current video frame belongs is a speech-type video clip, a first prompt information is output to prompt the target user that the target video clip needs to be re-recorded and guide the target user to adjust the face recording state to re-record the target video clip; if the target video clip to which the current video frame belongs is not a speech-type video clip, a second prompt information is output to prompt the target user that the current video frame needs to be re-recorded and guide the target user to adjust the face recording state to re-record the current video frame. This provides clear visual feedback during the recording process, guides the user to make adaptive adjustments in terms of face state and other aspects during the recording process to meet the video recording requirements, and further improves the visual effect of the video frame sequence. In addition, whether the current video frame meets the recording requirement information is a control condition for whether to execute the image storage event, that is, when the current video frame meets the recording requirement information, the image storage event will be automatically executed to store the current video frame in the target video file. When the current video frame does not meet the recording requirement information, the image storage event will be automatically terminated to complete the recording task for the video frame sequence. That is, whether the current video frame meets the recording requirement information is used to control whether the current video frame is stored in the target video file, thereby avoiding video frames that do not meet the recording requirements (noise) from interfering with the video frame sequence and thus interfering with the quality and visual effect of the final generated video material. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0010] Figure 1 A flowchart of a video recording method provided for an exemplary embodiment of the present application;

[0011] Figure 2 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0013] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0014] In the field of video recording, existing character recording lacks standardized prompts. For example, clear visual feedback is usually not provided during the character recording process, such as whether the face is located in the center, whether there is a side face, and whether the face size meets the standard. This may cause the recorded video to be difficult to meet the expected shooting requirements in terms of composition and character posture, increasing the possibility of shooting failure or poor visual effects of the recorded video.

[0015] In addition, existing character recording methods rely on specific backgrounds. For example, recording green screen video material requires a special green screen background and professional-level shooting equipment, which limits the popularization and application of such technology and increases shooting costs and the complexity of scene layout.

[0016] In addition, in traditional green screen shooting, the recorded video materials usually need to undergo tedious post-processing, and the post-processing requirements of video materials are high. For example, it is necessary to use matting technology to separate the characters from the background and generate materials with transparent backgrounds, which further increases the workload and production costs.

[0017] In view of the above technical problems, in an embodiment of the present application, in the process of recording a video frame sequence, for the current video frame being recorded, that is, when recording each video frame, face status recognition will be performed and it will be determined whether the recognized face status information meets the recording requirement information. If it does not meet the requirement, the recording will be re-performed. By performing status recognition while recording the current video frame, it is possible to timely identify whether the current video frame meets the recording requirement. If it does not meet the recording requirement information, the recording will be re-performed in time, which not only improves the visual effect of each video frame, but also improves the overall visual effect and recording efficiency of the video frame sequence. Furthermore, before re-recording, it is also necessary to determine the type of target video clip to which the current video frame belongs. Different video types correspond to different prompt information and re-recording schemes, so as to facilitate re-recording according to the adapted prompt information and recording scheme. For example, if the target video clip to which the current video frame belongs is a speech-type video clip, a first prompt information is output to prompt the target user that the target video clip needs to be re-recorded and guide the target user to adjust the face recording state to re-record the target video clip; if the target video clip to which the current video frame belongs is not a speech-type video clip, a second prompt information is output to prompt the target user that the current video frame needs to be re-recorded and guide the target user to adjust the face recording state to re-record the current video frame. This provides clear visual feedback during the recording process, guides the user to make adaptive adjustments in terms of face state and other aspects during the recording process to meet the video recording requirements, and further improves the visual effect of the video frame sequence. In addition, whether the current video frame meets the recording requirement information is a control condition for whether to execute the image storage event, that is, when the current video frame meets the recording requirement information, the image storage event will be automatically executed to store the current video frame in the target video file. When the current video frame does not meet the recording requirement information, the image storage event will be automatically terminated to complete the recording task for the video frame sequence. That is, whether the current video frame meets the recording requirement information is used to control whether the current video frame is stored in the target video file, thereby avoiding video frames that do not meet the recording requirements (noise) from interfering with the video frame sequence and thus interfering with the quality and visual effect of the final generated video material.

[0018] A solution provided by an embodiment of the present application is described in detail below in conjunction with the accompanying drawings.

[0019] Figure 1 The flowchart of the video recording method provided in the embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0020] 101. In response to a video recording operation initiated for a target user, start recording a video frame sequence that at least includes a face region and is interpreted based on script content, where the video frame sequence includes a plurality of video frames;

[0021] 102. During the process of recording the video frame sequence, face recognition is performed on the current video frame being recorded and it is determined whether the recognized face state information meets the recording requirement information;

[0022] 103. If the face recognition information of the current video frame does not meet the recording requirement information, determine whether the target video segment to which the current video frame belongs is a speech-type video segment, where the target video segment includes the current video frame and a specified number of video frames recorded before the current video frame;

[0023] 104. If the target video clip is a speech-type video clip, output a first prompt message, the first prompt message being used to prompt the target user to re-record the target video clip and guide the target user to adjust the face recording state to re-record the target video clip; and re-record the first interpretation process after the target user adjusts the face recording state according to the first prompt message, until a plurality of first re-recorded video frames that meet the recording requirement information corresponding to the target video clip are obtained, and an image storage event is executed for each first re-recorded video frame to store the plurality of first re-recorded video frames into the target video file;

[0024] 105. If the target video clip is not a speech-type video clip, output a second prompt message, the second prompt message being used to prompt the target user to re-record the current video frame and guide the target user to adjust the face recording state to re-record the current video frame; and re-record the second interpretation process after the target user adjusts the face recording state according to the second prompt message, until a second re-recorded video frame that meets the recording requirement information corresponding to the current video frame is obtained, and an image storage event is executed for the second re-recorded video frame to store the second re-recorded video frame into the target video file;

[0025] 106. In response to the recording end operation, end the execution of the image storage event to obtain a complete video frame sequence.

[0026] This application does not limit the execution subject of the above-mentioned video recording method. For example, the execution subject can be an intelligent video recording device or an APP installed on a terminal device, a small program or a web page that depends on the APP to run, etc. Among them, the APP, the small program that depends on the APP to run, and the web page itself can have a video recording module, and the video can be recorded through the video recording module; or, the video can be recorded by calling the video recording device of the terminal device; or, the video can be recorded by a third-party video recording device connected to the terminal device, which is not limited in this embodiment. Regardless of the form in which the execution subject is implemented, the device for video recording can be referred to as a video recorder in the following embodiments. In addition, this embodiment does not specifically limit the implementation form of the terminal device. For example, the terminal device can be a smart handheld device, such as a smart phone, a tablet computer, a laptop or a desktop computer, etc.; for another example, the terminal device can also be a smart wearable device, such as a smart watch, a smart bracelet, etc.; for another example, the terminal device can also be various smart home appliances with display screens, such as smart TVs, smart large screens or smart robots, etc.

[0027] In the embodiments of the present application, some video materials can be recorded according to actual needs. These video materials have multiple uses. For example, video materials can be used as live broadcast materials in the digital human live broadcast room, interview materials in the interview room, video materials for short video releases, and advertising materials, etc. Among them, some video materials require human participation. As the protagonist of the video material, the protagonist can be called the target user. The following embodiments of the present application focus on the recording of video materials with the participation of target users. The video material to be recorded corresponds to the script content, and the target user can perform interpretation based on the script content, that is, the target user is the person being recorded who plays the target role. This embodiment does not limit the role type of the target role, but depends on the script content and the script type to which the script content belongs. Among them, the script content refers to the content that the target user refers to when recording the target user. The script content includes but is not limited to: multiple interpretation stages and the interpretation content of each interpretation stage, and the interpretation content corresponding to different interpretation stages is different. Among them, the interpretation stage includes but is not limited to: a speech stage and a silent stage, the silent stage includes but is not limited to: a smiling stage and a serious stage, the interpretation content corresponding to the speech stage includes but is not limited to: content information narrated by the target user and expression description information used to describe the facial expression of the target user when speaking, the interpretation content corresponding to the smiling stage includes but is not limited to: description information used to describe smiling expressions, and the interpretation content corresponding to the serious stage includes but is not limited to: description information used to describe serious expressions. This embodiment does not limit the script type. For example, the script type can be a live broadcast script, an interview script, a tutor script, and a housekeeping script. Correspondingly, the target role can be an anchor role, an interviewer role, a tutor role, a housekeeping staff role, and so on. Specifically, when the script type is a live broadcast script, the corresponding target role is the anchor role; when the script type is an interview script, the corresponding target role is the interviewer role; when the script type is a tutor script, the corresponding target role is a tutor; when the script type is a housekeeping staff role, the corresponding target role is a housekeeping staff role, and so on.

[0028] In an embodiment of the present application, in response to a video recording operation initiated for a target user, a video frame sequence containing at least a face region performed by the target user based on the script content is recorded to obtain an initial video material based on the video frame sequence, wherein the video frame sequence includes multiple video frames.

[0029] This embodiment does not limit the object that initiates the video recording operation. For example, the object that initiates the video recording operation may be a staff member or a target user, but is not limited thereto. Specifically, the execution subject may have a graphical user interface, and a video recording control may be displayed on the graphical user interface. In response to a video recording request initiated for the video recording control, a video recorder is called to start recording a sequence of video frames that at least contain a face area and are interpreted by the target user based on the script content. Alternatively, the execution subject has a voice control function, and in response to a voice instruction to start a recording operation, a video recorder is called to start recording a sequence of video frames that at least contain a face area and are interpreted by the target user based on the script content, and so on. For another example, a trigger event for initiating a video recording operation is set in the execution subject, and the trigger event is set with a triggering time. When the triggering time arrives, a video recording operation for the target target user will be actively initiated, and in response to the initiation of the video recording operation, a video recorder is called to start recording a sequence of video frames that at least contain a face area and are interpreted based on the script content. For example, a specific time for the trigger may be set, and when the time arrives, a video recording operation will be actively initiated, but is not limited thereto. In addition, the current video frame may also be displayed on the graphical user interface.

[0030] In an embodiment of the present application, in order to improve the recording quality and visual effect of the video frame sequence, recording requirement information is set for the recording of the video frame sequence. The recording requirement information refers to reference information used to screen the recorded video frame sequence. The recording requirement information at least includes: a deformation threshold that the facial deformation features contained in the current video frame need to meet and a distance difference threshold that needs to be met between the target user and the video recorder.

[0031] Based on this, in the process of recording a sequence of video frames, face state recognition can be performed for the current video frame being recorded and it can be determined whether the recognized face state information meets the recording requirement information. That is, for the current video frame recorded at each recording moment, the face state information of the current video frame can be recognized in real time, so as to determine whether the recording requirement is met based on the face state information of the current video frame, so as to determine whether re-recording is required according to the judgment result. Among them, the face state information includes but is not limited to: facial features, facial deformation features, the position area of ​​the face in each video frame, the position of the facial features in the video frame, the tilt direction and angle of the face, the facial features include but are not limited to: eyebrows, eyes, nose, mouth, ears, facial deformation features include but are not limited to: the degree of deformation of the facial features of the target user relative to the serious expression when making facial expressions (such as the opening degree of the mouth, the eyes), and the tilt direction of the face includes but is not limited to: tilting upward, tilting downward, tilting to the left, tilting to the right, tilting to the upper left and tilting to the lower right. For the specific implementation method of determining whether the recognized face state information meets the recording requirement information, please refer to the relevant description of the following embodiment, which will not be repeated here.

[0032] Among them, the artificial intelligence AI model for face recognition can be used to identify the face state information of the current video frame. The artificial intelligence AI model can be obtained by training the initial artificial intelligence AI model based on the face state sample data of various fields. The initial artificial intelligence AI model has a simple function of identifying face state information, but its recognition accuracy is poor. In this regard, the initial artificial intelligence AI model and a large amount of face state sample data involved in various fields in the past can be obtained, and the initial artificial intelligence model can be trained using the face state sample data involved in various fields in the past to obtain an artificial intelligence AI model suitable for this application, which can be called a face recognition model to improve the accuracy of identifying face state information. Face recognition models include but are not limited to: ArcFace model (Additive Angle Margin Loss for Deep Face Recognition), TopoFR model (A Closer Look at Topology Alignment on Face Recognition), FaceNet model (A Unified Embedding for Face Recognition and Classification), and the like.

[0033] Specifically, a face recognition model is used to identify face status information of video frames, including: inputting the current video frame into the face recognition model for semantic segmentation to obtain a binary image containing facial features; obtaining a local image of the facial features based on the target current video frame and the binary image, the local image being a partial image in the target current video frame; inputting the local image into the feature extraction network layer in the face recognition model to extract initial facial features and segment the facial features, so as to obtain facial features, facial deformation features, the position area of ​​the face in each video frame, the position of the facial features in the video frame, and the tilt direction and angle of the face.

[0034] Similarly, the other models involved in the following embodiments may also be artificial intelligence AI models with corresponding functions, and the artificial intelligence AI models with corresponding functions may also be obtained by training the initial artificial intelligence model based on relevant sample data in various fields, which will not be repeated here.

[0035] In an embodiment of the present application, during the recording process, if the face recognition information of the current video frame recorded does not meet the recording requirements, it is necessary to re-record to improve the quality and visual effect of the video frame. It should be noted that the current video frame corresponds to a corresponding video clip, called a target video clip, and the target video clip includes the current video frame and a specified number of video frames recorded before the current video frame. For different types of video clips, the corresponding re-recording schemes may also be different. For example, in the case where the target video frame is a speech-type video clip, the entire target video clip can be re-recorded to avoid the mouth and other parts of the two video frames before and after the re-recording cannot be smoothly connected, so as to reduce the problem of unsmooth images between video frames in the target video clip; in the case where the current video frame is a silent video clip, only the current video frame can be re-recorded. For different types of video clips, different re-recording schemes are provided, and an adapted re-recording scheme can be provided for each type of video clip, which can further improve the recording quality and visual effect of the video frame.

[0036] In order to further improve the recording quality and visual effect of the video frame sequence, prompt information is also set for the recording of the video frame sequence. The prompt information includes at least: first prompt information and second prompt information, the first prompt information is used to prompt the target user to re-record the target video clip and guide the target user to adjust the face recording state to re-record the target video clip, and the second prompt information is used to prompt the target user to re-record the current video frame and guide the target user to adjust the face recording state to re-record the current video frame. The setting of the prompt information can enable the user to know in time whether it is the entire target video clip or the current video frame that needs to be re-recorded, so that the target user can adapt to the re-recording work in time; in addition, the prompt information adaptively guides the target user to adjust the face recording state, which can improve the recording efficiency, the accuracy of the video frame and the smoothness between each video frame. It should be noted that the prompt information can be displayed in the graphical user interface in the form of text for the target user to view in real time. Specifically, the prompt information can be displayed in the edge area of ​​the graphical user interface in the form of a pop-up window; or, the prompt information can also be played to the target user in the form of voice, etc., which is not limited in this embodiment.

[0037] It should be noted that the recording of a video frame sequence includes the following two processes: recording of video frames and storage of video frames. The recording of video frames is only an intermediate process. After recording any video frame, if the video frame is not stored, it is equivalent to that the video frame does not really exist, and it is impossible to generate the initial video material according to the video frame sequence. In this regard, an image storage event can be set, and the image storage event is used to store the recorded video frames.

[0038] It should be noted that, for the image storage event, it can be understood that whether the current video frame meets the recording requirement information is the control condition for whether to execute the image storage event, that is, when the current video frame meets the recording requirement information, the image storage event will be automatically executed to store the current video frame in the target video file. When the current video frame does not meet the recording requirement information, the execution of the image storage event will be automatically terminated to complete the recording task for the video frame sequence.

[0039] Based on this, the re-recording scheme in the embodiments of the present application includes at least the following two schemes.

[0040] Solution 1: If the target video clip is a speech-type video clip, the first prompt information is output to prompt the target user to re-record the target video clip and guide the target user to adjust the face recording state to re-record the target video clip. For example, the first prompt information may be "re-record the target video clip, move the face a certain distance in a certain direction or rotate a certain angle in a certain direction". Further, the first interpretation process after the target user adjusts the face recording state according to the first prompt information is re-recorded until a plurality of first re-recorded video frames that meet the recording requirement information corresponding to the target video clip are obtained. Further, an image storage event is executed for each first re-recorded video frame, and the image storage event is used to store the video frame, so as to store the plurality of first re-recorded video frames into the target video file. In the case that the target video clip is a speech-type video clip, re-recording the target video clip can avoid the problem that the target video clip is not smooth enough due to the large difference in the mouth features of the video frames shot before and after the interruption, thereby improving the visual effect corresponding to the target video clip. For the detailed implementation of each step in this solution, please refer to the relevant description of the following embodiment, which will not be repeated here.

[0041] It should be noted that during the recording of the first interpretation process, the facial state recognition information will continue to be identified and judged whether it meets the recording requirements. If the current video frame does not meet the recording requirements again, the re-recording will continue until multiple first re-recorded video frames that meet the recording requirements corresponding to the target video clip are obtained.

[0042] It should also be noted that the target user can adjust the face state according to the first prompt information. During the face state adjustment process, it may not be possible to adjust to a state that meets the recording requirements in one go, and it is necessary to adjust step by step. During the adjustment process, the face being adjusted will be recognized in real time, and the face state information will be judged in real time whether it meets the recording requirements. In addition, according to the degree of adjustment of the face state, the first prompt information will be updated in real time to more accurately guide the target user to adjust the face state. For example, the first prompt information guides the target user's face to rotate 10 degrees to the left and rear. During the rotation process of the target user, if it has rotated 5 degrees to the left and rear, the prompt information will be dynamically updated to "the face continues to rotate 5 degrees to the left and rear" until the face state recognition information meets the recording requirements.

[0043] Solution 2: If the target video clip is not a speech-type video clip, a second prompt message is output to prompt the target user that the current video frame needs to be re-recorded and guide the target user to adjust the face recording state to re-record the current video frame. Furthermore, the second interpretation process after the target user adjusts the face recording state according to the second prompt message is re-recorded until a second re-recorded video frame that meets the recording requirement information corresponding to the current video frame is obtained. Furthermore, an image storage event is executed for the second re-recorded video frame to store the second re-recorded video frame in the target video file. For the detailed implementation of each step in this solution, please refer to the relevant description of the above and following embodiments, which will not be repeated here.

[0044] Furthermore, after the target user completes recording of a video frame sequence that at least includes a face area based on the script content, in response to a recording end operation, the image storage event is terminated to obtain a complete video frame sequence.

[0045] This embodiment does not limit the object that initiates the end recording operation. For example, the object that initiates the end recording operation may be a staff member or a target user, but is not limited to this. Specifically, the execution subject may have a graphical user interface, and an end recording control may be displayed on the graphical user interface. In response to an end recording request initiated for the end recording control, the execution of the image storage event is stopped. Alternatively, the execution subject has a voice control function, and in response to a voice instruction for the end recording operation initiated for the target user, the execution of the image storage event is stopped, and so on. For another example, a trigger event for initiating an end recording operation is set in the execution subject, and the trigger event is set with a triggering time. When the triggering time arrives, the end recording operation will be actively initiated, and the execution of the image storage event will be stopped. For example, a specific time for the trigger can be set, and when the time arrives, the operation of stopping the execution of the image storage event will be actively initiated, but it is not limited to this.

[0046] The technical solution provided by the above-mentioned embodiment of the present application, in the process of recording a video frame sequence, performs face status recognition for the current video frame being recorded, that is, when recording each video frame, and determines whether the recognized face status information meets the recording requirement information, and re-records if it does not meet the requirement. Performing status recognition while recording the current video frame can timely identify whether the current video frame meets the recording requirement, and timely re-records if it does not meet the recording requirement information, which not only improves the visual effect of each video frame, but also improves the overall visual effect and recording efficiency of the video frame sequence. Furthermore, before re-recording, it is also necessary to determine the type of target video clip to which the current video frame belongs. Different video types correspond to different prompt information and re-recording schemes, so as to facilitate re-recording according to the adapted prompt information and recording scheme. For example, if the target video clip to which the current video frame belongs is a speech-type video clip, a first prompt information is output to prompt the target user that the target video clip needs to be re-recorded and guide the target user to adjust the face recording state to re-record the target video clip; if the target video clip to which the current video frame belongs is not a speech-type video clip, a second prompt information is output to prompt the target user that the current video frame needs to be re-recorded and guide the target user to adjust the face recording state to re-record the current video frame. This provides clear visual feedback during the recording process, guides the user to make adaptive adjustments in terms of face state and other aspects during the recording process to meet the video recording requirements, and further improves the visual effect of the video frame sequence. In addition, whether the current video frame meets the recording requirement information is a control condition for whether to execute the image storage event, that is, when the current video frame meets the recording requirement information, the image storage event will be automatically executed to store the current video frame in the target video file. When the current video frame does not meet the recording requirement information, the image storage event will be automatically terminated to complete the recording task for the video frame sequence. That is, whether the current video frame meets the recording requirement information is used to control whether the current video frame is stored in the target video file, thereby avoiding video frames that do not meet the recording requirements (noise) from interfering with the video frame sequence and thus interfering with the quality and visual effect of the final generated video material.

[0047] In an embodiment of the present application, facial state recognition is performed on the current video frame being recorded and it is determined whether the recognized facial state information meets the recording requirement information, including: obtaining script content, inputting the script content into an information reasoning model to map the script content to standard facial deformation features of each video frame in the video frame sequence and a standard distance between the target user and the video recorder; inputting the current video frame into a facial information recognition model, identifying the facial feature information of the current video frame, and obtaining the first facial deformation features of the current frame image; and collecting the facial deformation features of the target user and the video recorder when recording the current video frame. the first distance between the devices; the standard deformation feature, the standard distance, the first facial feature deformation feature, the first distance, the deformation difference threshold and the distance difference threshold are input into the information comparison model; based on the first facial feature deformation feature and the standard facial feature deformation feature, it is determined whether the first facial feature deformation feature meets the deformation threshold; based on the first distance and the standard distance, it is determined whether the first distance meets the distance difference threshold; when both judgment results are yes, it is determined that the recognized face state information meets the recording requirement information; when any judgment result is no, it is determined that the recognized face state information does not meet the recording requirement information.

[0048] Optionally, the information inference model includes information inference network layers corresponding to the facial features, namely, eyebrow deformation feature inference network layer, eye deformation feature inference network layer, nose deformation feature inference network layer, mouth deformation feature inference network layer, and ear deformation feature inference network layer. Accordingly, the script content is input into the information inference model to map the script content to the standard facial feature deformation features of each video frame in the video frame sequence and the standard distance between the target user and the video recorder, including: inputting the content information narrated by the target user contained in the script content and / or the expression description information used to describe the facial expression of the target user when speaking into the eyebrow deformation feature inference network layer, eye deformation feature inference network layer, nose deformation feature inference network layer, mouth deformation feature inference network layer, and ear deformation feature inference network layer, using each deformation inference network layer to infer the corresponding deformation features to obtain the deformation features of the facial features.

[0049] Among them, the content information narrated by the target user contained in the script content and / or the expression description information used to describe the facial expression of the target user when speaking are respectively input into the mouth deformation feature inference network layer, and the mouth deformation inference network layer is used to infer the deformation features of the mouth to obtain the deformation features of the mouth, including: extracting the tone feature vector corresponding to each character in the content information narrated by the target user and extracting the expression feature vector contained in the expression description information; performing weighted summation on the tone feature vector corresponding to each character and the expression feature vector contained in the expression description information to obtain multiple weighted feature vectors after weighted summation, each weighted feature vector is used to represent the comprehensive deformation of the mouth of the target user in the process of interpreting the script content, so as to obtain the standard deformation features of the mouth.

[0050] Similarly, the standard deformation features of eyebrows, eyes, ears and noses can be inferred. For details, please refer to the relevant description of the reasoning process of the standard deformation features of the mouth, which will not be repeated here.

[0051] Furthermore, the script content may also include description information of the swing of the target user's facial features during the recording process. For example, in a live broadcast scene, the host is required to wear and introduce a pair of earrings, and the host is required to fold the ear forward 45 degrees with his hands, so that users watching the live broadcast can clearly see the earrings displayed by the host. The description information of the swing of the facial features included in the script content is "fold the ear forward 45 degrees with your hands", and accordingly, the deformation features of the ear are inferred using the ear deformation inference network layer to obtain the deformation features of the ear, including: first inferring the visual morphology picture corresponding to the ear according to the content of "fold the ear forward 45 degrees with your hands"; extracting the features of the ear after folding the ear forward 45 degrees with your hands from the ear morphology picture; and using the inferred features as the deformation features of the ear. This is only exemplary content and does not limit the present application.

[0052] Optionally, the information comparison model includes a deformation feature comparison network layer and a distance comparison network layer. Based on this, in order to improve the recognition accuracy of facial status information and the accuracy of prompt information, the standard deformation feature, standard distance, the first facial feature deformation feature, the first distance, the deformation difference threshold and the distance difference threshold can be input into the information comparison model, and based on the first facial feature deformation feature and the standard facial feature deformation feature, it is determined whether the first facial feature deformation feature meets the deformation threshold, and based on the first distance and the standard distance, it is determined whether the first distance meets the distance difference threshold, including: inputting the standard deformation feature, the first facial feature deformation feature, the deformation difference threshold and the distance difference threshold into the deformation feature comparison network layer, calculating the deformation difference between the first facial feature deformation feature and the standard facial feature deformation feature; in determining the absolute value of the deformation difference When the value is less than the deformation difference threshold, it is determined that the deformation feature of the first facial features meets the deformation difference threshold; when it is determined that the absolute value of the deformation difference is greater than the deformation difference threshold, it is determined that the deformation feature of the first facial features does not meet the deformation difference threshold; and the standard distance and the first distance are input into the distance comparison network layer to calculate the distance difference between the first distance and the standard distance; when it is determined that the absolute value of the distance difference is less than the distance difference threshold, it is determined that the first distance meets the distance difference threshold; when it is determined that the absolute value of the distance difference is less than the distance difference threshold, it is determined that the first distance meets the distance difference threshold; when it is determined that the absolute value of the distance difference is greater than the distance difference threshold, it is determined that the first distance does not meet the distance difference threshold.

[0053] In the embodiment of the present application, the implementation method of executing an image storage event for each first re-recorded video frame to store multiple first re-recorded video frames into a target video file includes at least the following two methods.

[0054] In an optional embodiment, an image storage event is executed for each first re-recorded video frame to store multiple first re-recorded video frames into a target video file, including: executing an image storage event for each first re-recorded video frame, determining a video frame in a target video segment corresponding to the first re-recorded video frame from the target video file according to a re-recording order of the first re-recorded video frames, and using the first re-recorded video frame to cover the video frame in the target video segment corresponding to the first re-recorded video frame.

[0055] In another optional embodiment, an image storage event is executed for each first re-recorded video frame to store multiple first re-recorded video frames in a target video file, including: executing an image storage event for each first re-recorded video frame, determining a video frame in a target video segment corresponding to the first re-recorded video frame from the target video file according to the re-recording order of the first re-recorded video frame, assigning a timestamp of the video frame in the target video segment corresponding to the first re-recorded video frame to the first re-recorded video frame, and storing the first re-recorded video frame after the last video frame in the target video file in the recording order, and invalidating the video frames in the target video segment belonging to the target video segment.

[0056] In order to ensure smoothness between adjacent re-recorded video frames and non-re-recorded video frames, a frame interpolation operation may be performed on adjacent re-recorded video frames and non-re-recorded video frames. The frame interpolation operations for the first re-recorded video frame and the second re-recorded video frame are different, and specifically include the following two implementations.

[0057] In an optional embodiment, if there is a first re-recorded video frame in the target video file, the first first re-recorded video frame and the tail frame of the previous video segment adjacent to the target video segment are obtained, and at least one first transition frame is generated based on the first first re-recorded video frame and the tail frame of the previous video segment adjacent to the target video segment, and when the first re-recorded video frame replaces the corresponding video frame in the target video segment for storage, the at least one first transition frame is stored in sequence at a position between the first first re-recorded video frame and the tail frame of the previous video segment adjacent to the target video segment, or when the first re-recorded video frame is stored in the recording order, the at least one first transition frame is stored in sequence in the target video file, and a position pointing relationship is established between the at least one first transition frame and the first first re-recorded video frame and the tail frame of the previous video segment adjacent to the target video segment. Among them, the tail frame points to the first transition frame, and the last transition frame points to the first re-recorded frame.

[0058] Optionally, at least one first transition frame is generated based on the first first re-recorded video frame and the last frame of the previous video clip adjacent to the target video clip, including: inputting the first first re-recorded video frame and the last frame of the previous video clip adjacent to the target video clip into a transition frame inference model, inferring the facial feature deformation feature change trend between the second facial feature deformation feature and the third facial feature deformation feature based on the second facial feature deformation feature contained in the first first re-recorded video frame and the third facial feature deformation feature contained in the last frame of the previous video clip adjacent to the target video clip, and obtaining at least one transition frame between the first first re-recorded video frame and the last frame of the previous video clip adjacent to the target video clip based on the facial feature deformation feature change trend.

[0059] More specifically, the first first re-recorded video frame and the last frame of the previous video clip adjacent to the target video clip are input into the transition frame inference model, and the facial feature deformation feature change trend between the second facial feature deformation feature and the third facial feature deformation feature is inferred according to the second facial feature deformation feature contained in the first first re-recorded video frame and the third facial feature deformation feature contained in the last frame of the previous video clip adjacent to the target video clip, and at least one transition frame between the first first re-recorded video frame and the last frame of the previous video clip adjacent to the target video clip is obtained according to the facial feature deformation feature change trend, including: calculating the similarity between the first first re-recorded video frame and the last frame of the previous video clip adjacent to the target video clip; based on the similarity, determining the number of transition frames between the first first re-recorded video frame and the last frame of the previous video clip adjacent to the target video clip, the greater the similarity, the fewer the number of transition frames, and the smaller the similarity, the more the number of transition frames; according to the number of transition frames, performing image interpolation between the first first re-recorded video frame and the last frame of the previous video clip adjacent to the target video clip to obtain at least one transition frame.

[0060] Among them, calculating the similarity between the first first re-recorded video frame and the last frame of the previous video segment adjacent to the target video segment includes: calculating the similarity between the first first re-recorded video frame and the last frame of the previous video segment adjacent to the target video segment as the similarity between the first video segment and the second video segment.

[0061] In another optional embodiment, if the second re-recorded video frame exists in the target video file, the previous video frame and the second re-recorded video frame of the second re-recorded video frame are obtained, at least one second transition frame is generated based on the previous video frame and the second re-recorded video frame of the second re-recorded video frame, and the at least one second transition frame is sequentially stored in a position between the previous video frame and the second re-recorded video frame of the second re-recorded video frame in the target video file. The detailed implementation process of this embodiment can be found in the relevant description of the above embodiment, which will not be repeated here.

[0062] Further optionally, the target video clip may be a video type other than a speech video or a silent video, such as a video with background audio. Accordingly, the prompt information further includes: a third prompt information, which is used to prompt the target user to re-record the target video clip and guide the user to re-play the background audio to re-record the target video clip. If the target video clip is not a silent video clip, the third prompt information is output to prompt the target user to re-record the target video clip and guide the user to re-play the background audio to re-record the target video clip.

[0063] Furthermore, after the execution of the image storage event is finished, the video frames in the video sequence can also be input into the image processing model, and the background area in each video frame can be normalized to obtain an intermediate image, where the background area refers to other areas except the user area, and the user area has a first image size; an initial background image is obtained, and the initial background image has a second image size; based on the first image size and the second image size, a cropping position in the initial background image is determined, and the initial background image is cropped according to the cropping position to obtain a target background image, and the target background image includes a hollow area for carrying the user area; the intermediate image is merged with the target background image with the hollow area to obtain a target video frame. Therefore, on the one hand, when recording a video frame sequence, there is no need to rely on a specific recording background, and it can be recorded in any background. After recording the video frame sequence, the video frame sequence can be processed to obtain a target video frame with various backgrounds, and there is no need to rely on special shooting equipment, which reduces the shooting cost and shooting difficulty. On the other hand, without relying on a specific recording background, each video frame sequence is processed through an image processing model to directly obtain a target video frame with a target background image without any other additional operations, thereby reducing workload and production costs.

[0064] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 2 As shown, it includes: a memory 20a and a processor 20b; the memory 20a is used to store a computer program; the processor 20b is coupled to the memory 20a and is used to execute the computer program to implement the following steps:

[0065] In response to a video recording operation initiated for a target user, a video frame sequence containing at least a face area performed by the target user based on the script content is started to be recorded, and the video frame sequence includes multiple video frames; in the process of recording the video frame sequence, face status recognition is performed for the current video frame being recorded, and it is determined whether the recognized face status information meets the recording requirement information; if the face recognition information of the current video frame does not meet the recording requirement information, it is determined whether the target video segment to which the current video frame belongs is a speech-type video segment, and the target video segment includes the current video frame and a specified number of video frames recorded before the current video frame; if the target video segment is a speech-type video segment, a first prompt information is output, and the first prompt information is used to prompt the target user that the target video segment needs to be re-recorded and guide the target user to adjust the face recording status to re-record the target video segment; and the target user adjusts the face recording status according to the first prompt information The first deduction process after that is re-recorded until a plurality of first re-recorded video frames that meet the recording requirement information corresponding to the target video clip are obtained, and an image storage event is executed for each first re-recorded video frame to store the plurality of first re-recorded video frames into the target video file; if the target video clip is not a speech-type video clip, a second prompt information is output, and the second prompt information is used to prompt the target user that the current video frame needs to be re-recorded and guide the target user to adjust the face recording state to re-record the current video frame; and the second deduction process after the target user adjusts the face recording state according to the second prompt information is re-recorded until a second re-recorded video frame that meets the recording requirement information corresponding to the current video frame is obtained, and an image storage event is executed for the second re-recorded video frame to store the second re-recorded video frame into the target video file; in response to the recording end operation, the execution of the image storage event is terminated to obtain a complete video frame sequence.

[0066] Further optionally, after completing the execution of the image storage event, the processor 20b is also used to: input the video frames in the video sequence into the image processing model, normalize the background area in each video frame to obtain an intermediate image, the background area refers to other areas except the user area, and the user area has a first image size; obtain an initial background image, the initial background image has a second image size; based on the first image size and the second image size, determine the cropping position in the initial background image, and crop the initial background image according to the cropping position to obtain a target background image, the target background image includes a hollow area for carrying the user area; merge the intermediate image with the target background image with the hollow area to obtain a target video frame.

[0067] In an embodiment of the present application, a video frame sequence is recorded by a video recorder, and the recording requirement information includes at least a deformation threshold that the facial deformation features contained in the current video frame need to meet and a distance difference threshold that needs to be met between the target user and the video recorder. Based on this, when the processor 20b performs facial state recognition on the current video frame being recorded and determines whether the recognized facial state information meets the recording requirement information, it is specifically used to: obtain the script content, input the script content into the information reasoning model, so as to map the script content to the standard facial deformation features of each video frame in the video frame sequence and the standard distance between the target user and the video recorder; input the current video frame into the facial information recognition model, recognize the facial feature information of the current video frame, and obtain the first facial deformation features of the current frame image; and collect the facial state information of the target user and the video when recording the current video frame. The first distance between the recorders; input the standard deformation feature, the standard distance, the first facial feature deformation feature, the first distance, the deformation difference threshold and the distance difference threshold into the information comparison model, and determine whether the first facial feature deformation feature meets the deformation threshold based on the first facial feature deformation feature and the standard facial feature deformation feature, and determine whether the first distance meets the distance difference threshold based on the first distance and the standard distance; when both judgment results are yes, determine that the recognized face state information meets the recording requirement information; when any judgment result is no, determine that the recognized face state information does not meet the recording requirement information.

[0068] Optionally, the information comparison model includes a deformation feature comparison network layer and a distance comparison network layer; the processor 20b inputs the standard deformation feature, the standard distance, the first facial feature deformation feature, the first distance, the deformation difference threshold and the distance difference threshold into the information comparison model, and determines whether the first facial feature deformation feature satisfies the deformation threshold based on the first facial feature deformation feature and the standard facial feature deformation feature, and determines whether the first distance satisfies the distance difference threshold based on the first distance and the standard distance, and is specifically used for: inputting the standard deformation feature, the first facial feature deformation feature, the deformation difference threshold and the distance difference threshold into the deformation feature comparison network layer, calculating the deformation difference between the first facial feature deformation feature and the standard facial feature deformation feature; when determining the deformation difference When the absolute value of the value is less than the deformation difference threshold, it is determined that the deformation feature of the first facial features meets the deformation difference threshold; when it is determined that the absolute value of the deformation difference is greater than the deformation difference threshold, it is determined that the deformation feature of the first facial features does not meet the deformation difference threshold; and the standard distance and the first distance are input into the distance comparison network layer to calculate the distance difference between the first distance and the standard distance; when it is determined that the absolute value of the distance difference is less than the distance difference threshold, it is determined that the first distance meets the distance difference threshold; when it is determined that the absolute value of the distance difference is less than the distance difference threshold, it is determined that the first distance meets the distance difference threshold; when it is determined that the absolute value of the distance difference is greater than the distance difference threshold, it is determined that the first distance does not meet the distance difference threshold.

[0069] In the embodiment of the present application, when the processor 20b executes an image storage event for each first re-recorded video frame to store multiple first re-recorded video frames into a target video file, it is specifically used to: execute an image storage event for each first re-recorded video frame, determine the video frame in the target video segment corresponding to the first re-recorded video frame from the target video file according to the re-recording order of the first re-recorded video frame, and use the first re-recorded video frame to cover the video frame in the target video segment corresponding to the first re-recorded video frame; or, execute an image storage event for each first re-recorded video frame, determine the video frame in the target video segment corresponding to the first re-recorded video frame from the target video file according to the re-recording order of the first re-recorded video frame, assign the timestamp of the video frame in the target video segment corresponding to the first re-recorded video frame to the first re-recorded video frame, and store the first re-recorded video frame after the last video frame in the target video file according to the recording order, and invalidate the video frames in the target video segment that belong to the target video segment.

[0070] Further optionally, if there is a first re-recorded video frame in the target video file, the first first re-recorded video frame and the last frame of the previous video segment adjacent to the target video segment are obtained, and based on the first first re-recorded video frame and the last frame of the previous video segment adjacent to the target video segment, at least one first transition frame is generated, and when the first re-recorded video frame replaces the corresponding video frame in the target video segment for storage, the at least one first transition frame is stored in sequence at a position between the first first re-recorded video frame and the last frame of the previous video segment adjacent to the target video segment, or when the first re-recorded video frame is stored in the recording order, the at least one first transition frame is stored in sequence in the target video file, and a position pointing relationship is established between the at least one first transition frame and the first first re-recorded video frame and the last frame of the previous video segment adjacent to the target video segment.

[0071] If there is a second rerecorded video frame in the target video file, the previous video frame and the second rerecorded video frame of the second rerecorded video frame are obtained, at least one second transition frame is generated based on the previous video frame and the second rerecorded video frame of the second rerecorded video frame, and the at least one second transition frame is stored in sequence at a position between the previous video frame and the second rerecorded video frame of the second rerecorded video frame in the target video file.

[0072] Optionally, at least one first transition frame is generated based on the first first re-recorded video frame and the last frame of the previous video clip adjacent to the target video clip, including: inputting the first first re-recorded video frame and the last frame of the previous video clip adjacent to the target video clip into a transition frame inference model, inferring the facial feature deformation feature change trend between the second facial feature deformation feature and the third facial feature deformation feature based on the second facial feature deformation feature contained in the first first re-recorded video frame and the third facial feature deformation feature contained in the last frame of the previous video clip adjacent to the target video clip, and obtaining at least one transition frame between the first first re-recorded video frame and the last frame of the previous video clip adjacent to the target video clip based on the facial feature deformation feature change trend.

[0073] Further optionally, if the target video clip is not a speech type video clip, the processor 20b is also used to output a second prompt message, including: when the target video clip is not a speech type video clip, judging whether the target video clip is a silent type video clip according to whether the target video clip contains an audio frame; if the target video clip is a silent type video clip, outputting the second prompt message.

[0074] Further optionally, if the target video clip is not a silent video clip, the processor 20b is further used to output a third prompt message, and the third prompt message is used to prompt the target user to re-record the target video clip and guide the user to re-play the background audio to re-record the target video clip.

[0075] Furthermore, if Figure 2 As shown, the electronic device also includes: a communication component 20c, a display 20d, a power component 20e, an audio component 20f and other components. Figure 2 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 2 Components shown.

[0076] The detailed implementation methods and beneficial effects of each step provided in the embodiments of the present application have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0077] The embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor implements the steps in the above method.

[0078] The embodiments of the present application also provide a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the processor is enabled to implement the steps in the above method.

[0079] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, compact disk read-only memory (Compact Disk Read-Only Memory, CD-ROM), optical storage, etc.) that contain computer-usable program code.

[0080] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0081] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0082] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0083] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interface, network interface and memory.

[0084] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash memory (flashRAM). The memory is an example of a computer-readable medium.

[0085] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (Phase-change Random Access Memory, PRAM), static random access memory (SRAM), dynamic random access memory (Dynamic Random Access Memory, DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory, etc.

[0086] (EEPROM), flash memory or other memory technology, CD-ROM, Digital Versatile Disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0087] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0088] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A video recording method, characterized in that: include: In response to a video recording operation initiated for a target user, start recording a video frame sequence containing at least a face area performed by the target user based on script content, wherein the video frame sequence includes a plurality of video frames; During the process of recording the video frame sequence, face status recognition is performed on the current video frame being recorded and it is determined whether the recognized face status information meets the recording requirement information; If the face recognition information of the current video frame does not meet the recording requirement information, determine whether the target video segment to which the current video frame belongs is a speech-type video segment, wherein the target video segment includes the current video frame and a specified number of video frames recorded before the current video frame; If the target video clip is a speech-type video clip, outputting first prompt information, wherein the first prompt information is used to prompt the target user to re-record the target video clip and guide the target user to adjust the face recording state to re-record the target video clip; and re-recording the first interpretation process after the target user adjusts the face recording state according to the first prompt information, until a plurality of first re-recorded video frames that meet the recording requirement information corresponding to the target video clip are obtained, and executing an image storage event for each first re-recorded video frame to store the plurality of first re-recorded video frames into a target video file; If the target video clip is not a speech-type video clip, outputting second prompt information, wherein the second prompt information is used to prompt the target user to re-record the current video frame and guide the target user to adjust the face recording state to re-record the current video frame; and re-recording a second interpretation process after the target user adjusts the face recording state according to the second prompt information, until a second re-recorded video frame that meets the recording requirement information corresponding to the current video frame is obtained, and executing an image storage event for the second re-recorded video frame to store the second re-recorded video frame into the target video file; In response to the recording end operation, the image storage event is terminated to obtain the complete video frame sequence.

2. The method according to claim 1, characterized in that: After the image storage event is finished, the method further includes: Inputting video frames in the video sequence into an image processing model, and performing normalization processing on a background area in each video frame to obtain an intermediate image, wherein the background area refers to other areas except a user area, and the user area has a first image size; Acquire an initial background image, wherein the initial background image has a second image size; Based on the first image size and the second image size, determining a cropping position in the initial background image, and cropping the initial background image according to the cropping position to obtain a target background image, wherein the target background image includes a hollowed-out area for carrying the user area; The intermediate image is fused with a target background image with a hollowed-out area to obtain a target video frame.

3. The method according to claim 1, characterized in that The video frame sequence is recorded by a video recorder, and the recording requirement information includes at least a deformation threshold that the facial features included in the current video frame need to meet and a distance difference threshold that the target user needs to meet with the video recorder; Perform facial recognition on the current video frame being recorded and determine whether the recognized facial status information meets the recording requirements, including: Acquire the script content, and input the script content into an information reasoning model to map the script content to the standard facial deformation features of each video frame in the video frame sequence and the standard distance between the target user and the video recorder; Inputting the current video frame into a facial feature information recognition model, recognizing facial feature information of the current video frame, and obtaining a first facial feature deformation feature of the current frame image; and collecting a first distance between the target user and the video recorder when recording the current video frame; Inputting the standard deformation feature, the standard distance, the first facial feature deformation feature, the first distance, the deformation difference threshold, and the distance difference threshold into the information comparison model, determining whether the first facial feature deformation feature satisfies the deformation threshold based on the first facial feature deformation feature and the standard facial feature deformation feature, and determining whether the first distance satisfies the distance difference threshold based on the first distance and the standard distance; When both judgment results are yes, it is determined that the recognized face state information meets the recording requirement information; In the case where any judgment result is negative, it is determined that the recognized face status information does not meet the recording requirement information.

4. The method according to claim 3, characterized in that The information comparison model includes a deformation feature comparison network layer and a distance comparison network layer; Inputting the standard deformation feature, the standard distance, the first facial feature deformation feature, the first distance, the deformation difference threshold, and the distance difference threshold into the information comparison model, determining whether the first facial feature deformation feature satisfies the deformation threshold based on the first facial feature deformation feature and the standard facial feature deformation feature, and determining whether the first distance satisfies the distance difference threshold based on the first distance and the standard distance, including: Inputting the standard deformation feature, the first facial feature deformation feature, the deformation difference threshold and the distance difference threshold into the deformation feature comparison network layer, calculating the deformation difference between the first facial feature deformation feature and the standard facial feature deformation feature; when it is determined that the absolute value of the deformation difference is less than the deformation difference threshold, determining that the first facial feature deformation feature satisfies the deformation difference threshold; when it is determined that the absolute value of the deformation difference is greater than the deformation difference threshold, determining that the first facial feature deformation feature does not satisfy the deformation difference threshold; and The standard distance and the first distance are input into the distance comparison network layer, and the distance difference between the first distance and the standard distance is calculated; when it is determined that the absolute value of the distance difference is less than the distance difference threshold, it is determined that the first distance satisfies the distance difference threshold; when it is determined that the absolute value of the distance difference is less than the distance difference threshold, it is determined that the first distance satisfies the distance difference threshold; when it is determined that the absolute value of the distance difference is greater than the distance difference threshold, it is determined that the first distance does not satisfy the distance difference threshold.

5. The method according to claim 1, characterized in that: Executing an image storage event for each first re-recorded video frame to store the plurality of first re-recorded video frames into the target video file, comprising: executing an image storage event for each first re-recorded video frame, determining a video frame in the target video segment corresponding to the first re-recorded video frame from the target video file according to the re-recording order of the first re-recorded video frame, and overwriting the video frame in the target video segment corresponding to the first re-recorded video frame with the first re-recorded video frame; or, An image storage event is executed for each first re-recorded video frame, and according to the re-recording order of the first re-recorded video frame, a video frame in the target video segment corresponding to the first re-recorded video frame is determined from the target video file, a timestamp of the video frame in the target video segment corresponding to the first re-recorded video frame is assigned to the first re-recorded video frame, and the first re-recorded video frame is stored after the last video frame in the target video file in the recording order, and the video frames in the target video segment belonging to the target video segment are invalidated.

6. The method according to claim 5, characterized in that Also includes: If the first re-recorded video frame exists in the target video file, the first first re-recorded video frame and the tail frame of the previous video segment adjacent to the target video segment are obtained, and at least one first transition frame is generated based on the first first re-recorded video frame and the tail frame of the previous video segment adjacent to the target video segment, and when the first re-recorded video frame replaces the corresponding video frame in the target video segment for storage, the at least one first transition frame is stored in sequence at a position between the first first re-recorded video frame and the tail frame of the previous video segment adjacent to the target video segment, or when the first re-recorded video frame is stored in a recording order, the at least one first transition frame is stored in sequence in the target video file, and a position pointing relationship is established between the at least one first transition frame and the first first re-recorded video frame and the tail frame of the previous video segment adjacent to the target video segment; and / or If the second rerecorded video frame exists in the target video file, the previous video frame of the second rerecorded video frame and the second rerecorded video frame are obtained, and at least one second transition frame is generated based on the previous video frame of the second rerecorded video frame and the second rerecorded video frame, and the at least one second transition frame is stored in sequence at a position between the previous video frame of the second rerecorded video frame and the second rerecorded video frame in the target video file.

7. The method according to claim 6, characterized in that Generating at least one first transition frame based on the first first re-recorded video frame and the tail frame of the previous video segment adjacent to the target video segment, comprising: The first re-recorded video frame and the last frame of the previous video clip adjacent to the target video clip are input into the transition frame inference model, and the facial feature deformation feature change trend between the second facial feature deformation feature and the third facial feature deformation feature is inferred based on the second facial feature deformation feature contained in the first re-recorded video frame and the last frame of the previous video clip adjacent to the target video clip, and at least one transition frame between the first re-recorded video frame and the last frame of the previous video clip adjacent to the target video clip is obtained based on the facial feature deformation feature change trend.

8. The method according to any one of claims 1 to 7, characterized in that: If the target video segment is not a speech-type video segment, outputting second prompt information includes: In a case where the target video segment is not a speech-type video segment, judging whether the target video segment is a silent-type video segment according to whether the target video segment contains an audio frame; If the target video segment is a silent video segment, the second prompt information is output.

9. The method according to claim 8, characterized in that Also includes: If the target video segment is not a silent video segment, a third prompt message is output, wherein the third prompt message is used to prompt the target user to re-record the target video segment and guide the user to re-play the background audio to re-record the target video segment.

10. An electronic device, characterized in that: include: Memory and processor; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the method according to any one of claims 1 to 9.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to implement the steps in the method according to any one of claims 1 to 9.

12. A computer program product, characterized in that The computer program product comprises a computer program / instruction, which, when executed by a processor, enables the processor to implement the steps of any one of the methods of claims 1 to 9.

Images