MV recording method and display device

Abstract

The present application provides an MV recording method and display device, which identifies the first identity information corresponding to the face appearing in the received video data according to the correspondence between the face and the identity information stored in advance, determines the target identity information of the target face for video extraction, determines the target area where the target object containing the target identity information is located in the video frame in the video data, and extracts the target object from the target area. First, the first identity information of the face appearing in the video data is identified, and then the target identity information of the target face for video extraction is determined. Then, based on the target identity information, the target area where the target object of the target identity information is located is determined in the video frame of the video data, and the target object is extracted from the target area, so that when multiple objects appear in the video data, only one target object is extracted, which improves the user experience.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a MV recording method and display device. Background Art

[0002] With the development of random technology, display devices, such as smart TVs, have more and more functions. Users can make video calls with friends through display devices. Compared with video calls using mobile phones, display devices have larger screens and clearer displays. However, when making video calls based on display devices, the camera is not convenient to move, resulting in a single call scene and high limitations on its usage scenarios. Therefore, an MV recording function is added to the video call in the display device to achieve scene switching during the video call.

[0003] Specifically, when recording an MV, the display device usually uses a portrait segmentation algorithm to segment the object in the video displayed by the display device and add it to the configured background, thereby increasing the fun of the video call. However, in the prior art, when using the portrait segmentation algorithm for segmentation, only the object and the background are segmented. When multiple objects appear in a video frame, the multiple objects are segmented and the multiple segmented images are merged with the background. However, when actually making a video call, there is often only one person making a video call. Therefore, segmenting multiple objects and merging them with the background cannot meet the requirements of the video call scene, affecting the accuracy of the video obtained after fusion. Summary of the invention

[0004] The present application provides an MV recording method, a display device, an electronic device and a medium, which are used to solve the problem in the prior art that when an actual video call is made, multiple objects can only be separated and merged with the background, which cannot meet the requirements of the video call scene and affects the accuracy of the video obtained after the fusion.

[0005] In a first aspect, the present application further provides a method for recording an MV, the method comprising:

[0006] Identify first identity information corresponding to a face appearing in the received video data according to a pre-stored correspondence between a face and identity information;

[0007] The target identity information of the target face to be extracted from the video is determined, a target area where a target object containing the target identity information is located is determined in a video frame in the video data, and the target object is extracted from the target area.

[0008] In a second aspect, the present application provides a display device, the display device comprising:

[0009] A display, the display being used to display an image containing a target object;

[0010] A camera, wherein the camera is used to capture an image containing a target object;

[0011] A controller, the controller being configured to:

[0012] The display is controlled to display the MV video containing the target object.

[0013] In a third aspect, the present application further provides an electronic device, which includes at least a processor and a memory, and the processor is used to implement the steps of any of the above-mentioned MV recording methods when executing a computer program stored in the memory.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned MV recording methods.

[0015] Since the present application identifies the first identity information corresponding to the face appearing in the received video data based on the correspondence between the pre-saved face and the identity information, determines the target identity information of the target face for video extraction, determines the target area where the target object containing the target identity information is located in the video frame in the video data, and extracts the target object from the target area. In the present application, the first identity information of the face appearing in the video data is first identified, and then the target identity information of the target face for video extraction is determined. Then, based on the target identity information, the target area where the target object of the target identity information is located is determined in the video frame of the video data, and the target object is extracted from the target area. This achieves that when multiple objects appear in the video data, only one target object is extracted, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 A schematic diagram of the MV recording process provided by this application;

[0018] Figure 2 A schematic diagram of the interaction between the display device provided in the present application and the electronic device with which the display device makes a video call;

[0019] Figure 3aA schematic diagram of a display of a smart TV provided in this application;

[0020] Figure 3b A schematic diagram of a display of a smart TV provided in this application;

[0021] Figure 4 A schematic diagram showing the input position of the MV switching instruction provided in this application;

[0022] Figure 5 A schematic diagram of the MV recording process provided for this application;

[0023] Figure 6 A flowchart of the MV recording process provided for this application;

[0024] Figure 7 A schematic diagram of the structure of a display device provided in this application;

[0025] Figure 8 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0027] In the present application, when a video call is made based on a display device, if multiple objects appear in the video data collected by the audio and video acquisition device received by the display device, only one target object is extracted, thereby improving the user experience. In the present application, based on the pre-saved correspondence between the face and the identity information, the first identity information corresponding to the face appearing in the received video data is identified, and then the target identity information of the target face to be extracted is determined, and the target area where the target object containing the target face is located is determined from the video frame of the received video data, and the target object is extracted from the target area.

[0028] In order to achieve extraction of only one target object when multiple objects appear in video data and improve user experience, the present application provides a MV recording method, display device, electronic device and medium.

[0029] Figure 1 A schematic diagram of a MV recording process provided by this application, the process includes:

[0030] S101: Identify first identity information corresponding to a face appearing in received video data according to a pre-stored correspondence between a face and identity information.

[0031] An MV recording method provided in the present application is applied to a display device, which may be a smart TV, etc. The MV recording method may also be applied to a server.

[0032] In the present application, the correspondence between faces and identity identifiers is pre-stored, wherein the identity identifier may be the family identity of the user corresponding to each face, the name of the user corresponding to each face, or the number corresponding to each face, etc., which is information that uniquely identifies each face. When the correspondence between faces and identity identifiers is pre-stored, the identity identifier information corresponding to each face is different.

[0033] Receive an MV recording instruction. When video data is received and multiple objects appear in the video data, and you want to extract a target object, you can identify the first identity information corresponding to the face appearing in the received video data based on the face recognition method and the pre-saved correspondence between the face and the identity information. Specifically, in the present application, for the face appearing in the video data, the first identity information corresponding to the face is determined based on the pre-saved correspondence between the face and the identity information. The number of faces appearing in the video data may be one or more.

[0034] S102: Determine target identity information of a target face for video extraction, determine a target region where a target object containing the target identity information is located in a video frame in the video data, and extract the target object from the target region.

[0035] After the first identity information is identified, the target identity information of the target face for video extraction is determined. Specifically, after the first identity information is identified, if there is only one first identity information, the first identity information is determined as the target identity information, and if the number of the first identity information is greater than 1, the user is prompted to input a selection operation for the target identity information, and the target identity information is determined according to the user's selection.

[0036] For example, when a first identity identification information "Zhang San" is recognized, "Zhang San" is determined as the target identity identification information; when three first identity identification information are recognized, namely "Wang Er", "Zhang San" and "Li Si", the three first identity identification information are displayed, and the user is prompted to select the target identity identification information.

[0037] After determining the target identity information, the target area where the target object containing the target identity information is located is determined from the video frame in the video data. Specifically, through human body detection technology, the target area where the target object corresponding to the target identity information is located is determined, and the target area only includes the target object and the background environment where the target object is located in the video frame. Then, a portrait segmentation algorithm is used to separate the target object in the target area from the background environment to obtain the target object, and the target object is displayed.

[0038] Since the present application identifies the first identity information corresponding to the face appearing in the received video data according to the correspondence between the pre-saved face and the identity information, determines the target identity information of the target face for video extraction, determines the target area where the target object containing the target identity information is located in the video frame in the video data, and extracts the target object from the target area. In the present application, the target identity information of the target face for video extraction is first determined, and then the target area where the target object of the target identity information is located is determined in the video frame of the video data according to the target identity information, and finally the target object is extracted from the target area, so that when multiple objects appear in the video data, only one target object is extracted, thereby improving the user experience.

[0039] In order to achieve the extraction of only one target object when multiple objects appear in the video data, thereby improving the user experience, based on the above embodiments, in this application, the target identity information of the target face to be extracted from the video includes:

[0040] An input selection operation is received, wherein the selection operation carries selected second identity identification information, and the second identity identification information is determined as the target identity identification information.

[0041] In the present application, when the number of first identity identification information identified is greater than one, the first identity identification information will be displayed, and then the user will select the target identity identification information.

[0042] In the present application, if the MV recording method is applied to a display device, the user can input a selection operation through a remote control of the display device, can also input a selection operation through voice, can also input a selection operation through a display of the display device, etc. After receiving the selection operation, since the selection operation carries the selected second identity identification information, the second identity identification information is determined as the target identity identification information.

[0043] Specifically, in the present application, face detection is performed on the received video data to detect whether there is a face in the video data. If there is, the first identity information of the face appearing in the video data is identified, and it is determined whether the number of the first identity information currently identified is greater than 1. If the number of the first identity information is greater than 1, that is, the video data contains multiple faces, it is determined whether an input selection operation for selecting the target identity information is received. If so, the subsequent determination of the target area where the target object containing the target identity information is located in the video data is continued. If no input selection operation is received, the user is reminded to input the selection operation. If the number of the identified first identity information is 1, the first identity information corresponding to the face is directly determined as the target identity information.

[0044] For example, in the present application, the first identity identification information is a name, and the first identity identification information includes three names: "Zhang San", "Li Si" and "Wang Er". The three first identity identification information are displayed, and the user inputs a selection operation of selecting "Zhang San" through the remote control. After receiving the selection operation, "Zhang San" is used as the target identity identification information.

[0045] In order to achieve the extraction of only one target object when multiple objects appear in the video data, thereby improving the user experience, based on the above embodiments, in the present application, before identifying the first identity information corresponding to the face appearing in the received video data according to the pre-saved correspondence between the face and the identity information, the method further includes:

[0046] If an input MV recording instruction is received, a subsequent operation of identifying the first identity information corresponding to the face appearing in the received video data according to the pre-stored correspondence between the face and the identity information is performed;

[0047] If the MV recording instruction is not received, the video data is displayed.

[0048] In the present application, the display device has the function of MV recording, but the MV recording will be performed only when the display device receives the MV recording instruction, that is, only when the display device receives the input MV recording instruction, will the subsequent operation of identifying the first identity information corresponding to the face appearing in the received video data according to the pre-saved correspondence between the face and the identity information be performed. When the display device does not receive the MV recording instruction, the video data is directly displayed without processing the video data.

[0049] In order to achieve the extraction of only one target object when multiple objects appear in the video data, thereby improving the user experience, based on the above embodiments, in this application, after receiving the input MV recording instruction, the method also includes:

[0050] If after receiving the MV recording instruction, an input MV selection operation is received, the selected first target MV is determined, and each frame in the first target MV is superimposed with the target object extracted from the corresponding video frame in sequence, and the first target MV after superimposing the target object is displayed.

[0051] In the present application, a plurality of MVs are pre-stored in the display device, and the user can select a first target MV from the plurality of MVs after sending an MV recording instruction to the display device.

[0052] Specifically, after the display device receives the MV recording instruction, the user inputs the MV selection operation through the remote control or the display of the display device or voice, and selects the first target MV. After receiving the selection operation, the display device uses the MV selected by the selection operation as the first target MV, and sequentially superimposes each frame in the first target MV with the target object extracted from the corresponding video frame, and displays the first target MV with the superimposed target object.

[0053] After receiving the MV recording instruction, the first identity information corresponding to the face in the video data is identified, and the target identity information of the target face for video extraction is determined, the target area where the target object containing the target identity information is located is determined in the video frame in the video data, and the target object is extracted from the target area, and each frame in the first target MV is sequentially superimposed with the target object extracted from the video frame in the corresponding video data to achieve MV recording. Specifically, in the present application, after the first target MV is selected, the first target MV is played, and the time difference between the video frame and the start of the playback of the first target MV is determined based on the video frame in which the target object is extracted from the video data and the first target MV, thereby determining the target frame in the first target MV on which the target object is superimposed. In the present application, if there is a video frame in the video data that does not contain the target object, the target frame in the first target MV corresponding to the video frame will not be superimposed.

[0054] Specifically, in the present application, after the first target MV is selected, for the video data, the target objects are extracted from the video frames in the video data in chronological order, and the target object extracted from the first video frame of the video data is superimposed with the first frame of the first target MV. Other target objects extracted from the video data are superimposed with each frame in the first target MV in the order of extraction, and the first target MV with the superimposed target object is displayed. In the present application, if the target object is not extracted in the first video frame of the video data, the first frame of the first target MV will not be superimposed. Therefore, when displaying the first target MV with the superimposed target object, it may happen that a certain video frame of the first target MV with the superimposed target object does not contain the object.

[0055] In order to achieve the extraction of only one target object when multiple objects appear in the video data, thereby improving the user experience, based on the above embodiments, in the present application, before receiving the MV recording instruction, the method further includes:

[0056] Receiving first audio and video data containing the target object, and obtaining audio data and the video data in the first audio and video data;

[0057] If the MV recording instruction is received, the audio data and the first target MV with the target object superimposed thereon are sent to the cloud server; if the MV recording instruction is not received, the audio data and the video data are sent to the cloud server.

[0058] In the present application, a first audio and video device including a target object is also received. Specifically, if the MV recording method is applied to a display device, and the display device also includes a camera, the first audio and video data can be collected by the camera of the display device. If the MV recording method is applied to a display device, but the display device does not include a camera, or the MV recording method is applied to a server, the first audio and video data can be collected and sent by an audio and video acquisition device, and the audio and video acquisition device can be a camera with communication function. When the camera is working, it will collect the environment and objects within its video acquisition range, and also collect the sound within its acquisition range to obtain the first audio and video data. After receiving the first audio and video data, the first audio and video data is processed to obtain audio data and video data.

[0059] Among them, if an MV recording instruction is received, the target object can be extracted from the video data based on the video data, and the target object extracted from each frame of the video data is superimposed on each frame of the first target MV in sequence, the display device displays the first target MV with the superimposed target object, and sends the first target MV with the superimposed target object and the audio data obtained from the audio and video data to the cloud server, which is then sent by the cloud server to an electronic device that has a video call with the display device, and the electronic device displays the first target MV with the superimposed target object and plays the audio data.

[0060] If the MV recording instruction is not received, it means that there is no need to perform MV recording, and only a video call is being made, that is, there is no need to identify the first identity information corresponding to the face in the video data, and determine the target identity information of the target face for video extraction, nor is there any need to determine the target area where the target object containing the target identity information is located in the video data, and extract the target object from the target area, and sequentially superimpose each frame in the first target MV with the target object extracted from the corresponding video frame. Therefore, when the MV recording instruction is not received, the video data is displayed, and the video data and the audio data are sent to the cloud server.

[0061] Figure 2 A schematic diagram of the interaction between the display device provided in this application and the electronic device for video calling, such as the Figure 2 As shown, the display device sends the first target MV with the target object superimposed on it to the cloud server, and the cloud server then sends the first target MV with the target object superimposed on it to the electronic device with which the video call is being conducted. At the same time, the display device also obtains the audio and video data sent by the electronic device from the cloud server and displays it.

[0062] In order to achieve the extraction of only one target object when multiple objects appear in the video data, thereby improving the user experience, based on the above embodiments, in this application, the method further includes:

[0063] Receiving second audio and video data sent by the cloud server and sent by an electronic device that is having a video call with the display device;

[0064] Displaying second video data in the second audio and video data;

[0065] Play the second audio data in the second audio and video data.

[0066] In the present application, the display device can be used to make video calls. When making a video call, the display device can display a first target MV with a target object superimposed on it on the display, and send the first target MV with the target object superimposed to a cloud server, which is then sent by the cloud server to an electronic device that is talking to the display device, so that the display interface of the electronic device displays the first target MV with the target object superimposed on it.

[0067] In addition, in the present application, if the first target MV has its corresponding target music, before sending the audio data to the cloud server, the target music is superimposed on the audio data, and the audio data superimposed with the target music is sent to the cloud server.

[0068] In the present application, when second audio and video data sent by an electronic device that is having a video call with the display device is received from a cloud server, second audio data is obtained from the second audio and video data and played.

[0069] Figure 3a A display schematic diagram of a smart TV display provided in this application, such as the Figure 3a As shown, the display of the smart TV displays both the local video data and the second video data in the second audio and video data sent by the cloud server.

[0070] Figure 3b A display schematic diagram of a smart TV display provided in this application, such as the Figure 3b As shown, the first target MV with the target object superimposed is displayed on the display of the smart TV, and the second video data in the second audio and video data sent by the cloud server is also displayed.

[0071] In order to achieve the extraction of only one target object when multiple objects appear in the video data, thereby improving the user experience, based on the above embodiments, in this application, the method further includes:

[0072] Responding to a received barrage instruction, wherein the barrage instruction carries a barrage form and barrage content;

[0073] According to the bullet screen format, the bullet screen content is displayed in the first target MV after the target object is superimposed.

[0074] In order to increase the fun of the video, in this application, the display device will also receive a bullet screen instruction, and display the bullet screen content in the first target MV after the target object is superimposed according to the bullet screen format and bullet screen content carried in the bullet screen instruction. The bullet screen format can be text, gift special effects, etc.

[0075] In the present application, the barrage command may be input by a user through a remote control, voice or display, or may be a barrage command received by the display device through a cloud server from an electronic device that is making a video call with the display device.

[0076] In order to achieve the extraction of only one target object when multiple objects appear in the video data, thereby improving the user experience, based on the above embodiments, in this application, the method further includes:

[0077] If the last frame of the first target MV has been superimposed with the extracted first target object, and the second target object is extracted from the video frame and no MV switching instruction is received, then starting from the first frame of the first target MV, each frame of the first target MV is sequentially superimposed with the second target object extracted from the corresponding video frame;

[0078] If the last frame in the first target MV has been superimposed with the extracted first target object, and the second target object is extracted in the video frame and an MV switching instruction is received, the second target MV after switching is determined, and each frame in the second target MV is superimposed with the second target object extracted from the corresponding video frame in sequence, and the second target MV after superimposing the target object is displayed.

[0079] In the present application, the duration of the first target MV is limited, so when the extracted first target object is superimposed with each frame in the first target MV once, it may be superimposed to the last frame of the first target MV. After the last frame of the first target MV is superimposed with the extracted first target object, the user may switch the MV. If the user does not choose to switch the MV, and the second target object is extracted from the video frame of the received video data, that is, the MV switching instruction is not received and the second target object is extracted from the video frame, then starting from the first frame of the first target MV, each frame in the first target MV is superimposed with the second target object extracted from the corresponding video frame in sequence.

[0080] In the present application, when the last frame superimposed on the first target MV is superimposed with the extracted first target object, if an MV switching instruction is received, the second target MV after switching is determined, and each frame in the second target MV is superimposed with the second target object extracted from the corresponding video frame in sequence.

[0081] In addition, in the present application, if the frame currently superimposed with the extracted first target object is not the last frame in the first target MV, but an MV switching instruction is received, the target object extracted subsequently will no longer be superimposed with the first target MV, but the second target MV after switching will be determined, and each frame in the second target MV will be superimposed with the second target object extracted from the corresponding video frame in sequence.

[0082] Figure 4 A schematic diagram showing the input position of the MV switching instruction provided in this application, such as the Figure 4 As shown, when the smart TV is recording an MV, the user can switch MVs through 3, 4 or 6, where button 3 is used to select the previous song of the current first target MV as the second target MV, button 4 is used to select the next song of the current MV as the second target MV, and button 6 is used to display the directory of MVs saved in the smart TV. In addition, the display interface of the smart TV also displays button 1 for returning, button 2 for adjusting the volume, button 5 for pausing MV recording, and button 7 for exiting MV recording.

[0083] Figure 5 The process diagram of MV recording provided for this application includes:

[0084] S501: receiving a video call instruction and making a video call.

[0085] S502: Receive audio and video data sent by an audio and video acquisition device.

[0086] S503: Receive MV recording instruction.

[0087] S504: Receive a selection operation for selecting a first target MV.

[0088] S505: According to the pre-stored correspondence between the human face and the identity information, identify the first identity information corresponding to the human face appearing in the received video data, and determine the target identity information of the target face for video extraction.

[0089] S506: Determine a target region where a target object including target identity information is located in a video frame in the video data, and extract the target object from the target region.

[0090] S507: sequentially superimpose each frame in the first target MV with the target object extracted from the corresponding video frame, and display the first target MV with the superimposed target object.

[0091] S508: Send the first target MV with the target object superimposed thereon to the cloud server.

[0092] In order to achieve the extraction of only one target object when multiple objects appear in the video data, thereby improving the user experience, based on the above embodiments, in this application, the method further includes:

[0093] After extracting the target object, identifying the target posture information of the target object, and determining whether the target posture information is pre-saved posture information;

[0094] If yes, determining the target instruction corresponding to the target posture information according to the instruction corresponding to the pre-stored posture information;

[0095] According to the target instruction, the target special effect corresponding to the target instruction is determined and displayed.

[0096] In this application, users can send instructions using their own posture information when recording video data, thereby achieving the purpose of adding and displaying special effects.

[0097] Specifically, instructions corresponding to the posture information and special effects corresponding to each instruction are pre-saved in the display device. After the target object is extracted, the target posture information of the target object is identified, and it is determined whether the target posture information is pre-saved posture information. If so, the target instruction corresponding to the target posture information is determined according to the instruction corresponding to the pre-saved posture information, and the target special effect corresponding to the target instruction is determined and displayed.

[0098] For example, the pre-set posture information includes "squatting with knees hugged" and "making a heart shape with hands". The corresponding instruction for "squatting with knees hugged" is an instruction to imitate a rabbit. When the target posture information of the target object is identified as the user squatting with knees hugged, an animation effect of a bunny jumping is displayed, interacting with the target object. The corresponding instruction for "making a heart shape with hands" is an instruction to eject a heart shape with hands. When the target posture information of the target object is identified as the "making a heart shape with hands" gesture, a red heart will gradually pop up at the heart shape position of the target object.

[0099] Figure 6 The flowchart of the MV recording process provided by this application is as follows: Figure 6 As shown, the process includes:

[0100] S601: Collect audio and video data, and obtain video data and audio data from the audio and video data.

[0101] S602: Determine whether an MV recording instruction is received. If no MV recording instruction is received, execute S608; if an MV recording instruction is received, execute S603.

[0102] S603: receiving an input MV selection operation, determining a selected first target MV, and identifying first identity information corresponding to a face appearing in the received video data according to a pre-stored correspondence between a face and identity information.

[0103] S604: Determine the target identity information of the target face for video extraction, determine the target area where the target object containing the target identity information is located in the video frame in the video data, extract the target object from the target area, and superimpose each frame in the first target MV with the target object extracted from the corresponding video frame in sequence.

[0104] S605: Display the first target MV with the target object superimposed thereon, and send the first target MV with the target object superimposed thereon to the cloud server.

[0105] S606: Superimpose the audio data with the target music corresponding to the first target MV, and send the audio data superimposed with the target music to the cloud server.

[0106] In the present application, when performing the MV recording process, the execution order of S605 and S606 is not limited. S605 and S606 can be executed simultaneously, or S605 can be executed first and then S606, or S606 can be executed first and then S605.

[0107] S607: If the received barrage instruction carries a barrage form and barrage content, then according to the barrage form, the barrage content is displayed in the first target MV after the target object is superimposed.

[0108] S608: Display the video data, and send the audio and video data to the cloud server.

[0109] Figure 7 A schematic diagram of the structure of a display device provided in the present application, the display device comprising:

[0110] A display 701, the display is used to display the MV video containing the target object;

[0111] A camera 702, which is used to capture an image containing a target object;

[0112] A controller 703, wherein the controller is configured to:

[0113] The display is controlled to display the MV video containing the target object.

[0114] In a possible implementation manner, the controller 703 is further configured to:

[0115] Identify first identity information corresponding to a face appearing in the received video data according to a pre-stored correspondence between a face and identity information;

[0116] The target identity information of the target face to be extracted from the video is determined, a target area where a target object containing the target identity information is located is determined in a video frame in the video data, and the target object is extracted from the target area.

[0117] In a possible implementation manner, the controller 703 is further configured to:

[0118] An input selection operation is received, wherein the selection operation carries selected second identity identification information, and the second identity identification information is determined as the target identity identification information.

[0119] In a possible implementation manner, the controller 703 is further configured to:

[0120] If an input MV recording instruction is received, a subsequent operation of identifying the first identity information corresponding to the face appearing in the received video data according to the pre-stored correspondence between the face and the identity information is performed;

[0121] If the MV recording instruction is not received, the display is controlled to display the video data.

[0122] In a possible implementation manner, the controller 703 is further configured to:

[0123] After receiving the MV recording instruction, the input MV selection operation is received, the selected first target MV is determined, and each frame in the first target MV is superimposed with the target object extracted from the corresponding video frame in sequence, and the display is controlled to display the first target MV after superimposing the target object.

[0124] In a possible implementation manner, the controller 703 is further configured to:

[0125] Receiving first audio and video data sent by an audio and video acquisition device, and acquiring audio data and the video data in the first audio and video data;

[0126] The display device further includes:

[0127] Communicator 704, the communicator is used to send the audio data and the first target MV with the target object superimposed thereon to the cloud server if the MV recording instruction is received; if the MV recording instruction is not received, the audio data and the video data are sent to the cloud server.

[0128] In a possible implementation, the communicator 704 is further configured to receive second audio and video data sent by the cloud server and sent by an electronic device that is making a video call with the display device;

[0129] The display 701 is further used to display the second video data in the second audio and video data;

[0130] The display device further includes:

[0131] The speaker 705 is used to play the second audio data in the second audio and video data.

[0132] In a possible implementation manner, the controller 703 is further configured to:

[0133] In response to a received barrage instruction, wherein the barrage instruction carries a barrage form and barrage content; according to the barrage form, the display is controlled to display the barrage content in a first target MV after superimposing the target object.

[0134] In a possible implementation manner, the controller 703 is further configured to:

[0135] If the last frame of the first target MV has been superimposed with the extracted first target object, and the second target object is extracted from the video frame and no MV switching instruction is received, then starting from the first frame of the first target MV, each frame of the first target MV is sequentially superimposed with the second target object extracted from the corresponding video frame;

[0136] If the last frame in the first target MV has been superimposed with the extracted first target object, and the second target object is extracted in the video frame and an MV switching instruction is received, the second target MV after switching is determined, and each frame in the second target MV is superimposed with the second target object extracted from the corresponding video frame in sequence, and the display is controlled to display the second target MV after superimposing the target object.

[0137] In a possible implementation manner, the controller 703 is further configured to:

[0138] After extracting the target object, identifying the target posture information of the target object, and determining whether the target posture information is pre-saved posture information;

[0139] If yes, determining the target instruction corresponding to the target posture information according to the instruction corresponding to the pre-stored posture information;

[0140] According to the target instruction, the target special effect corresponding to the target instruction is determined and the display is controlled to display.

[0141] Figure 8 This is a schematic diagram of the structure of an electronic device provided by the present application. Based on the above embodiments, the present application also provides an electronic device, such as Figure 8 As shown, it includes: a processor 801, a communication interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804;

[0142] The memory 803 stores a computer program. When the program is executed by the processor 801, the processor 801 performs the following steps:

[0143] Identify first identity information corresponding to a face appearing in the received video data according to a pre-stored correspondence between a face and identity information;

[0144] The target identity information of the target face to be extracted from the video is determined, a target area where a target object containing the target identity information is located is determined in a video frame in the video data, and the target object is extracted from the target area.

[0145] In a possible implementation manner, determining the target identity information of the target face for video extraction includes:

[0146] An input selection operation is received, wherein the selection operation carries selected second identity identification information, and the second identity identification information is determined as the target identity identification information.

[0147] In a possible implementation, before identifying the first identity information corresponding to the face appearing in the received video data according to the pre-stored correspondence between the face and the identity information, the method further includes:

[0148] If an input MV recording instruction is received, a subsequent operation of identifying the first identity information corresponding to the face appearing in the received video data according to the pre-stored correspondence between the face and the identity information is performed;

[0149] If the MV recording instruction is not received, the video data is displayed.

[0150] In a possible implementation manner, after receiving the input MV recording instruction, the method further includes:

[0151] If after receiving the MV recording instruction, an input MV selection operation is received, the selected first target MV is determined, and each frame in the first target MV is superimposed with the target object extracted from the corresponding video frame in sequence, and the first target MV after superimposing the target object is displayed.

[0152] In a possible implementation manner, before receiving the MV recording instruction, the method further includes:

[0153] Receiving first audio and video data containing the target object, and acquiring audio data and the video data in the first audio and video data;

[0154] If the MV recording instruction is received, the audio data and the first target MV with the target object superimposed thereon are sent to the cloud server; if the MV recording instruction is not received, the audio data and the video data are sent to the cloud server.

[0155] In a possible implementation, the method further includes:

[0156] Receiving second audio and video data sent by the cloud server and sent by an electronic device that is having a video call with the display device;

[0157] Displaying second video data in the second audio and video data;

[0158] Play the second audio data in the second audio and video data.

[0159] In a possible implementation, the method further includes:

[0160] Responding to a received barrage instruction, wherein the barrage instruction carries a barrage form and barrage content;

[0161] According to the bullet screen format, the bullet screen content is displayed in the first target MV after the target object is superimposed.

[0162] In a possible implementation, the method further includes:

[0163] If the last frame of the first target MV has been superimposed with the extracted first target object, and the second target object is extracted from the video frame and no MV switching instruction is received, then starting from the first frame of the first target MV, each frame of the first target MV is sequentially superimposed with the second target object extracted from the corresponding video frame;

[0164] If the last frame in the first target MV has been superimposed with the extracted first target object, and the second target object is extracted in the video frame and an MV switching instruction is received, the second target MV after switching is determined, and each frame in the second target MV is superimposed with the second target object extracted from the corresponding video frame in sequence, and the second target MV after superimposing the target object is displayed.

[0165] In a possible implementation, the method further includes:

[0166] After extracting the target object, identifying the target posture information of the target object, and determining whether the target posture information is pre-saved posture information;

[0167] If yes, determining the target instruction corresponding to the target posture information according to the instruction corresponding to the pre-stored posture information;

[0168] According to the target instruction, the target special effect corresponding to the target instruction is determined and displayed.

[0169] Since the principle of solving the problem by the above electronic device is similar to that of the MV recording method, the implementation of the above electronic device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0170] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0171] The communication interface 802 is used for communication between the above electronic device and other devices.

[0172] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0173] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; it can also be a digital signal processing processor (Digital Signal Processing, DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0174] On the basis of the above embodiments, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program executable by a processor, and when the program runs on the processor, the processor implements the following steps when executing:

[0175] Identify first identity information corresponding to a face appearing in the received video data according to a pre-stored correspondence between a face and identity information;

[0176] The target identity information of the target face to be extracted from the video is determined, a target area where a target object containing the target identity information is located is determined in a video frame in the video data, and the target object is extracted from the target area.

[0177] In a possible implementation manner, determining the target identity information of the target face for video extraction includes:

[0178] An input selection operation is received, wherein the selection operation carries selected second identity identification information, and the second identity identification information is determined as the target identity identification information.

[0179] In a possible implementation, before identifying the first identity information corresponding to the face appearing in the received video data according to the pre-stored correspondence between the face and the identity information, the method further includes:

[0180] If an input MV recording instruction is received, a subsequent operation of identifying the first identity information corresponding to the face appearing in the received video data according to the pre-stored correspondence between the face and the identity information is performed;

[0181] If the MV recording instruction is not received, the video data is displayed.

[0182] In a possible implementation manner, after receiving the input MV recording instruction, the method further includes:

[0183] After receiving the MV recording instruction, the input MV selection operation is received, the selected first target MV is determined, and each frame in the first target MV is superimposed with the target object extracted from the corresponding video frame in sequence, and the first target MV after superimposing the target object is displayed.

[0184] In a possible implementation manner, before receiving the MV recording instruction, the method further includes:

[0185] Receiving first audio and video data containing the target object, and acquiring audio data and the video data in the first audio and video data;

[0186] If the MV recording instruction is received, the audio data and the first target MV with the target object superimposed thereon are sent to the cloud server; if the MV recording instruction is not received, the audio data and the video data are sent to the cloud server.

[0187] In a possible implementation, the method further includes:

[0188] Receiving second audio and video data sent by the cloud server and sent by an electronic device that is having a video call with the display device;

[0189] Displaying second video data in the second audio and video data;

[0190] Play the second audio data in the second audio and video data.

[0191] In a possible implementation, the method further includes:

[0192] Responding to a received barrage instruction, wherein the barrage instruction carries a barrage form and barrage content;

[0193] According to the bullet screen format, the bullet screen content is displayed in the first target MV after the target object is superimposed.

[0194] In a possible implementation, the method further includes:

[0195] If the last frame of the first target MV has been superimposed with the extracted first target object, and the second target object is extracted from the video frame and no MV switching instruction is received, then starting from the first frame of the first target MV, each frame of the first target MV is sequentially superimposed with the second target object extracted from the corresponding video frame;

[0196] If the last frame in the first target MV has been superimposed with the extracted first target object, and the second target object is extracted in the video frame and an MV switching instruction is received, the second target MV after switching is determined, and each frame in the second target MV is superimposed with the second target object extracted from the corresponding video frame in sequence, and the second target MV after superimposing the target object is displayed.

[0197] In a possible implementation, the method further includes:

[0198] After extracting the target object, identifying the target posture information of the target object, and determining whether the target posture information is pre-saved posture information;

[0199] If yes, determining the target instruction corresponding to the target posture information according to the instruction corresponding to the pre-stored posture information;

[0200] According to the target instruction, the target special effect corresponding to the target instruction is determined and displayed.

[0201] Since the principle of solving the problem by the computer-readable medium provided above is similar to the MV recording method, after the processor executes the computer program in the computer-readable medium above, the implemented steps can refer to the other embodiments above, and the repeated parts will not be repeated.

[0202] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0203] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0204] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0205] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0206] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A MV recording method, characterized in that: The method comprises: Identify first identity information corresponding to a face appearing in the received video data according to a pre-stored correspondence between a face and identity information; Determine target identity information of a target face to be extracted from the video, determine a target region where a target object containing the target identity information is located in a video frame in the video data, and extract the target object from the target region; If the number of the first identity information is one, the target identity information of the target face to be extracted from the video includes: Determine the first identity information as the target identity information; If the number of the first identity information is at least two, the target identity information of the target face to be extracted from the video includes: receiving an input selection operation, wherein the selection operation carries selected second identity identification information, and determining the second identity identification information as the target identity identification information; If the selection operation is not received, the user is prompted to input a selection operation.

2. The method according to claim 1, characterized in that Before identifying the first identity information corresponding to the face appearing in the received video data according to the pre-stored correspondence between the face and the identity information, the method further includes: If an input MV recording instruction is received, a subsequent operation of identifying the first identity information corresponding to the face appearing in the received video data according to the pre-stored correspondence between the face and the identity information is performed; If the MV recording instruction is not received, the video data is displayed.

3. The method according to claim 2, characterized in that After receiving the input MV recording instruction, the method further includes: If after receiving the MV recording instruction, an input MV selection operation is received, the selected first target MV is determined, and each frame in the first target MV is superimposed with the target object extracted from the corresponding video frame in sequence, and the first target MV after superimposing the target object is displayed.

4. The method according to claim 3, characterized in that Before receiving the MV recording instruction, the method further includes: Receiving first audio and video data containing the target object, and acquiring audio data and the video data in the first audio and video data; If the MV recording instruction is received, the audio data and the first target MV with the target object superimposed thereon are sent to the cloud server; if the MV recording instruction is not received, the audio data and the video data are sent to the cloud server.

5. The method according to claim 4, characterized in that The method further comprises: Receiving second audio and video data sent by the cloud server and sent by an electronic device that is having a video call with the display device; Displaying second video data in the second audio and video data; Play the second audio data in the second audio and video data.

6. The method according to claim 3, characterized in that: The method further comprises: Responding to a received barrage instruction, wherein the barrage instruction carries a barrage form and barrage content; According to the bullet screen format, the bullet screen content is displayed in the first target MV after the target object is superimposed.

7. The method according to claim 3, characterized in that The method further comprises: If the last frame of the first target MV has been superimposed with the extracted first target object, and the second target object is extracted from the video frame and no MV switching instruction is received, then starting from the first frame of the first target MV, each frame of the first target MV is sequentially superimposed with the second target object extracted from the corresponding video frame; If the last frame in the first target MV has been superimposed with the extracted first target object, and the second target object is extracted in the video frame and an MV switching instruction is received, the second target MV after switching is determined, and each frame in the second target MV is superimposed with the second target object extracted from the corresponding video frame in sequence, and the second target MV after superimposing the target object is displayed.

8. The method according to claim 1, characterized in that The method further comprises: After extracting the target object, identifying the target posture information of the target object, and determining whether the target posture information is pre-saved posture information; If yes, determining the target instruction corresponding to the target posture information according to the instruction corresponding to the pre-stored posture information; According to the target instruction, the target special effect corresponding to the target instruction is determined and displayed.

9. A display device, characterized in that: The display device comprises: A display, the display being used to display the MV video containing the target object; A camera, wherein the camera is used to capture an image containing a target object; A controller, the controller being configured to: Controlling the display to display the MV video containing the target object; Wherein, the controller is further configured as: Identify first identity information corresponding to a face appearing in the received video data according to a pre-stored correspondence between a face and identity information; Determine target identity information of a target face to be extracted from the video, determine a target region where a target object containing the target identity information is located in a video frame in the video data, and extract the target object from the target region; If the number of the first identity information is one, the target identity information of the target face to be extracted from the video includes: Determine the first identity information as the target identity information; If the number of the first identity information is at least two, the target identity information of the target face to be extracted from the video includes: receiving an input selection operation, wherein the selection operation carries selected second identity identification information, and determining the second identity identification information as the target identity identification information; If the selection operation is not received, the user is prompted to input a selection operation.

Images