MV Recording Method, Device, Electronic Device and Computer Readable Storage Medium

Abstract

The present application discloses an MV recording method, apparatus, electronic device, and computer-readable storage medium, which are applied to a smart TV. The smart TV is provided with a camera. The MV recording method includes: when detecting that the screen recording function is turned on, performing face recognition on the karaoke user through the camera to obtain the face features corresponding to the karaoke user; dynamically selecting corresponding MV recording special effects for the karaoke user according to the face features and the song selection information of the karaoke user; and performing MV recording on the karaoke user during karaoke through the camera according to the MV recording special effects to obtain an MV recording result. The present application solves the technical problem of poor MV recording effect on smart TVs in the prior art.

Description

Technical Field

[0001] The present application relates to the field of image processing technologies, and in particular, to an MV recording method, apparatus, electronic device, and computer-readable storage medium. Background Art

[0002] With the in-depth development of online entertainment, online karaoke on electronic devices has become an important form of entertainment in people's lives. However, due to the fact that the original Android native screen recording mechanism only supports recording the content of the OSD (on-screen display) layer and does not support recording the content of the video layer. That is to say, when recording a video on a TV, only the content with pictures and menus on the TV can be recorded, and the content related to the video cannot be recorded, which will result in missing content in the recorded video. Therefore, the effect of MV (Music Video) recording on smart TVs is poor. Summary of the Invention

[0003] The main purpose of the present application is to provide an MV recording method, apparatus, electronic device, and computer-readable storage medium, aiming to solve the technical problem of poor MV recording effect on smart TVs in the prior art.

[0004] To achieve the above object, the present application provides an MV recording method applied to a smart TV, where the smart TV is provided with a camera, and the MV recording method includes:

[0005] When detecting that the screen recording function is turned on, perform face recognition on the karaoke user through the camera to obtain the face features corresponding to the karaoke user;

[0006] According to the face features and the song selection information of the karaoke user, dynamically select corresponding MV recording special effects for the karaoke user;

[0007] According to the MV recording special effects, perform MV recording on the karaoke user through the camera when the karaoke user is singing karaoke to obtain an MV recording result.

[0008] Optionally, the MV recording special effects include foreground special effects and background special effects.

[0009] The step of performing MV recording on the karaoke user through the camera according to the MV recording special effects to obtain an MV recording result includes:

[0010] When detecting that the karaoke user starts singing karaoke, start performing MV recording on the karaoke user through the camera to obtain an MV recording picture and MV recording audio data;

[0011] Add foreground and background special effects to the recorded MV video to obtain a special-effects recorded video, and save the special-effects recorded video and the MV recorded audio data;

[0012] Return to execute the steps: Start recording the MV of the karaoke user through the camera to obtain a recorded MV video, and detect whether the karaoke user has finished singing karaoke;

[0013] If it is detected that the karaoke user has finished singing karaoke, then use the saved special-effects recorded video and the MV recorded audio data together as the MV recording result.

[0014] Optionally, the step of adding foreground and background special effects to the recorded MV video to obtain a special-effects recorded video includes:

[0015] Perform face recognition on the recorded MV video to separate the foreground and background of the recorded MV video, and obtain a foreground portrait area image and a background area image;

[0016] Adjust the foreground portrait area image according to the foreground special effect to obtain an adjusted foreground image;

[0017] Adjust the background area image according to the background special effect to obtain an adjusted background image;

[0018] Fuse the adjusted foreground image and the adjusted background image to obtain the special-effects recorded video.

[0019] Optionally, the song selection information includes a song type label, and the face feature includes a face feature vector.

[0020] The step of dynamically selecting a corresponding MV recording special effect for the karaoke user according to the face feature and the song selection information of the karaoke user includes:

[0021] Concatenate the face feature vector and the song type label to obtain a special-effect selection representation vector;

[0022] Select the MV recording special effect required by the karaoke user according to the special-effect selection representation vector.

[0023] Optionally, the step of dynamically selecting a corresponding MV recording special effect for the karaoke user according to the face feature and the song selection information of the karaoke user includes:

[0024] Collect each lyric paragraph in the song selection information, and perform text recognition on each lyric paragraph respectively to obtain the text semantic information corresponding to each lyric paragraph;

[0025] According to the face features and each piece of text semantic information, corresponding video recording special effects are selected for each lyric paragraph to obtain the MV recording special effects.

[0026] Optionally, the video recording special effects include foreground image special effects and background image special effects, the face features include face feature vectors, and the text semantic information includes text semantic vectors.

[0027] The step of selecting corresponding video recording special effects for each lyric paragraph according to the face features and each piece of text semantic information includes:

[0028] Obtain the user portrait representation vector corresponding to the KTV user, splice the user portrait representation vector and the text semantic vector to obtain a first spliced vector.

[0029] Splice the face feature vector and the text semantic vector to obtain a second spliced vector.

[0030] Classify the first spliced vector to obtain a first classification label, and classify the second spliced vector to obtain a second classification label.

[0031] Select the background image special effects according to the first classification label, and select the foreground image special effects according to the second classification label.

[0032] Optionally, before the step of performing face recognition on the KTV user through the camera to obtain the face features corresponding to the KTV user when detecting that the screen recording function is turned on, the MV recording method further includes:

[0033] Collect the voice information of the KTV user.

[0034] If the voice information matches the screen recording start voice command, control the screen recording function to turn on.

[0035] If the voice information does not match the screen recording start voice command, determine whether the KTV user enters the KTV preparation stage.

[0036] If the KTV user enters the KTV preparation stage, control the screen recording function to turn on.

[0037] To achieve the above object, the present application provides an MV recording device, which is applied to a smart TV. The smart TV is provided with a camera, and the MV recording device includes:

[0038] A face recognition module, configured to perform face recognition on a KTV user through the camera to obtain the face features corresponding to the KTV user when detecting that the screen recording function is turned on.

[0039] The MV recording special effect selection module is used to dynamically select corresponding MV recording special effects for the KTV user according to the face features and the song selection information of the KTV user;

[0040] The MV recording module is used to record an MV during the KTV user's singing through the camera according to the MV recording special effects, and obtain an MV recording result.

[0041] Optionally, the MV recording special effects include foreground special effects and background special effects, and the MV recording module is further used for:

[0042] When it is detected that the KTV user starts singing, start recording the MV of the KTV user through the camera to obtain an MV recording picture and MV recording audio data;

[0043] Add foreground special effects and background special effects to the MV recording picture to obtain a special effect recording picture, and save the special effect recording picture and the MV recording audio data;

[0044] Return to execute the steps: start recording the MV of the KTV user through the camera to obtain an MV recording picture, and detect whether the KTV user ends singing;

[0045] If it is detected that the KTV user ends singing, jointly use the saved special effect recording picture and the MV recording audio data as the MV recording result.

[0046] Optionally, the MV recording module is further used for:

[0047] Perform human face recognition on the MV recording picture to separate the foreground and background of the MV recording picture, and obtain a foreground human face area image and a background area image;

[0048] Adjust the foreground human face area image according to the foreground special effects to obtain a foreground adjusted image;

[0049] Adjust the background area image according to the background special effects to obtain a background adjusted image;

[0050] Fuse the foreground adjusted image and the background adjusted image to obtain the special effect recording picture.

[0051] Optionally, the song selection information includes a song type label, the face features include a face feature vector, and the MV recording special effect selection module is further used for:

[0052] Concatenate the face feature vector and the song type label to obtain a special effect selection characterization vector;

[0053] Select a characterization vector according to the special effects, and select the MV recording special effects for the KTV user's requirements.

[0054] Optionally, the MV recording special effect selection module is further configured to:

[0055] Collect each lyric paragraph in the song selection information, perform text recognition on each lyric paragraph respectively, and obtain the text semantic information corresponding to each lyric paragraph;

[0056] According to the face features and each text semantic information, select corresponding video recording special effects for each lyric paragraph respectively to obtain the MV recording special effects.

[0057] Optionally, the video recording special effects include foreground image special effects and background image special effects, the face features include face feature vectors, the text semantic information includes text semantic vectors, and the MV recording special effect selection module is further configured to:

[0058] Obtain the user portrait characterization vector corresponding to the KTV user, splice the user portrait characterization vector and the text semantic vector to obtain a first spliced vector;

[0059] Splice the face feature vector and the text semantic vector to obtain a second spliced vector;

[0060] Classify the first spliced vector to obtain a first classification label, and classify the second spliced vector to obtain a second classification label;

[0061] Select the background image special effect according to the first classification label, and select the foreground image special effect according to the second classification label.

[0062] Optionally, the MV recording device is further configured to:

[0063] Collect the voice information of the KTV user;

[0064] If the voice information matches the screen recording start voice command, control the screen recording function to start;

[0065] If the voice information does not match the screen recording start voice command, determine whether the KTV user enters the KTV preparation stage;

[0066] If the KTV user enters the KTV preparation stage, control the screen recording function to start.

[0067] The present application also provides an electronic device, which includes: a memory, a processor, and a program of the acoustic model deployment optimization method stored on the memory and executable on the processor. When the program of the acoustic model deployment optimization method is executed by the processor, the steps of the acoustic model deployment optimization method as described above can be implemented.

[0068] The present application also provides a computer-readable storage medium, on which a program for implementing the MV recording method is stored. When the program of the MV recording method is executed by the processor, the steps of the MV recording method as described above are implemented.

[0069] The present application also provides a computer program product, including a computer program, which when executed by the processor implements the steps of the MV recording method as described above.

[0070] The present application provides an MV recording method, device, electronic device, and computer-readable storage medium, which are applied to a smart TV. The smart TV is provided with a camera. Specifically, when it is detected that the screen recording function is turned on, face recognition is performed on the karaoke user through the camera to obtain the face features corresponding to the karaoke user; according to the face features and the song selection information of the karaoke user, corresponding MV recording special effects are dynamically selected for the karaoke user; according to the MV recording special effects, MV recording is performed through the camera when the karaoke user is singing karaoke to obtain an MV recording result. That is, the present application is provided with a camera on the smart TV, so that face recognition can be performed using the camera, and then according to the face recognition result and the song selection information of the karaoke user, matching MV recording special effects are dynamically selected for the karaoke user. According to the MV recording special effects, the MV during karaoke singing is recorded through the camera, thus achieving the purpose of MV recording on the smart TV, overcoming the technical defect in the prior art that the original Android native screen recording mechanism only supports recording the content of the OSD layer and does not support recording the content of the video layer, resulting in poor MV recording effects, and improving the MV recording effect. Description of the Drawings

[0071] The drawings here are incorporated into the specification and form a part of this specification, showing the embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0073] Figure 1 It is a schematic flowchart of the first embodiment of the MV recording method of the present application;

[0074] Figure 2 This is a schematic flowchart of the second embodiment of the MV recording method of this application;

[0075] Figure 3 This is a schematic diagram of the device structure of the hardware operating environment involved in the MV recording method in the embodiments of this application.

[0076] The realization of the purpose, functional characteristics and advantages of this application will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0077] To make the above objects, features and advantages of this application more obvious and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of this application without creative efforts shall fall within the protection scope of this application.

[0078] Embodiment 1

[0079] The embodiments of this application provide an MV recording method. In the first embodiment of the MV recording method of this application, with reference to Figure 1 , it is applied to a smart TV. The smart TV is provided with a camera. The MV recording method includes:

[0080] Step S10, when it is detected that the screen recording function is turned on, perform face recognition on the karaoke user through the camera to obtain the face features corresponding to the karaoke user;

[0081] Step S20, according to the face features and the song selection information of the karaoke user, dynamically select corresponding MV recording special effects for the karaoke user;

[0082] Step S30, according to the MV recording special effects, perform MV recording on the karaoke user when the karaoke user is singing through the camera to obtain an MV recording result.

[0083] In this embodiment, it should be noted that the smart TV is provided with a connected camera and microphone. The facial feature can be the key point feature on the human face, or the low-dimensional vector representation of the key point feature, that is, the embedding. The song selection information can be the lyric paragraph text or the lyric type label. The MV recording special effects include foreground special effects and background special effects. The foreground special effects are the facial special effects in MV recording, such as beauty effects, face slimming, filters, and stickers. The background special effects are the background environment special effects other than the portrait in MV recording, which can be background replacement or brightness adjustment of the background, etc.

[0084] As an example, steps S10 to S30 include: when detecting that the user enables the screen recording function, capturing the karaoke user through the camera to obtain a user image; performing face recognition on the user image to identify the key point feature of the face in the user image, where the key point information can be the coordinates of the key points on the human face; performing feature transformation on the key point feature to map the key point feature to a preset vector space to obtain the facial feature corresponding to the karaoke user; classifying the face of the karaoke user according to the facial feature to obtain a face classification label; selecting MV recording special effects from a preset MV recording special effects library according to the face classification label and the song type selected by the karaoke user; performing video recording and audio recording through the camera and the microphone when the karaoke user is singing karaoke, and adding the MV recording special effects to the recorded video to obtain an MV recording result. Further, a two-dimensional code corresponding to the MV recording result can be generated and transmitted to the smart terminal.

[0085] Among them, the MV recording special effects include foreground special effects and background special effects. The step of performing MV recording on the karaoke user through the camera according to the MV recording special effects to obtain an MV recording result includes:

[0086] Step S31, when detecting that the karaoke user starts singing karaoke, start performing MV recording on the karaoke user through the camera to obtain an MV recording screen and MV recording audio data;

[0087] Step S32, adding foreground special effects and background special effects to the MV recording screen to obtain a special effect recording screen, and saving the special effect recording screen and the MV recording audio data;

[0088] Step S33, return and execute the step: start performing MV recording on the karaoke user through the camera to obtain an MV recording screen, and detect whether the karaoke user ends singing karaoke;

[0089] Step S34: If it is detected that the karaoke user has ended karaoke, then use the saved special-effect recorded video and the MV recorded audio data together as the MV recording result.

[0090] As an example, steps S31 to S34 include: When it is detected that the karaoke user starts karaoke, start recording an MV of the karaoke user through the camera and the microphone to obtain the MV recorded video of the current time step and the MV recorded audio data of the current time step; add a foreground special effect to the foreground image area of the MV recorded video, and add a background special effect to the background image area of the MV recorded video to obtain a special-effect recorded video, and save the special-effect recorded video and the MV recorded audio data; return to execute the steps: start recording an MV of the karaoke user through the camera to obtain the MV recorded video to record the MV recorded video of the next time step and the MV recorded audio data of the next time step, and detect whether the karaoke user has ended karaoke. If it is detected that the karaoke user has ended karaoke, then use all the saved special-effect recorded videos and the MV recorded audio data together as the MV recording result. The purpose of real-time recording of an MV on a smart TV is achieved.

[0091] Among them, the step of adding a foreground special effect and a background special effect to the MV recorded video to obtain a special-effect recorded video includes:

[0092] Step S321: Through performing face recognition on the MV recorded video, separate the foreground and the background of the MV recorded video to obtain a foreground portrait area image and a background area image;

[0093] Step S322: Adjust the foreground portrait area image according to the foreground special effect to obtain a foreground adjusted image;

[0094] Step S323: Adjust the background area image according to the background special effect to obtain a background adjusted image;

[0095] Step S324: Fuse the foreground adjusted image and the background adjusted image to obtain the special-effect recorded video.

[0096] As an example, steps S321 to S324 include: performing face recognition on the MV recording screen to determine the portrait image area in the MV recording screen; segmenting the portrait image area in the MV recording screen to obtain a foreground portrait area image, and using the other image areas in the MV recording screen except the foreground portrait area image as the background area image; adding the foreground special effect to the foreground portrait area image to obtain a foreground-adjusted image; replacing the background area image with the background special effect image corresponding to the background special effect to obtain a background-adjusted image; summing the pixel matrix corresponding to the foreground-adjusted image and the pixel matrix corresponding to the background-adjusted image to obtain a summed pixel matrix, and using the image corresponding to the summed pixel matrix as the special effect recording screen.

[0097] Among them, the song selection information includes a song type label, the face feature includes a face feature vector, and the step of dynamically selecting a corresponding MV recording special effect for the KTV user according to the face feature and the song selection information of the KTV user includes:

[0098] Step S21, splicing the face feature vector and the song type label to obtain a special effect selection representation vector;

[0099] Step S22, selecting the MV recording special effect required by the KTV user according to the special effect selection representation vector.

[0100] In this embodiment, it should be noted that the face feature vector is a low-dimensional vector representation of the face feature, that is, an embedding, the song type label is an identifier of the song type, and the song type can be classical or popular, etc.

[0101] As an example, steps S21 to S22 include: splicing the face feature vector and the song type label, and using the obtained spliced vector as the special effect selection representation vector; by inputting the special effect selection representation vector into a preset classifier, converting the special effect selection representation vector into a corresponding special effect selection label, and using the special effect selection label as an index to select the MV recording special effect required by the KTV user in a preset MV recording special effect library.

[0102] Among them, before the step of performing face recognition on the KTV user through the camera to obtain the face feature corresponding to the KTV user when the screen recording function is detected to be turned on, the MV recording method further includes:

[0103] Step A10, collecting the voice information of the KTV user;

[0104] Step A20, if the voice information matches the screen recording start voice command, controlling the screen recording function to be turned on;

[0105] Step A30: If the voice information does not match the voice command for starting screen recording, determine whether the KTV user enters the KTV preparation stage.

[0106] Step A40: If the KTV user enters the KTV preparation stage, control the start of the screen recording function.

[0107] In this embodiment, it should be noted that the KTV user can control the start of the screen recording function through a remote control or by voice control.

[0108] As an example, steps A10 to A40 include: collecting the voice information of the KTV user, performing voice recognition on the voice information to determine whether the voice information is a voice command for starting screen recording. If so, it is determined that the voice information matches the voice command for starting screen recording, and the screen recording function is controlled to start; if not, it is determined that the voice information does not match the voice command for starting screen recording, and it is judged whether the current display screen of the smart TV is a KTV countdown screen. If the current display screen of the smart TV is a KTV countdown screen, it is determined that the KTV user enters the KTV preparation stage, and the screen recording function is controlled to start; if the current display screen of the smart TV is not a KTV countdown screen, it is determined that the KTV user does not enter the KTV preparation stage, and the screen recording function is not started.

[0109] The embodiment of the present application provides an MV recording method applied to a smart TV. The smart TV is provided with a camera. Specifically, when it is detected that the screen recording function is started, face recognition is performed on the KTV user through the camera to obtain the face features corresponding to the KTV user; according to the face features and the song selection information of the KTV user, corresponding MV recording special effects are dynamically selected for the KTV user; according to the MV recording special effects, MV recording is performed through the camera when the KTV user is singing KTV to obtain an MV recording result. That is, in the embodiment of the present application, a camera is provided on the smart TV, so that face recognition can be performed using the camera, and then according to the face recognition result and the song selection information of the KTV user, matching MV recording special effects are dynamically selected for the KTV user. According to the MV recording special effects, the MV during KTV singing is recorded through the camera, thus achieving the purpose of MV recording on the smart TV, overcoming the technical defect in the prior art that the original Android native screen recording mechanism only supports recording the content of the OSD layer and does not support recording the content of the video layer, resulting in poor MV recording effects, and improving the MV recording effect.

[0110] Embodiment 2

[0111] Further, referring to Figure 2, in another embodiment of the present application, for the same or similar content as in the above-mentioned Embodiment 1, reference can be made to the above introduction and will not be elaborated hereinafter. On this basis, the step of dynamically selecting corresponding MV recording special effects for the KTV user according to the face feature and the song selection information of the KTV user includes:

[0112] Step B10, collect each lyric paragraph in the song selection information, and perform text recognition on each lyric paragraph respectively to obtain the text semantic information corresponding to each lyric paragraph;

[0113] Step B20, select corresponding video recording special effects for each lyric paragraph respectively according to the face feature and each text semantic information to obtain the MV recording special effects.

[0114] In this embodiment, it should be noted that the song selection information includes at least one lyric paragraph. For example, all the lyrics displayed on the same TV display screen can be used as one lyric paragraph, and the text semantic information can be a text semantic vector, which is used to characterize the semantic features of the lyric paragraph.

[0115] As an example, steps B10 to B20 include: collecting each lyric paragraph in the song selection information, and respectively performing text semantic recognition on each lyric paragraph according to a preset text semantic recognition model to obtain the text semantic vectors corresponding to each lyric paragraph; splicing the face feature vectors with each text semantic vector respectively to obtain each second special effect selection representation vector; classifying each second special effect selection representation vector respectively to obtain the special effect selection labels corresponding to each lyric paragraph; according to the special effect selection labels corresponding to each lyric paragraph; respectively select the video recording special effects corresponding to each lyric paragraph in a preset MV recording special effect library with the special effect selection labels corresponding to each lyric paragraph; and jointly use the video recording special effects of each lyric paragraph as the MV recording special effects. The purpose of selecting a matching video recording special effect for each lyric paragraph is realized, the fineness of adding MV recording special effects during MV recording is improved, and the matching degree between the MV content and the added MV recording special effects is higher.

[0116] Among them, the video recording special effects include foreground image special effects and background image special effects, the face feature includes a face feature vector, the text semantic information includes a text semantic vector, and the step of selecting corresponding video recording special effects for each lyric paragraph respectively according to the face feature and each text semantic information includes:

[0117] Step B21, obtain the user portrait representation vector corresponding to the KTV user, and splice the user portrait representation vector and the text semantic vector to obtain a first splicing vector;

[0118] Step B22, concatenating the facial feature vector and the text semantic vector to obtain a second concatenated vector;

[0119] Step B23, classifying the first splicing vector to obtain a first classification label, and classifying the second splicing vector to obtain a second classification label;

[0120] Step B24, selecting the background image special effects according to the first classification label, and selecting the foreground image special effects according to the second classification label.

[0121] In this embodiment, it should be noted that the user portrait characterization vector is a feature vector that characterizes the user portrait, which is composed of different user feature values, wherein the user features may be the user's age, education level, hobby type, etc.

[0122] As an example, step B23 to step B24 include: inputting the first splicing vector into a preset first classification model, mapping the first splicing vector into a corresponding classification label, and obtaining a first classification label; inputting the second splicing vector into a preset second classification model, mapping the second splicing type into a corresponding classification label, and obtaining a second classification label; using the first classification label as an index, selecting background image effects from a preset MV recording special effects library, and using the second classification label as an index, selecting foreground image effects from a preset MV recording special effects library. The purpose of selecting background effects for MV recording based on user portraits and song selection information, and selecting foreground effects for MV recording based on facial features and song selection information is achieved, providing more decision-making basis for the selection of MV recording special effects, and improving the accuracy of automatic selection of MV recording special effects.

[0123] The embodiment of the present application provides a method for automatically selecting MV recording special effects, that is, collecting each lyric paragraph in the song selection information, performing text recognition on each lyric paragraph respectively, and obtaining the text semantic information corresponding to each lyric paragraph; according to the facial features and each text semantic information, selecting the corresponding video recording special effects for each lyric paragraph respectively, and obtaining the MV recording special effects. The purpose of automatically selecting MV recording special effects based on the lyric paragraph as the granularity is achieved, and the selection accuracy of MV recording special effects is improved. Therefore, when the MV is played, as the lyrics are different, the special effects screen that best matches the current lyrics will be automatically played, thereby improving the effect of MV recording.

[0124] Embodiment 3

[0125] The present application also provides a MV recording device, which is applied to a smart TV, wherein the smart TV is provided with a camera, and the MV recording device comprises:

[0126] A face recognition module, which is used to perform face recognition on the KTV user through the camera when the screen recording function is detected to be turned on, so as to obtain the face features corresponding to the KTV user;

[0127] An MV recording special effect selection module, which is used to dynamically select corresponding MV recording special effects for the KTV user according to the face features and the song selection information of the KTV user;

[0128] An MV recording module, which is used to perform MV recording on the KTV user through the camera when the KTV user is singing according to the MV recording special effects, so as to obtain an MV recording result.

[0129] Optionally, the MV recording special effects include foreground special effects and background special effects, and the MV recording module is further used for:

[0130] When it is detected that the KTV user starts singing, start performing MV recording on the KTV user through the camera to obtain an MV recording picture and MV recording audio data;

[0131] Add foreground special effects and background special effects to the MV recording picture to obtain a special effect recording picture, and save the special effect recording picture and the MV recording audio data;

[0132] Return to execute the steps: start performing MV recording on the KTV user through the camera to obtain an MV recording picture, and detect whether the KTV user finishes singing;

[0133] If it is detected that the KTV user finishes singing, use the saved special effect recording picture and the MV recording audio data together as the MV recording result.

[0134] Optionally, the MV recording module is further used for:

[0135] Perform portrait recognition on the MV recording picture to separate the foreground and background of the MV recording picture, so as to obtain a foreground portrait area image and a background area image;

[0136] Adjust the foreground portrait area image according to the foreground special effects to obtain a foreground adjusted image;

[0137] Adjust the background area image according to the background special effects to obtain a background adjusted image;

[0138] Fuse the foreground adjusted image and the background adjusted image to obtain the special effect recording picture.

[0139] Optionally, the song selection information includes a song type label, the face features include a face feature vector, and the MV recording special effect selection module is further used for:

[0140] Concatenate the face feature vector and the song type label to obtain a special effect selection representation vector;

[0141] Select the MV recording special effects required by the KTV user according to the special effect selection representation vector.

[0142] Optionally, the MV recording special effect selection module is further configured to:

[0143] Collect each lyric paragraph in the song selection information, and perform text recognition on each lyric paragraph respectively to obtain the text semantic information corresponding to each lyric paragraph;

[0144] Select corresponding video recording special effects for each lyric paragraph according to the face features and the text semantic information to obtain the MV recording special effects.

[0145] Optionally, the video recording special effects include foreground image special effects and background image special effects, the face features include face feature vectors, the text semantic information includes text semantic vectors, and the MV recording special effect selection module is further configured to:

[0146] Obtain the user portrait representation vector corresponding to the KTV user, and concatenate the user portrait representation vector and the text semantic vector to obtain a first concatenated vector;

[0147] Concatenate the face feature vector and the text semantic vector to obtain a second concatenated vector;

[0148] Classify the first concatenated vector to obtain a first classification label, and classify the second concatenated vector to obtain a second classification label;

[0149] Select the background image special effects according to the first classification label, and select the foreground image special effects according to the second classification label.

[0150] Optionally, the MV recording device is further configured to:

[0151] Collect the voice information of the KTV user;

[0152] If the voice information matches the screen recording start voice command, control the screen recording function to start;

[0153] If the voice information does not match the screen recording start voice command, determine whether the KTV user enters the KTV preparation stage;

[0154] If the KTV user enters the KTV preparation stage, control the screen recording function to start.

[0155] The MV recording device provided by this application adopts the MV recording method in the above-mentioned embodiment, and solves the technical problem of poor MV recording effect on smart TVs. Compared with the prior art, the beneficial effects of the MV recording device provided by the embodiments of this application are the same as those of the MV recording method provided by the above-mentioned embodiment, and other technical features in this MV recording device are the same as the features disclosed in the method of the above-mentioned embodiment, which will not be elaborated here.

[0156] Embodiment 4

[0157] The embodiments of this application provide an electronic device. The electronic device can be a smart TV. The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the MV recording method in Embodiment 1 above.

[0158] Next, refer to Figure 3 , which shows a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0159] As Figure 3 shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the programs stored in the read-only memory (ROM) or the programs loaded from the storage device into the random access memory (RAM). In the RAM, various programs and data required for the operation of the electronic device are also stored. The processing device, ROM, and RAM are connected to each other through a bus. The input / output (I / O) interface is also connected to the bus.

[0160] Generally, the following systems can be connected to the I / O interface: input devices including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices including, for example, a magnetic tape, a hard disk, etc.; and a communication device. The communication device can allow the electronic device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. Instead, more or fewer systems can be implemented or had.

[0161] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device, or installed from the storage device, or installed from the ROM. When the computer program is executed by the processing device, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0162] The electronic device provided in this application adopts the MV recording method in the above embodiment, and solves the technical problem of poor MV recording effect on smart TVs. Compared with the prior art, the beneficial effects of the electronic device provided in the embodiment of this application are the same as the beneficial effects of the MV recording method provided in the above Embodiment 1, and other technical features in this electronic device are the same as the features disclosed in the above embodiment method, and will not be elaborated here.

[0163] It should be understood that each part of the present disclosure can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0164] As described above, only the specific implementation manners of this application are provided, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claimed rights.

[0165] Embodiment 5

[0166] This embodiment provides a computer-readable storage medium with computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the MV recording method in the above Embodiment 1.

[0167] The computer-readable storage medium provided by the embodiments of the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0168] The above computer-readable storage medium may be included in an electronic device; or it may exist independently without being assembled into the electronic device.

[0169] The above computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device: when detecting that the screen recording function is turned on, performs face recognition on the KTV user through the camera to obtain the face features corresponding to the KTV user; based on the face features and the song selection information of the KTV user, dynamically selects corresponding MV recording special effects for the KTV user; and based on the MV recording special effects, performs MV recording through the camera when the KTV user is singing KTV to obtain an MV recording result.

[0170] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++, and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0171] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0172] The modules involved in the embodiments described in the present disclosure can be implemented in software or in hardware. In this case, the name of the module does not constitute a limitation on the unit itself.

[0173] The computer-readable storage medium provided in the present application stores computer-readable program instructions for executing the above MV recording method, and solves the technical problem of poor MV recording effect on smart TVs. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the embodiments of the present application are the same as those of the MV recording method provided in the above embodiments, and will not be elaborated here.

[0174] Embodiment Six

[0175] The present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the MV recording method as described above are implemented.

[0176] The computer program product provided in the present application solves the technical problem of poor MV recording effect on smart TVs. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiments of the present application are the same as those of the MV recording method provided in the above embodiments, and will not be elaborated here.

[0177] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural or equivalent process transformation made using the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent scope of the present application.

Claims

1. An MV recording method, characterized in that, Applied to a smart TV, the smart TV is provided with a camera, and the MV recording method includes: When it is detected that the screen recording function is turned on, perform face recognition on the karaoke user through the camera to obtain the face features corresponding to the karaoke user; According to the face features and the song selection information of the karaoke user, dynamically select the corresponding MV recording special effects for the karaoke user in a preset MV recording special effects library; According to the MV recording special effects, perform MV recording on the karaoke user through the camera when the karaoke user is singing karaoke to obtain an MV recording result; Among them, the step of dynamically selecting the corresponding MV recording special effects for the karaoke user in the preset MV recording special effects library according to the face features and the song selection information of the karaoke user includes: Collect each lyric paragraph in the song selection information, and perform text recognition on each lyric paragraph respectively to obtain the text semantic information corresponding to each lyric paragraph; According to the face features and each text semantic information, select the corresponding video recording special effects for each lyric paragraph respectively to obtain the MV recording special effects; Among them, the video recording special effects include foreground image special effects and background image special effects, the face features include face feature vectors, and the text semantic information includes text semantic vectors, The step of selecting the corresponding video recording special effects for each lyric paragraph according to the face features and each text semantic information includes: Obtain the user portrait representation vector corresponding to the karaoke user, splice the user portrait representation vector and the text semantic vector to obtain a first spliced vector; Splice the face feature vector and the text semantic vector to obtain a second spliced vector; Classify the first spliced vector to obtain a first classification label, and classify the second spliced vector to obtain a second classification label; According to the first classification label, select the background image special effect, and according to the second classification label, select the foreground image special effect.

2. The MV recording method according to claim 1, wherein The MV recording special effects include foreground special effects and background special effects, The step of performing MV recording on the karaoke user through the camera according to the MV recording special effects to obtain an MV recording result includes: When it is detected that the karaoke user starts singing karaoke, start performing MV recording on the karaoke user through the camera to obtain an MV recording picture and MV recording audio data; Add foreground special effects and background special effects to the MV recording picture to obtain a special effect recording picture, and save the special effect recording picture and the MV recording audio data; Return to execute the step: start performing MV recording on the karaoke user through the camera to obtain an MV recording picture, and detect whether the karaoke user ends singing karaoke; If it is detected that the karaoke user ends singing karaoke, then use the saved special effect recording picture and the MV recording audio data together as the MV recording result.

3. The MV recording method according to claim 2, wherein The step of adding foreground special effects and background special effects to the MV recording picture to obtain a special effect recording picture includes: By performing face recognition on the MV recording screen, separating the foreground and background of the MV recording screen, a foreground portrait area image and a background area image are obtained; According to the foreground special effect, the foreground portrait area image is adjusted to obtain a foreground adjusted image; According to the background special effect, the background area image is adjusted to obtain a background adjusted image; The foreground adjusted image and the background adjusted image are fused to obtain the special effect recording screen.

4. The MV recording method according to claim 1, wherein The song selection information includes song type tags, and the face features include face feature vectors. The step of dynamically selecting corresponding MV recording special effects for the KTV user according to the face features and the song selection information of the KTV user includes: The face feature vector and the song type tag are spliced to obtain a special effect selection representation vector; According to the special effect selection representation vector, the MV recording special effects required by the KTV user are selected.

5. The MV recording method according to claim 1, characterized in that Before the step of performing face recognition on the KTV user through the camera to obtain the face features corresponding to the KTV user when it is detected that the screen recording function is turned on, the MV recording method further includes: Collecting the voice information of the KTV user; If the voice information matches the screen recording start voice command, controlling the screen recording function to start; If the voice information does not match the screen recording start voice command, determining whether the KTV user enters the KTV preparation stage; If the KTV user enters the KTV preparation stage, controlling the screen recording function to start.

6. An MV recording device, characterized in that, Applied to a smart TV, the smart TV is provided with a camera, and the MV recording device includes: A face recognition module, configured to perform face recognition on the KTV user through the camera to obtain the face features corresponding to the KTV user when it is detected that the screen recording function is turned on; An MV recording special effect selection module, configured to dynamically select corresponding MV recording special effects for the KTV user in a preset MV recording special effect library according to the face features and the song selection information of the KTV user; An MV recording module, configured to perform MV recording on the KTV user during KTV singing through the camera according to the MV recording special effects to obtain an MV recording result; Among them, the MV recording special effect selection module is specifically used for: Collecting each lyric paragraph in the song selection information, respectively performing text recognition on each lyric paragraph to obtain the text semantic information corresponding to each lyric paragraph; According to the face features and each text semantic information, respectively selecting corresponding video recording special effects for each lyric paragraph to obtain the MV recording special effects; Among them, the video recording special effects include foreground image special effects and background image special effects, the face features include face feature vectors, the text semantic information includes text semantic vectors, and the MV recording special effect selection module is specifically used for: Obtaining a user portrait representation vector corresponding to the KTV user, splicing the user portrait representation vector and the text semantic vector to obtain a first splicing vector; Splicing the face feature vector and the text semantic vector to obtain a second splicing vector; Classify the first splicing vector to obtain a first classification label, and classify the second splicing vector to obtain a second classification label; Select the background image special effect according to the first classification label, and select the foreground image special effect according to the second classification label.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the steps of the MV recording method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, A program for implementing the MV recording method is stored on the computer-readable storage medium, and the program for implementing the MV recording method is executed by a processor to implement the steps of the MV recording method according to any one of claims 1 to 5.

Images