Virtual object generation method and device and electronic equipment

Abstract

The invention provides a virtual object generation method and device and electronic equipment, relates to the technical field of data processing, and solves the technical problem that the generation efficiency of a virtual object is relatively low. An electronic device provides a pre-trained intelligent model to execute the method, an audio sample for training the intelligent model corresponds to an image label of a virtual object, and the method comprises the following steps: receiving audio data corresponding to a virtual object generation task; identifying sound features of the audio data; generating virtual object data matched with the sound features; and generating a virtual object corresponding to the virtual object generation task according to the virtual object data.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a method, device and electronic device for generating a virtual object. Background Art

[0002] Currently, users can create virtual objects in many application scenarios, for example, in games, virtual reality, virtual social applications, etc., users can create virtual objects with personalized images. In the process of creating virtual objects, several initial virtual object images are generally provided for users to choose from. Users usually need to manually adjust the image features of the initial virtual objects they choose, such as hairstyle, facial features, skin color, body shape, clothing and other details, to achieve the virtual object image desired by the user.

[0003] However, such manual operations are relatively complex for users. For example, if some players want to create a virtual object with beautiful and distinctive features, they need to perform more and more complex manual operations, which results in low efficiency in generating virtual objects and degrades user experience. Summary of the invention

[0004] The purpose of the present disclosure is to provide a method, device and electronic device for generating a virtual object, so as to alleviate the technical problem of low efficiency in generating virtual objects.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for generating a virtual object, wherein an electronic device provides a pre-trained intelligent model to execute the method, and an audio sample of the intelligent model is trained to correspond to an image label of a virtual object; the method comprises:

[0006] Receiving audio data corresponding to a virtual object generation task;

[0007] Identifying sound features of the audio data;

[0008] generating virtual object data matching the sound characteristics;

[0009] A virtual object corresponding to the virtual object generation task is generated according to the virtual object data.

[0010] In a second aspect, a virtual object generation device is provided, wherein a pre-trained intelligent model is provided by an electronic device to execute a method corresponding to the device, and an audio sample of the training intelligent model corresponds to an image label of a virtual object, and the device comprises:

[0011] A receiving module, used for receiving audio data corresponding to a virtual object generation task;

[0012] An identification module, used for identifying the sound features of the audio data;

[0013] A first generating module, used to generate virtual object data matching the sound feature;

[0014] The second generating module is used to generate a virtual object corresponding to the virtual object generating task according to the virtual object data.

[0015] In a third aspect, an embodiment of the present disclosure provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0016] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions cause the processor to execute the method described in the first aspect.

[0017] The embodiments of the present disclosure bring the following beneficial effects:

[0018] The disclosed embodiments provide a method, device, and electronic device for generating a virtual object. The method is executed by providing a pre-trained intelligent model through an electronic device. The audio samples used to train the intelligent model correspond to image labels of virtual objects. The method can receive audio data corresponding to a virtual object generation task, identify the sound features of the audio data, generate virtual object data that matches the sound features, and generate a virtual object corresponding to the virtual object generation task based on the virtual object data. In this solution, the intelligent model generated by training with audio samples and corresponding virtual object image labels can identify the sound features of any audio and automatically generate a virtual object that matches the sound features, thereby reducing the difficulty and complexity of manual operations for users to generate virtual objects through manual operations, thereby improving the generation efficiency of virtual objects and user experience, and alleviating the technical problem of low virtual object generation efficiency.

[0019] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure is shown;

[0022] Figure 2 A schematic diagram of the structure of a mobile phone provided by an embodiment of the present disclosure is shown;

[0023] Figure 3 A flowchart of a method for generating a virtual object provided by an embodiment of the present disclosure;

[0024] Figure 4 Another schematic diagram of a flow chart of a method for generating a virtual object provided by an embodiment of the present disclosure;

[0025] Figure 5 An example of the facial image effect of a virtual object generated in the virtual object generation method provided in the embodiment of the present disclosure;

[0026] Figure 6 Another example of the facial image effect of a virtual object generated in the virtual object generation method provided in the embodiment of the present disclosure;

[0027] Figure 7 Another example of the facial image effect of a virtual object generated in the virtual object generation method provided in the embodiment of the present disclosure;

[0028] Figure 8 Another example of the facial image effect of a virtual object generated in the virtual object generation method provided in the embodiment of the present disclosure;

[0029] Fig. 9 A schematic diagram of the structure of a virtual object generation device provided by an embodiment of the present disclosure;

[0030] Fig.10 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0032] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present disclosure are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0033] At present, face-pinching technology is often used in image customization systems in games, virtual reality, social media and other applications, allowing users to create personalized virtual images by adjusting facial and other image features. Exemplarily, in the creation system of game characters (virtual objects), several initial models are generally provided, and players usually need to manually adjust the image features of the initial characters, such as hairstyle, body shape, skin color, and facial features such as eyes, eyebrows, and mouth to achieve the desired appearance. Usually, the game provides several preset facial features, body shape, skin color, and clothing for players to choose from, but the choice is limited, and such minor adjustments based on the initial model are generally difficult to obtain high-difference changes. However, as the degree of freedom of the face-pinching system becomes higher and higher, the number of face-pinching parameters that players can adjust is increasing. Not only facial features, but also makeup such as eye shadow, blush, lip color, and pupil color can be adjusted in detail. In theory, the facial difference space that players can obtain by adjusting parameters is large, but for players, the complexity and difficulty of operation are also increasing exponentially. Therefore, it takes a lot of time for some players to pinch out a beautiful and distinctive appearance. Players need to perform more and more complicated manual operations, which makes the generation efficiency of game characters (virtual objects) low. Moreover, in the end, the image expected by the player may not be achieved, and it may even lead to a decline in the gaming experience. At the same time, the face pinched by the player cannot reflect the player's own image and personality traits.

[0034] Based on this, the embodiments of the present disclosure provide a method, device and electronic device for generating a virtual object, by which the technical problem of low efficiency in generating virtual objects can be alleviated.

[0035] The virtual object generation method in one embodiment of the present disclosure can be run on a local terminal device or a server. When the virtual object generation method is run on a server, the method can be implemented and executed based on a cloud interaction system, wherein the cloud interaction system includes a server and a client device.

[0036] In an optional implementation, various cloud applications can be run under the cloud interaction system, such as cloud games. Taking cloud games as an example, cloud games refer to a game mode based on cloud computing. In the operation mode of cloud games, the operating body of the game program and the main body of the game screen presentation are separated. The storage and operation of the virtual object generation method are completed on the cloud game server. The role of the client device is used for receiving and sending data and presenting the game screen. For example, the client device can be a display device with data transmission function close to the user side, such as a mobile terminal, a TV, a computer, a handheld computer, etc.; but the cloud game server in the cloud is used for information processing. When playing the game, the player operates the client device to send an operation instruction to the cloud game server. The cloud game server runs the game according to the operation instruction, encodes and compresses the game screen and other data, and returns it to the client device through the network. Finally, the client device decodes and outputs the game screen.

[0037] In an optional embodiment, taking a game as an example, a local terminal device stores a game program and is used to present a game screen. The local terminal device is used to interact with the player through a graphical user interface, that is, the game program is downloaded and installed by an electronic device and run conventionally. The local terminal device may provide the graphical user interface to the player in a variety of ways, for example, it may be rendered and displayed on a display screen of the terminal, or provided to the player through a holographic projection. For example, the local terminal device may include a display screen and a processor, the display screen is used to present a graphical user interface, the graphical user interface includes a game screen, and the processor is used to run the game, generate a graphical user interface, and control the display of the graphical user interface on the display screen.

[0038] In a possible implementation, an embodiment of the present disclosure provides a method for generating a virtual object, providing a graphical user interface through a terminal device, wherein the terminal device may be the local terminal device mentioned above, or may be a client device in the cloud interaction system mentioned above.

[0039] For example, Figure 1 As shown, Figure 1 Schematic diagram of an application scenario provided by an embodiment of the present disclosure. The application scenario may include an electronic terminal (e.g., a mobile phone 102) and a server 101, and the electronic terminal may communicate with the server 101 via a wired network or a wireless network. The electronic terminal is used to run a virtual desktop, and through the virtual desktop, it may interact with the server 101 to edit the content in the server 101.

[0040] The electronic terminal of this embodiment is described by taking a mobile phone 102 as an example. The mobile phone 102 includes components such as a radio frequency (RF) circuit 110, a memory 120, a touch screen 130, and a processor 140. Those skilled in the art can understand that Figure 2 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. Those skilled in the art will understand that the touch screen 130 belongs to the user interface (UI), and the mobile phone 102 may include more or fewer user interfaces than shown in the figure.

[0041] The RF circuit 110 can also communicate with the network and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobilecommunication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0042] The memory 120 can be used to store software programs and modules. The processor 140 executes various functional applications and data processing of the mobile phone 102 by running the software programs and modules stored in the memory 120. The memory 120 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone 102, etc. In addition, the memory 120 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0043] Processor 140 is the control center of mobile phone 102. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of mobile phone 102 and processes data by running or executing software programs and / or modules stored in memory 120 and calling data stored in memory 120, thereby monitoring the mobile phone as a whole.

[0044] The embodiments of the present disclosure are further described below in conjunction with the accompanying drawings.

[0045] Figure 3 A flowchart of a method for generating a virtual object provided by an embodiment of the present disclosure. The method can be applied to an electronic device, and the electronic device provides a pre-trained intelligent model execution method, and the audio samples of the training intelligent model correspond to the image labels of the virtual object. Figure 3 As shown, the method includes:

[0046] Step S310, receiving audio data corresponding to the virtual object generation task.

[0047] The virtual object generation task can be a task for any part of the virtual object, such as hairstyle, facial features, skin color, body shape, and clothing, or a task for the entire body of the virtual object including all the above parts. For example, the virtual object's avatar can be generated through the face pinching task, and the virtual object's entire body can be generated through the full body image shaping task, and so on.

[0048] In practical applications, audio data can be obtained in any way, making the sound content of audio data more flexible and comprehensive. As an example, audio data may include at least one of the following: user voice data, audio data recorded by the client, and audio data obtained from the network server. Figure 4 As shown, during the audio input process, users are supported to upload their own recorded audio data, and can also upload audio data such as dubbing and songs of film and television dramas.

[0049] As a possible implementation method, the samples input into the intelligent (AI) model in the embodiment of the present disclosure during training are audio samples of various different sounds, and the label corresponding to each audio sample is the image label of the virtual object. For example, the audio sample with a sweet voice corresponds to the image label of a loli with large and round eyes, and the audio sample with a rough voice corresponds to the image label of an uncle with a beard and rough skin. These audio samples with corresponding image labels are used for training to obtain an intelligent model that can generate virtual objects. The trained intelligent model inputs the audio data corresponding to the above-mentioned virtual object generation task during the formal application process, and the output of the intelligent model is the virtual object data corresponding to the virtual object generation task. The trained intelligent model can be used as a whole, and it can specifically implement multiple steps in the application process, such as receiving audio data, identifying and analyzing sound features, and generating virtual objects.

[0050] Step S320, identifying sound features of the audio data.

[0051] The sound features may include a variety of different types of features. Exemplarily, the sound features in the audio data are extracted and identified from a variety of aspects, so that the sound features may include pitch, timbre, tone, volume, voice print, voice characteristics (e.g., gentle, quirky, etc.), voice tone, and the semantics of the speech.

[0052] In an optional embodiment, if Figure 4 As shown, in the audio recognition process, the audio data is analyzed through the sound feature recognition model, and other irrelevant information carried in the audio data, such as background noise in the audio data, is discarded to extract and identify the sound-related features of the audio data.

[0053] As for the specific identification method of sound features, for example, AI sound feature recognition technology is used to analyze the player's sound waveform, spectrum and other data to extract key sound features, which include but are not limited to pitch, timbre, intonation, etc. The classification and identification of sounds in audio data can be achieved through a machine learning model that can learn and recognize these features.

[0054] Step S330: Generate virtual object data matching the sound features.

[0055] It should be noted that there are many ways to generate virtual object data. As an example, the intelligent model can automatically adjust facial features, makeup, etc. based on the unified virtual object initial template according to the sound characteristics, so as to obtain a virtual object that meets the sound characteristics more quickly and efficiently. Based on this, the process of generating virtual object data that matches the sound characteristics can specifically include the following steps: adjusting the image parameters of the preset initial virtual object according to the sound characteristics to obtain the target virtual object data that matches the sound characteristics.

[0056] As another example, the image library contains multiple existing images, and the intelligent model can select the target existing image with the most suitable sound characteristics from the image library, so as to generate a virtual object with sound audio characteristics more quickly and efficiently. Based on this, the intelligent model corresponds to multiple preset image data; the process of generating virtual object data matching the sound characteristics can specifically include the following steps: selecting target preset image data from multiple preset image data according to the sound characteristics; generating virtual object data matching the sound characteristics based on at least one target preset image data.

[0057] For example, a facial style library is established, which contains a combination of multiple makeup styles and facial features. By analyzing the voice features, the intelligent model can select the makeup style and facial features that best match the voice features from the facial style library, generate a personalized virtual object facial image, and achieve the difference effect of multiple different facial images.

[0058] As a possible implementation, the intelligent model can automatically infer the image features that match the voice features by analyzing the voice features based on the correlation between the voice features and the image features, such as the shape of facial features, skin color, makeup style, hairstyle, etc. For example, the eye shape of a sweet voice feature usually matches the image features of big and round eyes and a full face.

[0059] In an optional implementation, the analysis of the audio may include multiple analysis dimensions, for example, Figure 4 The various sound recognition dimensions shown include well-known sound recognition, lyrics text recognition, and timbre feature recognition.

[0060] As an example, the image features such as gender, personality, age, emotion, etc. are analyzed based on the sound features such as pitch, timbre, tone, volume, voiceprint, and sound characteristics in the audio data, so as to more accurately match the virtual object image of the style of loli, uncle, royal sister, etc. Based on this, the process of generating virtual object data matching the sound features can specifically include the following steps: determining the image features of the virtual object based on the sound features; wherein the sound features include at least one of the following: pitch, timbre, tone, volume, voiceprint, sound characteristics, and voice tone; the image features include at least one of the following: gender, personality, age, emotion, and dialect; generating virtual object data matching the sound features based on the image features.

[0061] Through artificial intelligence and voice recognition technology, the intelligent model can extract the human voice and use it as a reliable biometric feature to analyze Figure 4 The intelligent model can analyze whether the sound in the input audio data comes from a real person and the image characteristics of the person, such as the person's gender, age, personality, regional dialect, emotions, facial bone features, common voice lines, accents, etc. The information of the facial features is formed by the person's gender, age, personality, regional dialect, emotions, facial bone features, common voice lines, accents, etc. Exemplarily, these sound features such as the frequency range and resonance characteristics of the sound are related to gender and age. For example, the voice of a man usually has a lower fundamental frequency, while the fundamental frequency of the voice of a woman and a child is relatively high. The intelligent model can analyze whether the sound in the input audio data comes from a real person and the image characteristics of the person, such as the gender and age group. By establishing a reasonable correlation between the sound features and the image features, the facial features and makeup features with the sound features are recombined into a facial image, and the facial image can reflect the sound features. For example, Figure 4 As shown, the output face pinching result is a face generated based on the features.

[0062] Exemplarily, the intelligent model can analyze the regional attributes of the sound, such as the dialects of various regions. By constructing the corresponding machine learning model for the regional dialect, the regional dialect to which the sound belongs can be identified. The intelligent model can also identify the tone style and emotional tendency of the sound, such as gentle, cold, mocking and other tones. By analyzing the emotions and expression patterns of the sound using a machine learning model trained with sentiment analysis capabilities, the intelligent model can more accurately identify the emotional response information of the sound. For example, the process of determining the image characteristics of a virtual object based on sound features can specifically include the following steps: performing sentiment analysis on the sound features through a sentiment analysis model to obtain the emotional characteristics of the virtual object corresponding to the sound features.

[0063] In practical applications, the facial images of virtual objects automatically generated by intelligent models can include many types, such as Figure 5 The tsundere lady-style facial image shown, for example, Figure 6 The cool girl-style facial image shown in FIG. Figure 7 The domineering CEO-style facial image shown in the figure, for example, Figure 8 The facial image of the shota boy style shown, etc.

[0064] It should be noted that the virtual object data does not necessarily reflect the real voice characteristics of the user who makes the sound in the audio data, but may also include the voice characteristics corresponding to the image that the user expects and intends to have. For example, the user can use a voice changer, professional dubbing methods, etc. to imitate the voice corresponding to the desired image, which is not necessarily the voice of the customer's real image. For example, if a male user wants to generate a virtual object with a lolita image, the male user can deliberately use a sweet voice or sweet sentence content to imitate the sweet voice corresponding to the lolita image, rather than the voice of the male user's real image.

[0065] Based on this, the voice characteristics imitated by different customers who imitate the same type of image can also be different, because different people have different voice characteristics. For example, they all imitate the image of a royal sister, and the voices of the royal sister images imitated by different people also have their own voice characteristics, so the virtual object finally generated is based on the image that the customer wants to imitate (such as the image of a royal sister) to achieve different final images for different users. For example, the royal sister imitated by a person with a relatively sexy voice is a sexy royal sister image, the royal sister imitated by a person with a relatively cold voice is a cold royal sister image, and the royal sister imitated by a person with a relatively gentle voice is a gentle royal sister image, and so on.

[0066] As another example of the above-mentioned audio analysis dimension, the intelligent model can recognize the sound in the audio as the voice of a well-known person or the sound of a well-known animal, and the image of the generated virtual object can include the well-known image of the well-known person or the well-known animal, thereby making the image of the generated virtual object more flexible and interesting, and improving the user experience. Based on this, the process of generating virtual object data matching the sound feature can specifically include the following steps: judging whether the sound feature meets the preset feature; each preset feature corresponds to a preset well-known image; if the sound feature meets the target preset feature among multiple preset features, then determining the target preset well-known image corresponding to the target preset feature from the preset well-known images corresponding to the multiple preset features; generating virtual object data matching the sound feature according to the target preset well-known image.

[0067] For example, the intelligent model can use the pre-trained acoustic model to determine whether the input sound is a well-known iconic sound through deep learning and feature comparison of the sound, so as to ensure a high degree of accuracy and recognition rate. Figure 4 The voices of stars, celebrities, IP characters, game characters, anime characters, etc., or animal sounds such as cats and dogs, in the matching special voice library are shown. Based on this, the audio samples used to train the intelligent model can include the voices of stars, and the corresponding training labels are the real images of the stars. In this way, when the user models the voice of the star, or the voice is like the star, the image of the virtual object generated in the end can be related to the real image of the star in some places, for example, the generated facial image is related to the face of the star.

[0068] As another example of the above-mentioned audio analysis dimension, the intelligent model can analyze famous texts such as famous quotes, classic lyrics and lines based on the semantics of the voice in the audio, and the image characteristics of the source of the famous text, thereby making the image of the generated virtual object more flexible and interesting, and improving the user experience. Based on this, the sound characteristics include the semantics of the voice in the audio data; the process of generating virtual object data matching the sound characteristics can specifically include the following steps: judging whether the semantics of the voice in the audio data conforms to the preset semantics; each preset semantic corresponds to a preset source image; if the semantics of the voice in the audio data conforms to the target preset semantics among multiple preset semantics, then determining the target preset source image corresponding to the target preset semantics from the preset source images corresponding to the multiple preset semantics; generating virtual object data matching the sound characteristics according to the target preset source image.

[0069] For example, the intelligent model can determine whether the audio input to the intelligent model contains common idioms, classic lines, or lyrics based on voice recognition. If the user deliberately imitates the voice of the star or the content of the star's iconic sentences, such as the star's catchphrases, common idioms, classic lines, or lyrics, the image of the virtual object generated in the end may be related to the real image of the source of the text, for example, Figure 4 As shown, the output face pinching result is related to the celebrity face.

[0070] Step S340: Generate a virtual object corresponding to the virtual object generation task according to the virtual object data.

[0071] As an optional implementation, when officially pinching the face, the intelligent model can determine the facial image corresponding to the virtual object generation task based on the results of the sound feature judgment, and output the facial image of the virtual object.

[0072] The intelligent model generated by training with audio samples and corresponding virtual object image labels can recognize the sound characteristics of any audio and automatically generate virtual objects that match the sound characteristics, reducing the difficulty and complexity of manual operations for users to generate virtual objects through manual operations, thereby improving the efficiency of virtual object generation and user experience, and alleviating the technical problem of low virtual object generation efficiency.

[0073] In the disclosed embodiment, after extracting the sound features of the user input audio, a correlation is established between it and the image features, so as to automatically generate a virtual object matching the sound features. The image effect of the generated virtual object not only has sufficient differentiation, but also has a high correlation with the sound in the audio, so that each sound can correspond to a virtual object different from other images, thereby achieving the generation of more virtual objects with different image styles, allowing users to easily generate virtual objects matching the sound of the user input audio in the digital world, thereby improving the efficiency and personalization of virtual object creation, and increasing the connection between users and virtual objects, thereby enhancing the diversity, immersion and personalized experience of application scenarios such as games, virtual social platforms, and virtual reality, and improving user experience.

[0074] The above steps are described in detail below.

[0075] In some embodiments, the intelligent model can analyze and aesthetically evaluate the geometric structure of the virtual object's face, body shape, etc., and make adjustments based on this so that the final image of the virtual object meets certain aesthetic standards. As an example, step S330 may specifically include the following steps: generating intermediate virtual object data that matches the sound features; analyzing the appearance geometric structure of the intermediate virtual object data; evaluating the appearance geometric structure according to the preset geometric standard data to obtain the evaluation results; adjusting the intermediate virtual object data according to the evaluation results to obtain the final virtual object data that meets the preset geometric standard data.

[0076] Exemplarily, the intelligent model can perform a geometric analysis on the image features of the virtual object to obtain a geometric analysis result, and then perform an aesthetic evaluation on the geometric analysis result according to preset aesthetic standards (such as harmonious proportions of facial features, three-part and five-eye facial image standards, etc.), and then adjust the image of the virtual object based on the aesthetic evaluation results to obtain a final virtual object whose image meets the preset aesthetic standards.

[0077] By combining the evaluation and adjustment of geometric aesthetic standards, the final generated virtual object can not only reflect the sound characteristics of the audio provided by the user, but also meet certain aesthetic standards. For example, by combining aesthetic standards such as facial anatomy, facial proportions, face aesthetics and makeup art, it can ensure that the generated virtual object image not only meets aesthetic principles but also has personalized characteristics.

[0078] In some embodiments, the user can adjust the image of the virtual object generated by the intelligent model by himself, so that the image of the virtual object finally created is more flexible and more in line with the user's expectations. As an example, after step S340, the method may also include the following steps: in response to the adjustment operation of the virtual object corresponding to the virtual object generation task, the image of the virtual object corresponding to the virtual object generation task is adjusted according to the adjustment operation to obtain the adjusted virtual object. In the embodiment of the present disclosure, the user can further perform personalized fine-tuning on the image of the virtual object generated by the intelligent model, thereby obtaining a virtual object that better meets the user's ideal type.

[0079] Fig. 9 A schematic diagram of the structure of a virtual object generation device is provided. The device is provided with a pre-trained intelligent model by an electronic device to execute a corresponding method, and the audio samples of the training intelligent model correspond to the image labels of the virtual objects. Fig. 9 As shown, the virtual object generating device 900 includes:

[0080] Receiving module 901, used to receive audio data corresponding to the virtual object generation task;

[0081] An identification module 902, configured to identify the sound features of the audio data;

[0082] A first generating module 903, used to generate virtual object data matching the sound feature;

[0083] The second generating module 904 is configured to generate a virtual object corresponding to the virtual object generating task according to the virtual object data.

[0084] Through the above method, the virtual object generation model obtained by training with multiple audio samples and corresponding virtual object labels can recognize the sound characteristics of any audio and automatically generate virtual objects matching the sound characteristics, reducing the complexity of manual operation of virtual objects by users, thereby improving the generation efficiency of virtual objects and user experience, and alleviating the technical problem of low virtual object generation efficiency.

[0085] In a feasible implementation scheme, the first generation module is specifically used for:

[0086] Determining the image features of the virtual object according to the sound features; wherein the sound features include at least one of the following: pitch, timbre, tone, volume, voice print, voice quality, and voice tone; the image features include at least one of the following: gender, personality, age, emotion, and dialect;

[0087] Virtual object data matching the sound feature is generated based on the image feature.

[0088] In a feasible implementation scheme, the first generation module is specifically used for:

[0089] Determine whether the sound feature meets a preset feature; each of the preset features corresponds to a preset well-known image;

[0090] If the sound feature matches a target preset feature among the plurality of preset features, determining a target preset publicly known image corresponding to the target preset feature from preset publicly known images respectively corresponding to the plurality of preset features;

[0091] Virtual object data matching the sound features is generated according to the preset known image of the target.

[0092] In a feasible implementation manner, the sound feature includes the semantics of the speech in the audio data; the first generation module is specifically used for:

[0093] Determine whether the semantics of the speech in the audio data conforms to the preset semantics; each of the preset semantics corresponds to a preset source image;

[0094] If the semantics of the speech in the audio data conforms to a target preset semantics among the plurality of preset semantics, determining a target preset source image corresponding to the target preset semantics from preset source images corresponding to the plurality of preset semantics respectively;

[0095] Virtual object data matching the sound features is generated according to the target preset source image.

[0096] In a feasible implementation scheme, the first generation module is specifically used for:

[0097] generating intermediate virtual object data matching the sound features;

[0098] analyzing the appearance geometry of the intermediate virtual object data;

[0099] Evaluate the appearance geometric structure according to preset geometric standard data to obtain an evaluation result;

[0100] The intermediate virtual object data is adjusted according to the evaluation result to obtain final virtual object data that meets the preset geometric standard data.

[0101] In a feasible implementation scheme, the first generation module is specifically used for:

[0102] The image parameters of the preset initial virtual object are adjusted according to the sound characteristics to obtain target virtual object data matching the sound characteristics.

[0103] In a feasible implementation scheme, the intelligent model corresponds to a plurality of preset image data; the first generation module is specifically used for:

[0104] selecting target preset image data from the plurality of preset image data according to the sound feature;

[0105] Virtual object data matching the sound feature is generated based on at least one of the target preset image data.

[0106] In a feasible embodiment, the device further comprises:

[0107] The adjustment module is used to, after generating the virtual object corresponding to the virtual object generation task according to the virtual object data, respond to an adjustment operation on the virtual object corresponding to the virtual object generation task, adjust the image of the virtual object corresponding to the virtual object generation task according to the adjustment operation to obtain the adjusted virtual object.

[0108] In a feasible implementation scheme, the first generation module is further used for:

[0109] The sound feature is subjected to sentiment analysis through a sentiment analysis model to obtain sentiment features of a virtual object corresponding to the sound feature.

[0110] In a feasible implementation manner, the audio data includes at least one of the following:

[0111] The user's voice data, the audio data recorded by the client, and the audio data obtained from the network server.

[0112] The virtual object generation device provided in the embodiment of the present disclosure has the same technical features as the virtual object generation method provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0113] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure is shown, including: a memory 1001, a processor 1002 and a bus 1003, wherein the memory 1001 stores machine-readable instructions executable by the processor 1002, and when the electronic device runs a virtual object generation method as in the embodiment, the processor 1002 communicates with the memory 1001 through the bus 1003, and the processor 1002 executes the machine-readable instructions, and the processor 1002 executes the preamble of the method item to perform the following steps:

[0114] Receiving audio data corresponding to a virtual object generation task;

[0115] Identifying sound features of the audio data;

[0116] generating virtual object data matching the sound characteristics;

[0117] A virtual object corresponding to the virtual object generation task is generated according to the virtual object data.

[0118] In a feasible implementation manner, when the processor generates virtual object data matching the sound feature, it is specifically used to:

[0119] Determining the image features of the virtual object according to the sound features; wherein the sound features include at least one of the following: pitch, timbre, tone, volume, voice print, voice quality, and voice tone; the image features include at least one of the following: gender, personality, age, emotion, and dialect;

[0120] Virtual object data matching the sound feature is generated based on the image feature.

[0121] In a feasible implementation manner, when the processor generates virtual object data matching the sound feature, it is specifically used to:

[0122] Determine whether the sound feature meets a preset feature; each of the preset features corresponds to a preset well-known image;

[0123] If the sound feature matches a target preset feature among the plurality of preset features, determining a target preset publicly known image corresponding to the target preset feature from preset publicly known images respectively corresponding to the plurality of preset features;

[0124] Virtual object data matching the sound features is generated according to the preset known image of the target.

[0125] In a feasible implementation, the sound feature includes the semantics of the speech in the audio data; when the processor generates virtual object data matching the sound feature, it is specifically used to:

[0126] Determine whether the semantics of the speech in the audio data conforms to the preset semantics; each of the preset semantics corresponds to a preset source image;

[0127] If the semantics of the speech in the audio data conforms to a target preset semantics among the plurality of preset semantics, determining a target preset source image corresponding to the target preset semantics from preset source images corresponding to the plurality of preset semantics respectively;

[0128] Virtual object data matching the sound features is generated according to the target preset source image.

[0129] In a feasible implementation manner, when the processor generates virtual object data matching the sound feature, it is specifically used to:

[0130] generating intermediate virtual object data matching the sound features;

[0131] analyzing the appearance geometry of the intermediate virtual object data;

[0132] Evaluate the appearance geometric structure according to preset geometric standard data to obtain an evaluation result;

[0133] The intermediate virtual object data is adjusted according to the evaluation result to obtain final virtual object data that meets the preset geometric standard data.

[0134] In a feasible implementation manner, when the processor generates virtual object data matching the sound feature, it is specifically used to:

[0135] The image parameters of the preset initial virtual object are adjusted according to the sound characteristics to obtain target virtual object data matching the sound characteristics.

[0136] In a feasible implementation scheme, the intelligent model corresponds to a plurality of preset image data; when the processor generates virtual object data matching the sound feature, it is specifically used to:

[0137] selecting target preset image data from the plurality of preset image data according to the sound feature;

[0138] Virtual object data matching the sound feature is generated based on at least one of the target preset image data.

[0139] In a feasible implementation manner, after generating the virtual object corresponding to the virtual object generation task according to the virtual object data, the processor is further configured to:

[0140] In response to an adjustment operation on the virtual object corresponding to the virtual object generation task, the image of the virtual object corresponding to the virtual object generation task is adjusted according to the adjustment operation to obtain an adjusted virtual object.

[0141] In a feasible implementation manner, when the processor determines the image feature of the virtual object according to the sound feature, it is specifically used to:

[0142] The sound feature is subjected to sentiment analysis through a sentiment analysis model to obtain sentiment features of a virtual object corresponding to the sound feature.

[0143] In a feasible implementation manner, the audio data includes at least one of the following:

[0144] The user's voice data, the audio data recorded by the client, and the audio data obtained from the network server.

[0145] Through the above method, the virtual object generation model obtained by training with multiple audio samples and corresponding virtual object labels can recognize the sound characteristics of any audio and automatically generate virtual objects matching the sound characteristics, reducing the complexity of manual operation of virtual objects by users, thereby improving the generation efficiency of virtual objects and user experience, and alleviating the technical problem of low virtual object generation efficiency.

[0146] In practical applications, the memory 1001 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 1004 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0147] The bus 1003 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0148] Among them, the memory 1001 is used to store programs, and the processor 1002 executes the program after receiving the execution instruction. The method executed by the device defined by the process disclosed in any embodiment of the present disclosure can be applied to the processor 1002 or implemented by the processor 1002.

[0149] The processor 1002 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 1002. The above processor 1002 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present disclosure can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present disclosure can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1001, and the processor 1002 reads the information in the memory 1001 and completes the steps of the above method in combination with its hardware.

[0150] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored. The computer program is executed when a processor is running, and the processor performs the following steps:

[0151] Receiving audio data corresponding to a virtual object generation task;

[0152] Identifying sound features of the audio data;

[0153] generating virtual object data matching the sound characteristics;

[0154] A virtual object corresponding to the virtual object generation task is generated according to the virtual object data.

[0155] In a feasible implementation manner, when the processor generates virtual object data matching the sound feature, it is specifically used to:

[0156] Determining the image features of the virtual object according to the sound features; wherein the sound features include at least one of the following: pitch, timbre, tone, volume, voice print, voice quality, and voice tone; the image features include at least one of the following: gender, personality, age, emotion, and dialect;

[0157] Virtual object data matching the sound feature is generated based on the image feature.

[0158] In a feasible implementation manner, when the processor generates virtual object data matching the sound feature, it is specifically used to:

[0159] Determine whether the sound feature meets a preset feature; each of the preset features corresponds to a preset well-known image;

[0160] If the sound feature matches a target preset feature among the plurality of preset features, determining a target preset publicly known image corresponding to the target preset feature from preset publicly known images respectively corresponding to the plurality of preset features;

[0161] Virtual object data matching the sound features is generated according to the preset known image of the target.

[0162] In a feasible implementation, the sound feature includes the semantics of the speech in the audio data; when the processor generates virtual object data matching the sound feature, it is specifically used to:

[0163] Determine whether the semantics of the speech in the audio data conforms to the preset semantics; each of the preset semantics corresponds to a preset source image;

[0164] If the semantics of the speech in the audio data conforms to a target preset semantics among the plurality of preset semantics, determining a target preset source image corresponding to the target preset semantics from preset source images corresponding to the plurality of preset semantics respectively;

[0165] Virtual object data matching the sound features is generated according to the target preset source image.

[0166] In a feasible implementation manner, when the processor generates virtual object data matching the sound feature, it is specifically used to:

[0167] generating intermediate virtual object data matching the sound features;

[0168] analyzing the appearance geometry of the intermediate virtual object data;

[0169] Evaluate the appearance geometric structure according to preset geometric standard data to obtain an evaluation result;

[0170] The intermediate virtual object data is adjusted according to the evaluation result to obtain final virtual object data that meets the preset geometric standard data.

[0171] In a feasible implementation manner, when the processor generates virtual object data matching the sound feature, it is specifically used to:

[0172] The image parameters of the preset initial virtual object are adjusted according to the sound characteristics to obtain target virtual object data matching the sound characteristics.

[0173] In a feasible implementation scheme, the intelligent model corresponds to a plurality of preset image data; when the processor generates virtual object data matching the sound feature, it is specifically used to:

[0174] selecting target preset image data from the plurality of preset image data according to the sound feature;

[0175] Virtual object data matching the sound feature is generated based on at least one of the target preset image data.

[0176] In a feasible implementation manner, after generating the virtual object corresponding to the virtual object generation task according to the virtual object data, the processor is further configured to:

[0177] In response to an adjustment operation on the virtual object corresponding to the virtual object generation task, the image of the virtual object corresponding to the virtual object generation task is adjusted according to the adjustment operation to obtain an adjusted virtual object.

[0178] In a feasible implementation manner, when the processor determines the image feature of the virtual object according to the sound feature, it is specifically used to:

[0179] The sound feature is subjected to sentiment analysis through a sentiment analysis model to obtain sentiment features of a virtual object corresponding to the sound feature.

[0180] In a feasible implementation manner, the audio data includes at least one of the following:

[0181] The user's voice data, the audio data recorded by the client, and the audio data obtained from the network server.

[0182] Through the above method, the virtual object generation model obtained by training with multiple audio samples and corresponding virtual object labels can recognize the sound characteristics of any audio and automatically generate virtual objects matching the sound characteristics, reducing the complexity of manual operation of virtual objects by users, thereby improving the generation efficiency of virtual objects and user experience, and alleviating the technical problem of low virtual object generation efficiency.

[0183] In the embodiments of the present disclosure, the computer program can also execute other machine-readable instructions when run by the processor to execute other methods described in the embodiments. For the specific execution method steps and principles, please refer to the description of the embodiments, which will not be repeated here.

[0184] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0185] For another example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0186] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0187] In addition, each functional unit in the embodiments provided in the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0188] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the virtual object generation method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0189] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0190] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the aforementioned embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present disclosure. They should all be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A method for generating a virtual object, characterized in that: The method is performed by providing a pre-trained intelligent model through an electronic device, and the audio samples of the training intelligent model correspond to the image labels of the virtual objects. The method includes: Receiving audio data corresponding to a virtual object generation task; Identifying sound features of the audio data; generating virtual object data matching the sound characteristics; A virtual object corresponding to the virtual object generation task is generated according to the virtual object data.

2. The method according to claim 1, characterized in that The generating virtual object data matching the sound feature comprises: Determining the image features of the virtual object according to the sound features; wherein the sound features include at least one of the following: pitch, timbre, tone, volume, voice print, voice quality, and voice tone; the image features include at least one of the following: gender, personality, age, emotion, and dialect; Virtual object data matching the sound feature is generated based on the image feature.

3. The method according to claim 1, characterized in that The generating virtual object data matching the sound feature comprises: Determine whether the sound feature meets a preset feature; each of the preset features corresponds to a preset well-known image; If the sound feature matches a target preset feature among the plurality of preset features, determining a target preset publicly known image corresponding to the target preset feature from preset publicly known images respectively corresponding to the plurality of preset features; Virtual object data matching the sound features is generated according to the preset known image of the target.

4. The method according to claim 1, characterized in that: The sound features include the semantics of the speech in the audio data; The generating virtual object data matching the sound feature comprises: Determine whether the semantics of the speech in the audio data conforms to the preset semantics; each of the preset semantics corresponds to a preset source image; If the semantics of the speech in the audio data conforms to a target preset semantics among the plurality of preset semantics, determining a target preset source image corresponding to the target preset semantics from preset source images corresponding to the plurality of preset semantics respectively; Generate virtual object data matching the sound features according to the target preset source image.

5. The method according to claim 1, characterized in that: The generating virtual object data matching the sound feature comprises: generating intermediate virtual object data matching the sound characteristics; analyzing the appearance geometry of the intermediate virtual object data; Evaluate the appearance geometric structure according to preset geometric standard data to obtain an evaluation result; The intermediate virtual object data is adjusted according to the evaluation result to obtain final virtual object data that meets the preset geometric standard data.

6. The method according to claim 1, characterized in that The generating virtual object data matching the sound feature comprises: The image parameters of the preset initial virtual object are adjusted according to the sound characteristics to obtain target virtual object data matching the sound characteristics.

7. The method according to claim 1, characterized in that The intelligent model corresponds to a plurality of preset image data; The generating virtual object data matching the sound feature comprises: selecting target preset image data from the plurality of preset image data according to the sound feature; Virtual object data matching the sound feature is generated based on at least one of the target preset image data.

8. The method according to claim 1, characterized in that After generating the virtual object corresponding to the virtual object generation task according to the virtual object data, the method further includes: In response to an adjustment operation on the virtual object corresponding to the virtual object generation task, the image of the virtual object corresponding to the virtual object generation task is adjusted according to the adjustment operation to obtain an adjusted virtual object.

9. The method according to claim 2, characterized in that: The step of determining the image feature of the virtual object according to the sound feature comprises: The sound feature is subjected to sentiment analysis through a sentiment analysis model to obtain sentiment features of a virtual object corresponding to the sound feature.

10. The method according to claim 1, characterized in that The audio data includes at least one of the following: The user's voice data, the audio data recorded by the client, and the audio data obtained from the network server.

11. A virtual object generation device, characterized in that: The device is provided with a pre-trained intelligent model by an electronic device to execute the method, and the audio samples of the intelligent model are trained to correspond to the image labels of the virtual objects. The device includes: A receiving module, used for receiving audio data corresponding to a virtual object generation task; An identification module, used for identifying the sound features of the audio data; A first generating module, used to generate virtual object data matching the sound feature; The second generating module is used to generate a virtual object corresponding to the virtual object generating task according to the virtual object data.

12. An electronic terminal, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the method according to any one of claims 1 to 10.

Images