Animation rendering method and apparatus, and electronic device

Abstract

Disclosed in the present application are an animation rendering method and apparatus, and an electronic device. The method comprises: acquiring text information and animation data that are required for animation rendering, wherein the text information is used for describing the animation data; determining speech data and morpheme data that correspond to the text information; converting the morpheme data into weight coefficients of corresponding control points in the animation data, wherein the control points are used for controlling the shape of a model in the animation data; and on the basis of the weight coefficients and the speech data, rendering the animation data.

Description

Animation rendering method, device and electronic equipment

[0001] Related applications

[0002] The present application claims priority to the Chinese patent application No. 202410528324.6, filed on April 28, 2024, and entitled "Method, device and electronic equipment for animation rendering", the contents of which are hereby incorporated by reference in its entirety. TECHNICAL FIELD

[0003] The present application relates to the technical field of graphics rendering, and in particular to an animation rendering method, device and electronic equipment. BACKGROUND

[0004] In the field of game animation, etc., the common model phoneme rendering technology needs to be designed by an art designer in advance, for example, audio data of specified text is recorded, and phoneme animation effects are designed, and then processed during rendering. This method needs to design corresponding data files in advance, so the corresponding model only has limited text phoneme information, and cannot realize real-time diversity of dialogue effects, and is difficult to adapt to diverse environments.

[0005] At present, no effective solution has been proposed for the above problems. SUMMARY

[0006] The embodiments of the present application provide a method, device and electronic equipment for animation rendering, to at least solve the technical problem that the animation rendering model in the related art can only be rendered under limited text phonemes, and needs to obtain audio data in advance, causing the rendering requirements of different scenes to be unable to be met.

[0007] In a first aspect, the present application provides an animation rendering method, comprising: obtaining text information and animation data required for animation rendering, wherein the text information is used to describe the animation data; determining voice data and phoneme data corresponding to the text information; converting the phoneme data into a weight coefficient of a corresponding control point in the animation data, wherein the control point is used to control the shape of a model in the animation data; and rendering the animation data according to the weight coefficient and the voice data.

[0008] In some embodiments, after obtaining the text information and the animation data required for animation rendering, the method further comprises: obtaining a preset character set, wherein the preset character set includes characters that can be recognized by a computer; determining all text characters contained in the text information; determining whether each character in the text characters is contained in the preset character set to obtain a determination result corresponding to each character; retaining characters in the text information indicated by the determination result as being contained in the preset character set, and deleting characters in the text information indicated by the determination result as not being contained in the preset character set.

[0009] In some embodiments, before determining the phonetic data and morpheme data corresponding to the text information, the method further comprises: determining computer resources required for processing the text information, and obtaining locally remaining available computer resources; comparing the required computer resources and the available computer resources to obtain a comparison result; and determining a processing manner of the text information according to the comparison result.

[0010] In some embodiments, the processing manner of the text information according to the comparison result comprises: in a case where the comparison result indicates that the computer resources required for processing the text information are less than the available computer resources, processing the text information by a first manner, wherein the first manner is processing the text information locally; and in a case where the comparison result indicates that the computer resources required for processing the text information are greater than the available computer resources, processing the text information by a second manner, wherein the second manner is processing the text information in a server.

[0011] In some embodiments, the determining the phonetic data and morpheme data corresponding to the text information comprises: performing phonetic conversion on the text information by using a phonetic model to obtain the phonetic data corresponding to the text information, wherein the phonetic model is used to convert text into phonetic; performing word segmentation processing on the text information, and performing part-of-speech tagging on each segmented word obtained; determining keywords from the segmented word results according to the tagged part-of-speech, and determining the keywords as the morpheme data corresponding to the text information.

[0012] In some embodiments, the converting the morpheme data into the weight coefficients of the corresponding control points in the animation data comprises: determining a mapping relationship between the morpheme data and the control points; determining morpheme feature vectors according to the morpheme data, wherein the morpheme feature vectors are used to represent the association relationship between the morpheme data; and mapping the morpheme feature vectors to the weight coefficients of the control points by using a mapping model according to the mapping relationship, wherein the mapping model is used to convert the feature vectors into corresponding weight coefficients.

[0013] In some embodiments, the rendering the animation data according to the weight coefficients and the phonetic data comprises: placing the phonetic data in a first thread, and placing the weight coefficients in a second thread; obtaining a playing time corresponding to the phonetic data played by the first thread in a process of updating the weight coefficients in the second thread; determining a corresponding keyword in the morpheme data according to the playing time; determining a target weight coefficient corresponding to the keyword, and adjusting the animation data according to the target weight coefficient; and rendering the adjusted animation data at the playing time.

[0014] In some embodiments, the method further comprises: determining an original format of the voice data and a target format required when the voice data is played by the playing device; comparing the original format and the target format to obtain a comparison result; obtaining an original number of channels, an original sampling rate in the original format, and obtaining a target number of channels and a target sampling rate in the target format, in a case where the comparison result indicates that the original format is inconsistent with the target format; changing the original number of channels to the target number of channels, and changing the original sampling rate to the target sampling rate.

[0015] The application further provides an animation rendering device in a second aspect, comprising: an obtaining module configured to obtain text information and animation data required for animation rendering, wherein the text information is used to describe the animation data; a determining module configured to determine voice data and morpheme data corresponding to the text information; a converting module configured to convert the morpheme data into a weight coefficient of a corresponding control point in the animation data, wherein the control point is used to control a shape of a model in the animation data; and a rendering module configured to render the animation data according to the weight coefficient and the voice data.

[0016] In some embodiments, the obtaining module is further configured to: obtain a preset character set, wherein the preset character set comprises characters recognizable by a computer; determine all text characters contained in the text information; determine whether each character in the text characters is contained in the preset character set to obtain a determination result corresponding to each character; retain characters in the text information whose determination results indicate that the characters are contained in the preset character set, and delete characters in the text information whose determination results indicate that the characters are not contained in the preset character set.

[0017] In some embodiments, the determining module is further configured to: determine computer resources required for processing the text information, and obtain locally remaining available computer resources; compare the required computer resources and the available computer resources to obtain a comparison result; and determine a processing manner of the text information according to the comparison result.

[0018] In some embodiments, the determining module is further configured to: process the text information in a first manner in a case where the comparison result indicates that the computer resources required for processing the text information are less than the available computer resources, wherein the first manner is to process the text information locally; and process the text information in a second manner in a case where the comparison result indicates that the computer resources required for processing the text information are greater than the available computer resources, wherein the second manner is to process the text information in a server.

[0019] In some embodiments, the determining module is further configured to: convert the text information into speech data corresponding to the text information by using a speech model, wherein the speech model is used to convert text into speech; perform word segmentation on the text information, and perform part-of-speech tagging on each segmented word obtained; determine a keyword from the segmented word results according to the tagged part-of-speech, and determine the keyword as the morpheme data corresponding to the text information.

[0020] In some embodiments, the converting module is further configured to: determine a mapping relationship between the morpheme data and the control points; determine a morpheme feature vector according to the morpheme data, wherein the morpheme feature vector is used to represent the association relationship between the morpheme data; and map the morpheme feature vector to the weight coefficients of the control points by using a mapping model according to the mapping relationship, wherein the mapping model is used to convert the feature vector into the corresponding weight coefficients.

[0021] In some embodiments, the converting module is further configured to: determine an original format of the speech data and a target format required when the speech data is played by a playback device; compare the original format and the target format to obtain a comparison result; in a case where the comparison result indicates that the original format is inconsistent with the target format, obtain an original number of channels, an original sampling rate in the original format, and obtain a target number of channels and a target sampling rate in the target format; change the original number of channels to the target number of channels, and change the original sampling rate to the target sampling rate.

[0022] In some embodiments, the rendering module is further configured to: place the speech data in a first thread, and place the weight coefficients in a second thread; in the process of updating the weight coefficients in the second thread, obtain a playback time corresponding to the playback of the speech data in the first thread; determine a corresponding keyword in the morpheme data according to the playback time; determine a target weight coefficient corresponding to the keyword, and adjust the animation data according to the target weight coefficient; and render the adjusted animation data at the playback time.

[0023] In a third aspect, the present application further provides an electronic device, comprising: a memory configured to store computer program instructions; and a processor connected to the memory, configured to execute the following steps when running the computer program instructions: obtaining text information and animation data required for animation rendering, wherein the text information is used to describe the animation data; determining speech data and morpheme data corresponding to the text information; converting the morpheme data into weight coefficients of corresponding control points in the animation data, wherein the control points are used to control the shape of a model in the animation data; and rendering the animation data according to the weight coefficients and the speech data.

[0024] The application also provides, in a fourth aspect, a nonvolatile computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, performs the animation rendering method in the first aspect.

[0025] The application also provides, in a fifth aspect, a computer program product comprising computer instructions, which, when executed by a processor, implement the animation rendering method in the first aspect.

[0026] According to the embodiments of the application, the text information and the animation data required for animation rendering are acquired, wherein the text information is used to describe the animation data; the phonetic data and the morpheme data corresponding to the text information are determined; the morpheme data is converted into the weight coefficient of the corresponding control point in the animation data, wherein the control point is used to control the shape of the model in the animation data; and the animation data is rendered according to the weight coefficient and the phonetic data. Thus, the purpose of rendering different scene animations based on the phonetic morpheme data corresponding to different text information is achieved, thereby realizing text-based animation production and flexible control of animation shape, improving the automation and rapidity of animation rendering, and further solving the technical problem that the animation rendering model in the related art can only be rendered under limited text phonetic morpheme and needs to acquire audio data in advance, thereby failing to meet the rendering requirements of different scenes. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the application or the related art, the following will briefly introduce the drawings needed to be used in the embodiments or the related art description. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0028] FIG. 1 is a hardware structure diagram of a computer device for implementing an animation rendering method according to an embodiment of the application.

[0029] FIG. 2 is a flowchart of an animation rendering method according to an embodiment of the application.

[0030] FIG. 3 is a schematic diagram of the overall flow of an animation rendering process according to an embodiment of the application.

[0031] FIG. 4 is a structural diagram of an animation rendering device according to an embodiment of the application. DETAILED DESCRIPTION

[0032] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application, so that those skilled in the art can better understand the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present application.

[0033] It should be noted that the terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in other than the order illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0034] The animation rendering method provided by the embodiments of the present application can be executed in a mobile terminal, a computer device or a similar computing device. FIG. 1 shows a hardware structure block diagram of a computer device for implementing an animation rendering method. As shown in FIG. 1, the computer device 10 can include one or more processors (the processor can include but is not limited to a microprocessor (MCU) or a programmable logic device (FPGA) and the like processing device), a memory 104 for storing data, and a transmission module 106 for communication function connected through wired and / or wireless network. In addition, it can also include a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be a port of the I / O interface), a network interface, a bus (BUS). Those skilled in the art can understand that the structure shown in FIG. 1 is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the computer device 10 can include more or less components than those shown in FIG. 1, or have a different configuration from that shown in FIG. 1.

[0035] It should be noted that the one or more processors and / or other data processing circuitry described above can be referred to herein generically as "data processing circuitry". The data processing circuitry can be embodied as software, hardware, firmware, or any combination thereof, in whole or in part. Moreover, the data processing circuitry can be a single standalone processing module, or incorporated in whole or in part within any one of the other elements of the computer device 10. As referred to in the embodiments of the present application, the data processing circuitry serves as a processor to control, for example, the selection of the variable resistance terminal path connected to the interface.

[0036] The memory 104 can be used to store software programs of application software and modules, such as program instructions / data storage means corresponding to the animation rendering method in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, i.e. implements the animation rendering method described above. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory disposed remotely with respect to the processor, which can be connected to the computer device 10 through a network. The network in the above embodiments includes but is not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0037] The transmission module 106 is used to receive or send data via a network. The network in the above embodiments can specifically include a wireless network provided by a communication provider of the computer device 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet in a wireless manner.

[0038] The display can be, for example, a touch screen type liquid crystal display (LCD), which can enable a user to interact with the user interface of the computer device 10.

[0039] It should be noted that in some optional embodiments, the computer terminal shown in FIG. 1 can include hardware elements (including circuitry), software elements (including computer code stored on a computer readable medium), or combinations of both hardware and software elements. It should be noted that FIG. 1 is merely one example and is intended to illustrate the types of components that can be present in the computer device described above.

[0040] In the above operating environment, the embodiment of the present application provides an animation rendering method. It should be noted that the steps shown in the flowchart can be executed by a group of computer executable instructions in a computer system. In addition, although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0041] FIG. 2 is a flowchart of an animation rendering method according to an embodiment of the present application. As shown in FIG. 2, the method comprises the following steps S202 to S208.

[0042] Step S202: obtaining text information and animation data required for animation rendering, wherein the text information is used to describe the animation data.

[0043] In the above step S202, the text information is the text information required for animation rendering in any scene, which can include text information output by a large model and text information manually input by a person. It should be noted that the text information can be real-time changing, rather than a pre-designed specified animation content. In some embodiments, the animation rendering can be rendering of facial and lip shape animation of a three-dimensional character model, or rendering of other parts of the three-dimensional character model, which is not limited here.

[0044] Step S204: determining phonetic data and morpheme data corresponding to the text information.

[0045] In the above step S204, the system determines phonetic data and morpheme data corresponding to the text information according to the text information, wherein the phonetic data is used to represent the audio features of the text information, and the morpheme data is used to represent the semantic features of the text information.

[0046] Step S206: converting the morpheme data into weight coefficients of corresponding control points in the animation data, wherein the control points are used to control the shape of the model in the animation data.

[0047] In the above step S206, the morpheme data can be converted into weight coefficients of corresponding control points in the animation data through a mapping relationship between the morpheme data and the control points, wherein the control points are used to control the shape of the model in the animation data, and the weight coefficients determine the influence degree of each control point on the animation data.

[0048] Step S208: rendering the animation data according to the weight coefficients and the phonetic data.

[0049] In the above step S208, the output of the animation audio is determined by the phonetic data, and the shape of the animation model is determined by the weight coefficients of the control points corresponding to the morpheme data, and the synchronous rendering of the animation data can be completed by combining the playing time of the animation audio.

[0050] Through steps S202 to S208, the purpose of rendering different scene animations based on phoneme data corresponding to different text information is achieved, thereby realizing text-based animation production, flexible control of animation shape, and improving the technical effects of animation rendering automation and rapidity, thereby solving the technical problem that the animation rendering model can only be rendered under limited text phonemes, and audio data needs to be obtained first, causing the rendering requirement of different scenes to be unable to be met. Details are described below.

[0051] In some embodiments, after obtaining the text information and animation data required for animation rendering, the above method can further include: obtaining a preset character set, wherein the preset character set includes characters that can be recognized by a computer; determining all text characters contained in the text information; determining whether each character in the text characters is contained in the preset character set to obtain a determination result corresponding to each character; retaining characters in the text information indicated by the determination result to be contained in the preset character set, and deleting characters in the text information indicated by the determination result to be not contained in the preset character set.

[0052] In the embodiments of the present application, TextInput class can be used to process text information, and the irregular parts (including special characters, line breaks, and meaningless characters such as carriage returns) in the text information can be removed. These irregular characters will not appear in the preset character set to ensure that the final used text is regular and does not contain meaningless characters. Specifically, first, a preset character set containing characters that can be recognized by a computer is defined, which can usually include Chinese, English, numbers, and various language types; second, TextInput class is used to process the text information to obtain all text characters contained therein; third, for each character in the text information, it is determined whether it is contained in the preset character set, and a determination result corresponding to each character is obtained; finally, according to the determination result of each character, characters in the text information indicated by the determination result to be contained in the preset character set are retained, and characters in the text information indicated by the determination result to be not contained in the preset character set are deleted.

[0053] In some embodiments, before determining the phoneme data and phoneme data corresponding to the text information, the above method can further include: determining computer resources required for processing the text information, and obtaining locally remaining available computer resources; comparing the required computer resources and the available computer resources to obtain a comparison result; and determining a processing mode of the text information according to the comparison result.

[0054] In some embodiments, the step of determining the manner of processing the text information according to the comparison result can include: in a case where the comparison result indicates that the computer resources required for processing the text information are less than the available computer resources, processing the text information by a first manner, wherein the first manner is processing the text information locally; in a case where the comparison result indicates that the computer resources required for processing the text information are greater than the available computer resources, processing the text information by a second manner, wherein the second manner is processing the text information in a server.

[0055] In the embodiments of the present application, the ProcessText class can be used to generate the phonetic data and morpheme data corresponding to the text information, and the generation process can be performed in real time, locally or on a server. The ProcessText class is also responsible for processing the text information. First, the computer resources required for processing the text information need to be determined, and the available computer resources remaining locally need to be obtained. By comparing the computer resources required for processing the text information with the available computer resources remaining locally, the manner of processing the text information is determined. Specifically:

[0056] In a case where the computer resources required for processing the text information are less than the available computer resources remaining locally, it indicates that the computer resources locally are sufficient, and the text information can be processed locally, i.e., the first manner described above. At this time, the ProcessText class will perform local interaction, using computer resources (such as CPU, memory, etc.) to process the text information and generate corresponding phonetic data and morpheme data.

[0057] In a case where the computer resources required for processing the text information are greater than the available computer resources remaining locally, it indicates that the computer resources locally are insufficient to process the text information, and the text information can be processed in a server, i.e., the second manner described above. At this time, the ProcessText class will be responsible for communicating with the server, obtaining the phonetic data and morpheme data generated by the server through interaction with the server, and returning the phonetic data and morpheme data.

[0058] In some embodiments, the step of determining the phonetic data and morpheme data corresponding to the text information can include: performing voice conversion on the text information using a voice model to obtain the phonetic data corresponding to the text information, wherein the voice model is used to convert text into voice; performing word segmentation processing on the text information, and performing part-of-speech tagging on each word segment obtained; determining a keyword from the word segmentation result according to the tagged part-of-speech, and determining the keyword as the morpheme data corresponding to the text information.

[0059] In this embodiment, the text information needs to be processed and converted, such as converting the text information into voice data using a voice model, and processing the text information using word segmentation and part-of-speech tagging to generate morpheme data, thereby generating voice data and morpheme data required for animation rendering. Specifically,

[0060] 1. Convert the input text information into corresponding voice data using a voice model. For example: using a TTS model to input text information, through natural language processing and speech synthesis technology, to generate realistic voice output, which can include phoneme division, tone adjustment, and speech rate adjustment of the text.

[0061] 2. Perform word segmentation on the input text information and perform part-of-speech tagging on each word. Word segmentation is the process of dividing continuous text sequences into meaningful words or phrases, while part-of-speech tagging determines the grammatical role each word plays in a sentence, such as nouns, verbs, adjectives, etc.

[0062] 3. According to the results of part-of-speech tagging, key words can be determined from the word segmentation results. Among them, key words are considered to be morpheme data corresponding to text information, which can be used for further processing or marking to extract important information or features of the text, for example, in animation rendering, these key words may represent specific characters, scenes or actions.

[0063] In some embodiments, the step of converting the morpheme data into the weight coefficients of the corresponding control points in the animation data can include: determining the mapping relationship between the morpheme data and the control points; determining a morpheme feature vector according to the morpheme data, wherein the morpheme feature vector is used to represent the association relationship between the morpheme data; and mapping the morpheme feature vector to the weight coefficients of the control points using a mapping model according to the mapping relationship, wherein the mapping model is used to convert the feature vector into corresponding weight coefficients.

[0064] In the embodiments of the present application, the SpeechProcess class can be used to convert the different initial and final data of the morphemes into the corresponding control point coefficients during voice conversion, while considering the smooth transition of the mouth shape, and the converted coefficients are smoothed to ensure the continuity of the action. The specific steps are as follows:

[0065] 1. Determine the mapping relationship between the morpheme data and the control points: first, the mapping relationship between the morpheme data and the control points needs to be determined, which can be modeled and determined according to professional knowledge, phonetics rules or training data, according to different initial and final pronunciations, to determine the corresponding mouth shape and oral shape.

[0066] 2. Determine the morpheme feature vector: according to the morpheme data and its corresponding mouth shape and oral shape, determine the morpheme feature vector, wherein the morpheme feature vector is used to represent the association between the morpheme data, and can include information such as mouth opening degree, tongue position, lip shape, etc.

[0067] 3. Map the morpheme feature vector to the weight coefficient of the control point using the mapping model: use the mapping model to map the morpheme feature vector to the weight coefficient of the control point, to realize the conversion from the morpheme data to the control point coefficient, wherein the mapping model can be a statistical learning-based model, such as a neural network or other machine learning algorithm.

[0068] 4. Smooth the converted coefficients: since the mouth shape is smooth during speaking, the converted coefficients need to be smoothed to ensure the continuity of the motion, and signal processing techniques or filters can be used to achieve smoothing, such as using a low-pass filter to remove high-frequency noise or rapidly changing parts.

[0069] Optionally, the step of rendering the animation data according to the weight coefficient and the voice data can include: putting the voice data into a first thread and the weight coefficient into a second thread; in the process of updating the weight coefficient in the second thread, obtaining the playing time corresponding to the voice data played by the first thread; determining the corresponding keyword in the morpheme data according to the playing time; determining the target weight coefficient corresponding to the keyword, and adjusting the animation data according to the target weight coefficient; and rendering the adjusted animation data at the playing time.

[0070] In the embodiments of the present application, by using the SyncRender class to control the processing and rendering thread of the audio, the synchronization between the voice and the animation during the playing process can be ensured, which can include:

[0071] (1) Audio processing thread: that is, the first thread described above, in this process, the SyncRender class is responsible for passing the voice data to the audio playing module in a separate audio processing thread to ensure the continuity of the audio, and continuously provides audio data to the audio playing module.

[0072] (2) Rendering thread: that is, the second thread described above, in this process, the interpolation and rendering of the control points are processed by updating the weight coefficient. In order to synchronize with the voice, the rendering thread obtains the current audio playing time, and determines the keyframe data (i.e. the keyword described above) to be used according to the current audio playing time. At the same time, smoothing and interpolation are performed to ensure the continuity of the animation.

[0073] For smoothing processing, the weight coefficients can be processed using a smoothing algorithm to reduce abrupt changes and discontinuities. For example, a simple linear smoothing algorithm can be used to smooth the changes in weight coefficients by interpolating between adjacent frames to ensure smoother animation transitions.

[0074] For interpolation processing, when selecting keyframe data, an interpolation method can be used to obtain more accurate weight coefficients. By interpolating between known keyframes, weight coefficients for intermediate time points can be obtained to avoid abrupt changes in weight coefficients and make the animation transition more natural.

[0075] (3) Adjusting animation data and rendering: based on the selected keyframe data and target weight coefficients, the rendering thread adjusts the animation data and renders at the playback time to achieve synchronized rendering of animation and voice.

[0076] In some embodiments, the above method can further include: determining an original format of the voice data and a target format required by a playback device when playing the voice data; comparing the original format and the target format to obtain a comparison result; in a case where the comparison result indicates that the original format is inconsistent with the target format, obtaining an original number of channels and an original sampling rate in the original format, and obtaining a target number of channels and a target sampling rate in the target format; changing the original number of channels to the target number of channels, and changing the original sampling rate to the target sampling rate.

[0077] In the embodiments of the present application, the AudioProcess class can be used to process the original voice data into the format required for playback to ensure that the audio file can be played well on different playback devices. At the same time, the following aspects need to be considered in the process of AudioProcess:

[0078] 1. Determine the original format and the target format: first, the original format of the voice data needs to be determined, such as the number of channels, the sampling rate, the integer / floating point type, etc. information, and the target format required by the playback device when playing the voice data also needs to be determined, including single channel / dual channel, sampling rate, integer / floating point type, etc. information.

[0079] 2. Compare the original format and the target format: compare the original format and the target format to obtain a comparison result. If the original format and the target format are consistent, no processing is required; if they are inconsistent, the next step of processing is required.

[0080] 3. Obtain the original number of channels and the original sampling rate: in a case where the comparison result indicates that the original format is inconsistent with the target format, the original number of channels and the original sampling rate in the original format need to be obtained as parameters for subsequent processing. At the same time, the target number of channels and the target sampling rate in the target format also need to be obtained as parameters for subsequent processing.

[0081] 4. Adjust the number of channels and the sampling rate: according to the obtained parameter information, change the original number of channels to the target number of channels, and change the original sampling rate to the target sampling rate, so that the audio file has better compatibility and playability, thereby ensuring that users can have a unified audio experience on different playback devices.

[0082] Fig. 3 is a schematic diagram of the overall process of an animation rendering process according to an embodiment of the present application. As shown in Fig. 3, first, text information is input, and the text information is processed using a TextInput class to remove incorrect content in the text information. Second, the text information is processed using a ProcessText class to determine a processing method for the text information, which can include local processing or server processing, to generate phonetic data and morpheme data corresponding to the text information. Third, a SpeechProcess class is used to convert different phonetic data into corresponding weight coefficients of control points in a speech conversion process, i.e., the SpeechProcess class processes the morpheme data, and an AudioProcess class is used to process original speech data into a format required for playback. Finally, a SyncRender class is used to control the processing of audio and the updating of a rendering thread to ensure that speech and animation remain synchronized during playback, i.e., the SyncRender class uses morpheme data and speech data according to rendering time.

[0083] In the above process, by converting text information into speech data and morpheme data in real time and applying the data to a model for animation rendering, corresponding speech content can be generated at the same time as the user inputs the text information, achieving a more timely and dynamic interactive experience. At the same time, since the text information does not need to be pre-designed, and the model only needs to increase the standard of key point weight coefficients, the animation rendering is suitable for a variety of different models of speech and morpheme, and has strong flexibility and versatility, thereby improving the automation and speed of animation rendering.

[0084] According to an embodiment of the present application, an animation rendering device is provided. It should be noted that the animation rendering device of the present application can be used to execute the above animation rendering method. The animation rendering device according to an embodiment of the present application is described below.

[0085] Fig. 4 is a structural diagram of an animation rendering device according to an embodiment of the present application. As shown in Fig. 4, the device includes an acquisition module 40, a determination module 42, a conversion module 44, and a rendering module 46.

[0086] The acquisition module 40 is configured to acquire text information and animation data required for animation rendering, wherein the text information is used to describe the animation data.

[0087] The determining module 42 is configured to determine phonetic data and morpheme data corresponding to the text information.

[0088] The converting module 44 is configured to convert the morpheme data into weight coefficients of corresponding control points in the animation data, where the control points are used to control the shape of the model in the animation data.

[0089] The rendering module 46 is configured to render the animation data according to the weight coefficients and the phonetic data.

[0090] The obtaining module 40, the determining module 42, the converting module 44 and the rendering module 46 in the animation rendering device achieve the purpose of rendering different scene animations based on phonetic morpheme data corresponding to different text information, thereby realizing text-based animation production and flexible control of animation shape, improving the automation and rapidity of animation rendering, and solving the technical problem that the animation rendering model can only be rendered under limited text phonetic morphemes and needs to obtain audio data first, which causes the rendering requirement of different scenes to be unable to be met.

[0091] In the animation rendering device provided in the embodiments of the present application, the obtaining module is further configured to obtain a preset character set, where the preset character set includes characters recognizable by a computer; determine all text characters contained in the text information; determine whether each character in the text characters is contained in the preset character set to obtain a determination result corresponding to each character; retain characters in the text information indicated by the determination result to be contained in the preset character set and delete characters in the text information indicated by the determination result to be not contained in the preset character set.

[0092] In the animation rendering device provided in the embodiments of the present application, the determining module is further configured to determine computer resources required for processing the text information and obtain available computer resources remaining in the local; compare the required computer resources and the available computer resources to obtain a comparison result; and determine a processing manner of the text information according to the comparison result.

[0093] In the animation rendering device provided in the embodiments of the present application, the determining module is further configured to process the text information in a first manner in a case where the comparison result indicates that the computer resources required for processing the text information are less than the available computer resources, where the first manner is to process the text information in the local; and process the text information in a second manner in a case where the comparison result indicates that the computer resources required for processing the text information are greater than the available computer resources, where the second manner is to process the text information in a server.

[0094] In the animation rendering device provided in the embodiments of the present application, the determining module is further configured to convert the text information into speech data corresponding to the text information by using a speech model, wherein the speech model is used to convert text into speech; perform word segmentation on the text information, and perform part-of-speech tagging on each word segment obtained; determine a keyword from the word segmentation result according to the tagged part-of-speech, and determine the keyword as morpheme data corresponding to the text information.

[0095] In the animation rendering device provided in the embodiments of the present application, the converting module is further configured to determine a mapping relationship between the morpheme data and the control points; determine morpheme feature vectors according to the morpheme data, wherein the morpheme feature vectors are used to represent the association relationship between the morpheme data; and map the morpheme feature vectors to the weight coefficients of the control points by using a mapping model according to the mapping relationship, wherein the mapping model is used to convert the feature vectors into corresponding weight coefficients.

[0096] In the animation rendering device provided in the embodiments of the present application, the converting module is further configured to determine an original format of the speech data and a target format required when the speech data is played by a playback device; compare the original format and the target format to obtain a comparison result; in a case where the comparison result indicates that the original format is inconsistent with the target format, obtain an original channel number and an original sampling rate in the original format, and obtain a target channel number and a target sampling rate in the target format; change the original channel number to the target channel number, and change the original sampling rate to the target sampling rate.

[0097] In the animation rendering device provided in the embodiments of the present application, the rendering module is further configured to put the speech data into a first thread, and put the weight coefficients into a second thread; in the process of updating the weight coefficients in the second thread, obtain a playback time corresponding to the playback of the speech data in the first thread; determine a corresponding keyword in the morpheme data according to the playback time; determine a target weight coefficient corresponding to the keyword, and adjust the animation data according to the target weight coefficient; and render the adjusted animation data at the playback time.

[0098] The embodiments of the present application further provide an electronic device, including a memory configured to store computer program instructions, and a processor connected with the memory, wherein the processor is configured to execute the following steps when running the stored computer program instructions: obtaining text information and animation data required for animation rendering, wherein the text information is used to describe the animation data; determining speech data and morpheme data corresponding to the text information; converting the morpheme data into weight coefficients of corresponding control points in the animation data, wherein the control points are used to control the shape of a model in the animation data; and rendering the animation data according to the weight coefficients and the speech data.

[0099] It should be noted that the electronic device is used to execute the animation rendering method shown in FIG. 2, and therefore the related explanations in the animation rendering method are also applicable to the electronic device, which will not be described herein again.

[0100] The embodiment of the present application further provides a nonvolatile computer readable storage medium, which stores a computer program, and the computer program is executable to execute the animation rendering method when the computer program is run by a processor.

[0101] It should be noted that the nonvolatile computer readable storage medium is used to execute the animation rendering method shown in FIG. 2, and therefore the related explanations in the animation rendering method are also applicable to the nonvolatile computer readable storage medium, which will not be described herein again.

[0102] The embodiment of the present application further provides a computer program product, which comprises a nonvolatile computer readable storage medium, and the nonvolatile computer readable storage medium stores a computer program, and the computer program is executable to execute the steps of the animation rendering method in the embodiments of the present application when the computer program is run by a processor.

[0103] The embodiment of the present application further provides a computer program product, which comprises computer instructions, and the computer instructions are executable to execute the steps of the animation rendering method in the embodiments of the present application when the computer instructions are run by a processor.

[0104] The embodiment of the present application further provides a computer program, and the computer program is executable to execute the steps of the animation rendering method in the embodiments of the present application when the computer program is run by a processor.

[0105] The serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0106] In the above embodiments of the present application, the descriptions of the embodiments are each focused on, and the parts not described in detail in an embodiment can be referred to the related descriptions of other embodiments.

[0107] In the several embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. Of course, the unit embodiment described above is only schematic. For example, the division of the units can be a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, units or modules, and can be electrical or other forms.

[0108] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0109] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0110] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical scheme of the present application essentially or the part that contributes to the prior art or the whole or part of the technical scheme can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0111] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0112] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0113] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for animation rendering, comprising: obtaining text information and animation data required for animation rendering, wherein the text information is used to describe the animation data; determining phonetic data and morpheme data corresponding to the text information; converting the morpheme data into weight coefficients of corresponding control points in the animation data, wherein the control points are used to control shapes of a model in the animation data; rendering the animation data according to the weight coefficients and the phonetic data. 2.The method of claim 1, after obtaining the text information and the animation data required for animation rendering, the method further comprises: obtaining a preset character set, wherein the preset character set comprises characters recognizable by a computer; determining all text characters contained in the text information; judging whether each character in the text characters is contained in the preset character set to obtain a judgment result corresponding to each character; retaining characters in the text information indicated by the judgment result as being contained in the preset character set, and deleting characters in the text information indicated by the judgment result as not being contained in the preset character set. 3.The method of claim 1, before determining the phonetic data and the morpheme data corresponding to the text information, the method further comprises: determining computer resources required for processing the text information, and obtaining locally remaining available computer resources; comparing the required computer resources and the available computer resources to obtain a comparison result; and determining a processing manner of the text information according to the comparison result. 4.The method of claim 3, wherein determining the processing manner of the text information according to the comparison result comprises: in a case where the comparison result indicates that the computer resources required for processing the text information are less than the available computer resources, processing the text information by a first manner, wherein the first manner is processing the text information locally; and in a case where the comparison result indicates that the computer resources required for processing the text information are greater than the available computer resources, processing the text information by a second manner, wherein the second manner is processing the text information in a server. 5.The method of claim 1, wherein determining the phonetic data and the morpheme data corresponding to the text information comprises: performing phonetic conversion on the text information by using a phonetic model to obtain phonetic data corresponding to the text information, wherein the phonetic model is used to convert text into phonetics; performing word segmentation processing on the text information, and performing part-of-speech tagging on each segmented word obtained; determining a keyword from the segmented word result according to the tagged part-of-speech, and determining the keyword as morpheme data corresponding to the text information. 6.The method of claim 1, wherein converting the morpheme data into weight coefficients of corresponding control points in the animation data comprises: determining a mapping relationship between the morpheme data and the control points. determine a morpheme feature vector according to the morpheme data, wherein the morpheme feature vector is used to represent a correlation between the morpheme data; map the morpheme feature vector to the weight coefficients of the control points according to the mapping relationship, wherein a mapping model is used to convert a feature vector into corresponding weight coefficients.

7. The method of claim 1, wherein rendering the animation data according to the weight coefficients and the voice data comprises: putting the voice data into a first thread and putting the weight coefficients into a second thread; acquiring a playing time corresponding to the voice data played by the first thread in a process of updating the weight coefficients in the second thread; determining a keyword corresponding to the morpheme data according to the playing time; determining a target weight coefficient corresponding to the keyword and adjusting the animation data according to the target weight coefficient; and rendering the adjusted animation data at the playing time.

8. The method of any one of claims 1 to 7, further comprising: determining an original format of the voice data and a target format required by a playing device when playing the voice data; comparing the original format and the target format to obtain a comparison result; in a case where the comparison result indicates that the original format is inconsistent with the target format, acquiring an original number of channels, an original sampling rate in the original format, and acquiring a target number of channels and a target sampling rate in the target format; changing the original number of channels to the target number of channels and changing the original sampling rate to the target sampling rate.

9. An apparatus for animation rendering, comprising: an acquiring module configured to acquire text information and animation data required for animation rendering, wherein the text information is used to describe the animation data; a determining module configured to determine voice data and morpheme data corresponding to the text information; a converting module configured to convert the morpheme data into weight coefficients of control points in the animation data, wherein the control points are used to control a shape of a model in the animation data; a rendering module configured to render the animation data according to the weight coefficients and the voice data.

10. The apparatus of claim 9, wherein the acquiring module is further configured to: acquire a preset character set, wherein the preset character set includes characters recognizable by a computer; determine all text characters included in the text information; determine whether each character in the text characters is included in the preset character set to obtain a determination result corresponding to each character; retain characters in the text information indicated by the determination result as being included in the preset character set and delete characters in the text information indicated by the determination result as not being included in the preset character set.

11. The apparatus of claim 9, wherein the determining module is further configured to: determine computer resources required for processing the text information and acquire locally remaining available computer resources; compare the required computer resources and the available computer resources to obtain a comparison result; and determine a target weight coefficient corresponding to the keyword and adjust the animation data according to the target weight coefficient; and render the adjusted animation data at the playing time. determine a manner of processing the text information according to the comparison result. 12.The apparatus of claim 11, wherein the determining module is further configured to: process the text information in a first manner in a case that the comparison result indicates that computer resources required for processing the text information are less than the available computer resources, wherein the first manner is processing the text information locally; process the text information in a second manner in a case that the comparison result indicates that computer resources required for processing the text information are greater than the available computer resources, wherein the second manner is processing the text information in a server. 13.The apparatus of claim 9, wherein the determining module is further configured to: convert the text information into speech data corresponding to the text information by using a speech model, wherein the speech model is used to convert text into speech; perform word segmentation processing on the text information, and perform part-of-speech tagging on each segmented word obtained; determine a key word from the segmented word result according to the tagged part-of-speech, and determine the key word as morpheme data corresponding to the text information. 14.The apparatus of claim 9, wherein the converting module is further configured to: determine a mapping relationship between the morpheme data and the control points; determine a morpheme feature vector according to the morpheme data, wherein the morpheme feature vector is used to represent an association relationship between the morpheme data; map the morpheme feature vector to a weight coefficient of the control points by using a mapping model according to the mapping relationship, wherein the mapping model is used to convert a feature vector into a corresponding weight coefficient. 15.The apparatus of claim 9, wherein the converting module is further configured to: determine an original format of the speech data, and a target format required when a playback device plays the speech data; compare the original format and the target format to obtain a comparison result; in a case that the comparison result indicates that the original format is inconsistent with the target format, obtain an original number of channels, an original sampling rate in the original format, and obtain a target number of channels and a target sampling rate in the target format; change the original number of channels to the target number of channels, and change the original sampling rate to the target sampling rate. 16.The apparatus of claim 9, wherein the rendering module is further configured to: put the speech data into a first thread, and put the weight coefficient into a second thread; obtain a playback time corresponding to the speech data played by the first thread in a process of updating the weight coefficient in the second thread; determine a corresponding key word in the morpheme data according to the playback time; determine a target weight coefficient corresponding to the key word, and adjust the animation data according to the target weight coefficient; render the adjusted animation data at the playback time. 17.An electronic device, comprising: a memory configured to store computer program instructions; and A processor, connected with the memory, for executing the following steps when running the computer program instructions: obtaining text information and animation data required for animation rendering, wherein the text information is used to describe the animation data; determining phonetic data and morpheme data corresponding to the text information; converting the morpheme data into weight coefficients of corresponding control points in the animation data, wherein the control points are used to control the shape of a model in the animation data; and rendering the animation data according to the weight coefficients and the phonetic data. 18.A non-volatile computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, performs the animation rendering method of any one of claims 1 to 8. 19.A computer program product comprising computer instructions, wherein the computer instructions, when executed by a processor, implement the method of animation rendering of any one of claims 1 to 8.

Images