Virtual digital human voice broadcast system and method

Abstract

The invention provides a virtual digital human voice broadcast system and method. The virtual digital human voice broadcast system comprises a virtual digital human generation module, a broadcast text acquisition module, a broadcast text configuration module, a broadcast text-to-voice module and a broadcast module. Wherein the virtual digital person generation module is used for generating a virtual digital person video according to the target person image. And the broadcast text acquisition module is used for extracting the index name and the data value corresponding to the index name according to the query result. And the broadcast text configuration module is used for querying and determining the verbal skill template from the fixed verbal skill database according to the query result, and combining the verbal skill template with the data value corresponding to the index name to generate a broadcast text. And the broadcast text-to-voice module is used for calling the AI voice synthesis service to convert the broadcast text into broadcast voice. And the broadcasting module is used for playing the virtual digital human video and broadcasting voice. In this way, the query result can be broadcasted by voice through the virtual digital human, so that the use experience of the user is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of digital human, and in particular to a virtual digital human voice broadcasting system and method. Background Art

[0002] As telecom operators' operation and management systems gradually improve, the basic management unit has been refined to grid-based unit management. Specifically, telecom operators can divide the operation area into multiple grid areas according to regions, and collect and manage information such as the population, service users, and business attributes in each grid.

[0003] The operation management system can track the grid dispatch execution and the changing trends of various business indicators, and can perform grid diagnosis from the dimensions of grid revenue, key indicators, scale, etc. In addition, the operation management system can also quickly query information based on user needs based on the grid area.

[0004] However, the existing operation management system of telecom operators usually presents the query results in text form, and users can only browse the query results visually. The operation management system has a single display method for query results, which reduces the user experience. Summary of the invention

[0005] The technical problem to be solved by the present invention is the problem of low user experience in the operator's operation management system.

[0006] In order to solve the above technical problems, the present invention provides a virtual digital human voice broadcasting system and method, which specifically adopts the following technical solutions:

[0007] In the first aspect, the present invention provides a virtual digital human voice broadcast system, including: a virtual digital human generation module, a broadcast text acquisition module, a broadcast text configuration module, a broadcast text to speech module and a broadcast module. Among them, the virtual digital human generation module is used to generate a virtual digital human video based on a target person image. The broadcast text acquisition module is used to extract the indicator name and the data value corresponding to the indicator name according to the query result, and the query result is determined according to the query request. The broadcast text configuration module is used to query and determine the speech template from a fixed speech database according to the query result, and combine the speech template and the data value corresponding to the indicator name to generate a broadcast text. The broadcast text to speech module is used to call an artificial intelligence (AI) speech synthesis service to convert the broadcast text into a broadcast voice. The broadcast module is used to play virtual digital human videos and broadcast voices.

[0008] The system can generate a virtual digital human image through a virtual digital human generation module using virtual technology, can realize text broadcasting through a broadcast text-to-speech module and a broadcast module using text-to-speech technology, can realize configuration of broadcast content, acquisition of voice content, and call of text-to-speech capability through JAVA technology, and can realize virtual digital human interaction and voice broadcasting through front-end interactive technology. In this way, voice broadcasting of query results can be realized through virtual digital humans, thereby improving the user experience.

[0009] In combination with the first aspect, in an optional implementation, in the above-mentioned generation of a virtual digital human video based on a target person image, the virtual digital human generation module is specifically used to: first, generate a preset type image based on the target person image. Then, add lip movement information to the preset type image image to generate a preset type image with added lip shape. Next, perform image cutout on the preset type image with added lip shape to generate a virtual person losslessly compressed bitmap graphic format (portable network graphics, png) sequence. Finally, perform dynamic processing and image compression on the virtual person png sequence to generate a virtual digital human video.

[0010] In combination with the first aspect, in an optional implementation method, the above-mentioned broadcast text configuration module is based on the SpringCloud microservice framework and dynamically queries the speech template through structured query language (SQL).

[0011] In combination with the first aspect, in an optional implementation, the system further includes: a virtual digital human control module. Specifically, the virtual digital human control module is used to control the broadcast module to start playing the virtual digital human video and broadcast voice, and is also used to control the broadcast module to hide the virtual digital human video and stop playing the broadcast voice.

[0012] In conjunction with the first aspect, in an optional implementation, the system further includes: a voice content interaction module. Specifically, the voice content interaction module is used to highlight the broadcast text when playing the virtual digital human video and broadcasting the voice.

[0013] In combination with the first aspect, in an optional implementation, in the above-mentioned highlighting of the broadcast text, the voice content interaction module is specifically used to: highlight the broadcast text through js element positioning and highlight rendering processing.

[0014] In the second aspect, the present invention provides a method for broadcasting a virtual digital human voice, the method comprising: first, generating a virtual digital human video based on a target person image. Then, extracting the indicator name and the data value corresponding to the indicator name according to the query result, and the query result is determined according to the query request. Secondly, according to the query result, query and determine the speech template from the fixed speech database, combine the speech template and the data value corresponding to the indicator name, and generate a broadcast text. Next, call the AI ​​speech synthesis service to convert the broadcast text into a broadcast voice. Finally, play the virtual digital human video and the broadcast voice.

[0015] In combination with the second aspect, in an optional implementation, the method further includes: highlighting the broadcast text when playing the virtual digital human video and the broadcast voice.

[0016] According to a third aspect, an electronic device is provided, comprising: a memory and one or more processors; the memory is coupled to the processor; wherein the memory stores computer program code, the computer program code comprises computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the method according to the second aspect and any one of the optional methods thereof.

[0017] According to a fourth aspect, a computer-readable storage medium is provided, comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method according to the second aspect and any optional method thereof.

[0018] It can be understood that the beneficial effects that can be achieved by the virtual digital human voice broadcast method provided in the second aspect, the electronic device in the third aspect, and the computer-readable storage medium in the fourth aspect can refer to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 An architectural diagram of a virtual digital human voice broadcasting system provided by an embodiment of the present invention;

[0020] Figure 2 A display schematic diagram of the virtual digital human voice broadcasting system provided by an embodiment of the present invention applied to an operation management system;

[0021] Figure 3 A schematic diagram of a flow chart of a virtual digital human voice broadcasting method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0023] As the operation and management systems of telecom operators are gradually improved, the basic management units have been refined to grid-based unit management. Specifically, telecom operators can divide the operation area into multiple grid areas according to regions, and collect and manage information such as the population, service users, and business attributes in each grid. In this way, the operation and management system can quickly query information based on user needs based on the grid area.

[0024] The operation management system can track the grid dispatch execution and the changing trends of various business indicators, and can perform grid diagnosis from the dimensions of grid revenue, key indicators, scale, etc. In addition, the operation management system can also quickly query information based on user needs based on the grid area.

[0025] However, the existing operation management system of telecom operators usually presents the query results in text form, and users can only browse the query results visually. The operation management system has a single display method for query results, which reduces the user experience.

[0026] In order to solve the above problems, the embodiment of the present application provides a virtual digital human voice broadcast system and method, which can be applied to the operation management system of telecommunications operators. Specifically, the system is composed of modules such as a virtual digital human generation module, a broadcast text acquisition module, a broadcast text configuration module, a broadcast text to voice module, and a broadcast module. The system can generate a virtual digital human image by using virtual technology through the virtual digital human generation module, and can realize text broadcasting by using the broadcast text to voice module and the broadcast module using text to voice technology. The configuration of the broadcast content, the acquisition of the voice content, and the call of the text to voice capability can be realized through JAVA technology, and the interaction of the virtual digital human and voice broadcast can be realized through the front-end interactive technology, so as to realize voice broadcasting through the virtual digital human, thereby improving the user experience of the operation management system.

[0027] The solution provided by the embodiment of the present application is introduced below in conjunction with the accompanying drawings.

[0028] For details, see Figure 1 , is an architecture diagram of a virtual digital human voice broadcasting system provided by an embodiment of the present invention, such as Figure 1As shown, the virtual digital human voice broadcasting system 100 provided in the embodiment of the present application includes: a virtual digital human generation module 110, a broadcast text acquisition module 120, a broadcast text configuration module 130, a broadcast text to voice conversion module 140 and a broadcast module 150.

[0029] The virtual digital human generation module 110 can be used to generate a virtual digital human video according to a target person image. Specifically, the target person image can be a photo of the target person, and the virtual digital human generation module 110 can use AI technology to convert the target person image into a virtual digital human video.

[0030] In some embodiments, in generating a virtual digital human video according to a target person image, the virtual digital human generation module 110 is specifically used to:

[0031] First, a preset type image is generated based on the target person image.

[0032] The preset type of image may be a cartoon image. For example, the virtual digital human generation module 110 may collect materials and assemble puzzles based on the target person image and target actions, and perform conversion processing through SD software to generate a cartoon image.

[0033] Then, the lip movement information is added to the preset type image to generate the preset type image with the added lip movement.

[0034] In one implementation, the lip movement information can be added through the D-ID software to generate a preset type image with added lip movement.

[0035] Next, the preset type image with the lip shape added is cut out to generate a virtual character png sequence.

[0036] Finally, the virtual character PNG sequence is dynamically processed and compressed to generate a virtual digital human video.

[0037] Exemplarily, the virtual digital human generation module 110 can generate two sets of images from the virtual human png sequence. The width of the image can be 450 pixels (PX), and the height can be a single image*40. The two sets of images are then spliced ​​and exported in png format to obtain an image stream of the virtual digital human. The image stream of the virtual digital human is then compressed to generate a virtual digital human video.

[0038] For example, Figure 2 A schematic diagram of a virtual digital human voice broadcast system provided by an embodiment of the present invention for use in an operation management system. The virtual digital human voice broadcast system is applied to the operation management system of operator B. Figure 2As shown, a generated virtual digital person is displayed in the display area 201 of the display interface of the operation management system of operator B for voice broadcasting.

[0039] The report text acquisition module 120 can be used to extract the indicator name and the data value corresponding to the indicator name according to the query result.

[0040] The query result is determined according to the query request. Taking the application of the virtual digital human voice broadcast system provided by the embodiment of the present invention to the operator's operation management system as an example, the operation management system can determine the query result according to the user's query request and display the query result on the smart screen.

[0041] Furthermore, the broadcast text acquisition module 120 can obtain the data (i.e., query results) of the front-end page (i.e., the smart screen display page) in real time, and obtain the indicator names and data on the smart screen in real time, that is, extract the indicator names in the query results and the data values ​​corresponding to the indicator names.

[0042] For example, when the query request is "query the permanent population, area, number of communities and number of group customers in grid A", the query result may be "220,100 permanent population, 18.06 square kilometers, 190 communities and 126 group customers". The indicator names include: permanent population, area, number of communities and number of group customers. The data values ​​corresponding to the above indicator names include: 22.01, 18.06, 190 and 126.

[0043] In one implementation, the broadcast text acquisition module 120 can extract the indicator name and the data value corresponding to the indicator name by querying the script (for example: / getScripts) and obtaining all paragraph details instructions under the script ( / fetchScriptParagraphs).

[0044] The broadcast text configuration module 130 can be used to query and determine the speech template from the fixed speech database according to the query results, and combine the speech template with the data value corresponding to the indicator name to generate the broadcast text.

[0045] Specifically, the virtual digital human voice broadcasting system provided by the embodiment of the present invention also includes a fixed speech database. The fixed speech database stores preset speech templates, and the corresponding speech template can be determined according to the indicator name included in the query result.

[0046] For example, taking the indicator names including: permanent population, area, number of communities and number of group customers as an example, the speech template corresponding to the indicator name can be "Let me show you below, please see: () grid has a permanent population of () million households, an area of ​​() square kilometers, and has entered the grid () communities and () group customers."

[0047] Furthermore, the broadcast text configuration module 130 combines the speech template and the data value corresponding to the indicator name, and the generated broadcast text may be "Let me show you below, please see: Grid A has a permanent population of 220,100 households, an area of ​​18.06 square kilometers, and has 190 communities and 126 group customers."

[0048] In some embodiments, the broadcast text configuration module 130 can be based on the SpringCloud microservice framework, dynamically query the speech template through SQL, so as to realize the configuration of fixed speech, and support the introduction of parameters (ie, the data value corresponding to the indicator name) to form a configurable broadcast text.

[0049] The broadcast text to speech module 140 can be used to call the AI ​​speech synthesis service to convert the broadcast text into the broadcast speech.

[0050] Next, the broadcast text to speech module 140 can convert the broadcast text generated by the broadcast text configuration module 130 into a broadcast speech by calling the AI ​​speech synthesis service. In one implementation, the broadcast text to speech module 140 can convert the broadcast text into a broadcast speech by using a Java method to call the AI ​​speech synthesis service through the Base64 encoding to language file technology.

[0051] The broadcast module 150 can be used to play the virtual digital human video and broadcast voice.

[0052] Finally, the broadcast module 150 can play the virtual digital human video and broadcast voice through the smart screen to broadcast the query results corresponding to the user's query request through the virtual digital human voice.

[0053] In one implementation, the broadcast module 150 can play the virtual digital human video and broadcast voice by calling the voice service interface and the batch real-time broadcast paragraph instruction ( / quickBatchBroadcastParagraph).

[0054] In some embodiments, Figure 1 As shown, the system also includes: a virtual digital human control module 160.

[0055] Specifically, the virtual digital human control module 160 can be used to control the broadcast module 150 to start playing the virtual digital human video and broadcast voice, and also to control the broadcast module 150 to hide the virtual digital human video and stop playing the broadcast voice.

[0056] In one implementation, the virtual digital human control module 160 can integrate the entrance of the virtual digital human in the overview page of the smart screen through the front-end interactive event. The user can wake up (i.e., display) the virtual digital human by clicking on the "virtual digital human" displayed on the page, and can start playing the virtual digital human video and voice broadcast. The user can also exit the virtual digital human by clicking on the "hide virtual digital human" on the page, i.e., hide the virtual digital human video and stop playing the voice broadcast.

[0057] In some embodiments, Figure 1 As shown, the system also includes: a voice content interaction module 170.

[0058] The voice content interaction module 170 can be used to highlight the broadcast text when playing the virtual digital human video and broadcasting the voice.

[0059] For example, Figure 2 As shown, the voice content interaction module 170 can highlight the announcement text "I will show you below: Grid A has a permanent population of 220,100 households, an area of ​​18.06 square kilometers, and has 190 communities and 126 group customers in the grid" in the display area 202 of the display interface of the operation management system of operator B. In this way, the user can browse the query results more intuitively, further improving the user experience.

[0060] In some embodiments, highlighting the broadcast text may be highlighting the broadcast text. Then the voice content interaction module 170 may be specifically used to highlight the broadcast text through js element positioning and highlight rendering processing.

[0061] The virtual digital human voice broadcast system provided by the embodiment of the present application can generate a virtual digital human image by using virtual technology through the virtual digital human generation module, realize text broadcasting through the broadcast text-to-speech module and the broadcast module using text-to-speech technology, realize the configuration of broadcast content, the acquisition of voice content, the call of text-to-speech capability through JAVA technology, and realize virtual digital human interaction and voice broadcasting through front-end interactive technology. In this way, the query results can be voice broadcasted by the virtual digital human, thereby improving the user experience.

[0062] The present application also provides a virtual digital human voice broadcasting method, which can be applied to Figure 1 The virtual digital human voice broadcasting system shown in FIG. Figure 3 A flow chart of a virtual digital human voice broadcasting method provided by an embodiment of the present invention, as shown in FIG. Figure 3 As shown, the method includes the following steps S101-S105:

[0063] S101, generating a virtual digital human video according to a target person image.

[0064] S102: extracting the indicator name and the data value corresponding to the indicator name according to the query result, wherein the query result is determined according to the query request.

[0065] S103. Query and determine a speech template from a fixed speech database according to the query results, combine the speech template with the data value corresponding to the indicator name, and generate a report text.

[0066] S104. Call the AI ​​speech synthesis service to convert the broadcast text into broadcast speech.

[0067] S105: Play the virtual digital human video and broadcast the voice.

[0068] In some embodiments, the virtual digital human voice broadcasting method further includes: highlighting the broadcasting text while playing the virtual digital human video and the broadcasting voice.

[0069] The virtual digital human voice broadcast method provided in the embodiment of the present application uses virtual technology to generate a virtual digital human image, uses text-to-speech technology to realize text broadcasting, uses JAVA technology to realize the configuration of broadcast content, the acquisition of voice content, and the call of text-to-speech capability, and realizes virtual digital human interaction and voice broadcasting through front-end interactive technology. In this way, the query results are broadcasted by the virtual digital human, thereby improving the user experience.

[0070] An embodiment of the present invention further provides an electronic device, which may include: a display screen, a memory, and one or more processors. The display screen, the memory, and the processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device can execute the various methods or steps executed in the above-mentioned virtual digital human voice broadcast method embodiment. Of course, the electronic device includes but is not limited to the above-mentioned display screen, the memory, and one or more processors.

[0071] The embodiment of the present invention also provides a computer-readable storage medium for storing computer instructions for running the above-mentioned virtual digital human voice broadcasting method.

[0072] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0073] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0074] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0075] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0076] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A virtual digital human voice broadcasting system, characterized in that: include: Virtual digital human generation module, broadcast text acquisition module, broadcast text configuration module, broadcast text to speech module and broadcast module; Wherein, the virtual digital human generation module is used to generate a virtual digital human video according to the target person image; The report text acquisition module is used to extract the indicator name and the data value corresponding to the indicator name according to the query result, and the query result is determined according to the query request; The report text configuration module is used to query and determine the speech template from the fixed speech database according to the query result, and combine the speech template with the data value corresponding to the indicator name to generate the report text; The broadcast text to speech module is used to call the AI ​​speech synthesis service to convert the broadcast text into broadcast speech; The broadcast module is used to play the virtual digital human video and the broadcast voice.

2. The system according to claim 1, characterized in that In the process of generating a virtual digital human video according to the target person image, the virtual digital human generation module is specifically used for: Generate a preset type of image according to the target person image; Adding lip movement information to the preset type image to generate a preset type image with added lip movement; Cut out the preset type image with added lip shape to generate a virtual character png sequence; The virtual character PNG sequence is dynamically processed and image compressed to generate the virtual digital human video.

3. The system according to claim 1, characterized in that The broadcast text configuration module is based on the SpringCloud microservice framework and dynamically queries the speech template through SQL.

4. The system according to claim 1, characterized in that The system further comprises: a virtual digital human control module; The virtual digital human control module is used to control the broadcast module to start playing the virtual digital human video and the broadcast voice, and is also used to control the broadcast module to hide the virtual digital human video and stop playing the broadcast voice.

5. The system according to any one of claims 1 to 4, characterized in that: The system also includes: a voice content interaction module; The voice content interaction module is used to highlight the announcement text when playing the virtual digital human video and the announcement voice.

6. The system according to claim 5, characterized in that In the step of highlighting the broadcast text, the voice content interaction module is specifically used to: The broadcast text is highlighted through js element positioning and highlight rendering processing.

7. A virtual digital human voice broadcasting method, characterized in that: include: Generate a virtual digital human video based on the target person image; Extracting the indicator name and the data value corresponding to the indicator name according to the query result, wherein the query result is determined according to the query request; According to the query result, a speech template is queried from a fixed speech database to determine the speech template, and the speech template is combined with the data value corresponding to the indicator name to generate a report text; Calling the AI ​​speech synthesis service to convert the broadcast text into broadcast speech; Play the virtual digital human video and the broadcast voice.

8. The method according to claim 7, characterized in that The method further comprises: When playing the virtual digital human video and the announcement voice, the announcement text is highlighted.

9. An electronic device, characterized in that: include: A memory and one or more processors; the memory is coupled to the processor; wherein the memory stores computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the method as claimed in claim 7 or 8.

10. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method as claimed in claim 7 or 8.

Images