A method and system for establishing and communicating with digital characters

Abstract

The present invention discloses a method and system for establishing a digital character and communicating with the digital character. The present invention proposes a new method for establishing a digital character from the character's non-chat record information and chat record information; proposes the concept of an era database and a method for linking an era database and a character database; proposes a new method for analyzing user questions, searching for relevant answers through the character and era databases and returning them to the user. The present invention changes the previous practice of only understanding the situation of each character through non-interactive media such as text introductions and statue portraits. When the system of the present invention is placed near various museums, memorial halls, monuments, statues, etc., it can enhance the interaction between users and characters, allowing historical figures to interact with users vividly, and increasing users' interest in learning and understanding character-related information.

Description

Technical Field

[0001] The present invention belongs to the technical field of human-computer interaction, and in particular relates to a method and system for establishing and communicating with a digital character. Background Art

[0002] When existing chatbots or chat systems talk to users, they are usually based on the training results of big data, but the answers are the same for everyone, not for a specific person. Each person has his / her era mark and background, but existing chatbots or chat systems do not take into account the information and characteristics of the era in which the person lives. Summary of the invention

[0003] The purpose of the present invention is to address the deficiencies of the prior art and provide a method and system for establishing and communicating with digital characters, thereby changing the traditional way of recording characters using text / photos / videos and making the characters "come alive" and able to communicate with users.

[0004] The purpose of the present invention is achieved through the following technical solutions: the present invention proposes a new method for establishing a digital character from non-chat record data (such as the Internet, dictionaries, resumes, archives, etc.) and chat record data of the character; proposes the concept of era database and the linking method of era database and character database; proposes a new method for analyzing user questions (rather than traditional large corpus training), searching for relevant answers through character and era databases and returning them to the user. The specific implementation steps of the present invention are as follows:

[0005] S1: Establishing a digital character database: Analyze the non-chat record data and chat record data of each character respectively, and construct a character information automatic classification result table respectively. In the character information automatic classification result table, each character corresponds to multiple tables, and each table is divided into several subcategories; after merging the character information automatic classification result tables constructed by the non-chat record data and the chat record data, a digital character database is formed;

[0006] S2: Establish an era database: Create a chronological table framework for each region of the world in chronological order, and construct an automatic classification result table of era information, including the year, major events that occurred, and major figures; use the figures as clues to establish a connection between the digital character database and the era database;

[0007] S3: Establish a dialogue module: The user selects a digital character and distinguishes whether the user's question is a personal question or a non-personal question; for personal questions, first activate the digital character database corresponding to the character, and when the digital character database cannot answer, activate the era database; for non-personal questions, activate the era database; generate answers to the user through database keyword matching.

[0008] Furthermore, in step S1, the materials of each character are sorted into a digital character database, and there are two different implementation methods according to different data sources:

[0009] S11: Analysis of the non-chat record data of the character and establishment of character database:

[0010] Step 1: Find information about person X from various sources other than chat records, including text descriptions, photos, videos, and audio information;

[0011] Step 2: Set up multiple forms for characters, divide each form into subcategories, and create a table for automatically classifying character information;

[0012] Step 3: Use text classification technology to automatically classify the character's text description into various subcategories in the character information automatic classification result table;

[0013] Step 4: Use computer vision technology to analyze and classify photos and videos, and automatically classify the original files of photos and videos into various subcategories in the character information automatic classification result table, and process them in the following two situations:

[0014] Case 1: If the classification error rate during the sub-classification process is greater than the threshold, the original files of the photos and videos are automatically classified into the various forms of the person in the person information automatic classification result table instead of the sub-classification; if the error rate of automatic classification into the form is still greater than the threshold, the original files of the photos or videos are not classified, and the original files are placed in the photo / video row of a person that cannot be classified in the person information automatic classification result table, or they are manually classified into sub-classifications;

[0015] Case 2: If the classification error rate is less than or equal to the threshold during the classification process, the original photo or video file is automatically classified into each subcategory of the person in the person information automatic classification result table;

[0016] Step 5: Analyze the audio file:

[0017] 1) Use audio analysis technology to divide the audio into voice audio and non-voice audio, and retain the original audio file;

[0018] 2) For the speech audio, the information in the audio is converted into text using speech recognition technology, and the content of the converted text is added to each subcategory of the character information automatic classification result table according to step 3; according to the classification result of the recognized text, the original audio file is automatically classified into the corresponding subcategory of the character information automatic classification result table;

[0019] 3) For non-speech audio, it will not be automatically classified. The original audio file will be placed in the audio row of a certain person that cannot be classified in the character information automatic classification result table, or manually classified into a sub-category;

[0020] Step 6: Repeat steps 1 to 5 to integrate each character into the character information automatic classification result table;

[0021] S12: Analyze the chat records of the character and establish a character database:

[0022] Step 1: Identify which party a person X is in the chat record, mark what X said as XS, and what Y who talked to X said as YS;

[0023] Step 2: Analyze the text information in the chat content, specifically:

[0024] 1) Classify the content XS said by X according to the category of digital characters;

[0025] 2) In a round of dialogue, for the content YS said by Y, as long as X's reply is affirmative, the content YS said by Y is classified according to the category of digital characters;

[0026] Step 3: Analyze the non-text information such as photos, videos and audio in the chat content, specifically:

[0027] 1) According to the method in steps 4 and 5 of S11, the content XS said by X is summarized into the character information automatic classification result table;

[0028] 2) In a round of dialogue, for a non-text file sent by Y, as long as X's reply is affirmative, the content YS said by Y is summarized into the character information automatic classification result table according to the method in steps 4 and 5 of S11;

[0029] Step 4: Repeat steps 1 to 3 to integrate each character into the character information automatic classification result table.

[0030] Furthermore, in step S2, establishing the era database specifically includes the following sub-steps:

[0031] Step 1: Create a chronological table framework for each region of the world in chronological order, and construct an automatic classification result table of era information, including the year, major events, and major figures;

[0032] Step 2: According to the method of establishing a digital character database, using the era as the starting clue, automatically search the Internet or various electronic dictionaries / encyclopedias to obtain information, and automatically classify it into various subcategories of the era information automatic classification result table;

[0033] Step 3: Use characters as clues to link the character database and the era database.

[0034] Furthermore, in step S3, the dialogue module includes four submodules:

[0035] Submodule 1: Module for distinguishing between impersonal and personal issues

[0036] Use artificial neural network to distinguish whether it is a personal problem or a non-personal problem. If it is a personal problem, enter submodule 2; if it is a non-personal problem, enter submodule 3;

[0037] Submodule 2: Personal question answering module, the specific implementation includes the following steps:

[0038] Step 1: According to the character selected by the user, activate all rows corresponding to the character in the digital character database;

[0039] Step 2: Segment the words in the text information line activated by the character in the digital character database to obtain "data keywords";

[0040] Step 3: Extract keywords from the user's question. If no keywords are extracted, enter submodule 3;

[0041] Step 4: Generate synonyms of keywords;

[0042] Step 5: According to the question keywords and their synonyms, search for matching data keywords and their synonyms in the activation row of the character in the digital character database;

[0043] 1) If no data keyword or synonym is matched, the user is asked and guided to change the wording, or other questions are asked, and the submodule 1 is returned;

[0044] 2) If there is only one matching data keyword including its synonyms, then find a grid to which the keyword belongs, and send the content of the activated column corresponding to the grid where the matching data keyword is located to submodule 4;

[0045] 3) If there are multiple matching data keywords including their synonyms, the following operations are performed:

[0046] 3.1) If the matched data keywords including their synonyms belong to the same grid in the character information automatic classification result table, the content of the activated column corresponding to the grid is sent to submodule 4;

[0047] 3.2) If these matched data keywords including their synonyms do not belong to the same grid in the result table of the automatic classification of personal information and are different columns of the same form, first find the column names corresponding to these grids, then ask the user which column to select specifically, and send the contents of the grid corresponding to the column selected by the user to submodule 4 according to the user's answer;

[0048] 3.3) If these matched data keywords including their synonyms do not belong to the same grid but to different forms, first find the form names corresponding to these grids, then ask the user which form to select. After the user selects the form, 3.1) and 3.2) are executed again in this form, and the content of the grid corresponding to the column selected by the user is sent to submodule 4 according to the user's answer;

[0049] Submodule 3: Non-personal question answering module, the specific implementation includes the following steps:

[0050] Step 1: Activate the row corresponding to the person selected by the user in the era database and find the era and region where the person lived;

[0051] Step 2: According to the era and region where the person lives, find the relevant data of the era and region in the activation era database, and then segment the text to obtain "data keywords";

[0052] Step 3: Extract keywords from the user's question. If keywords are extracted, proceed to step 4. If keywords are not extracted, search the entire content of the question on the Internet to obtain relevant information, and send the information to submodule 4.

[0053] Step 4: Generate synonyms of keywords;

[0054] Step 5: Based on the question keywords and their synonyms, search and try to match the data keywords and their synonyms of the era database obtained in step 2;

[0055] 1) If a "data keyword" is matched, the content of the grid where the data keyword of the era database is located is sent to submodule 4;

[0056] 2) If no "data keyword" is matched, the network is searched for relevant information based on the keyword in the user's question, and the information is sent to submodule 4;

[0057] Submodule 4: Answer generation module, which answers the user's questions based on the information obtained from submodules 2 and 3.

[0058] Furthermore, in step S1, character information is collected, organized and analyzed, and a digital character database is established. The character information collection and organization includes two methods. The first method is to manually organize the material of a certain character into the character database; the second method is to automatically organize the material of the character into the character database through an intelligent algorithm. The intelligent algorithm organizes the material based on two different data sources: non-chat record information and chat record information.

[0059] Furthermore, in step S1, the multiple forms of the character information automatic classification result table include a basic information form, a work information form, a personal experience information form, etc.; the basic information form includes the following subcategories: the character's name, gender, birth time, education level, and marital status; the work information form includes the following subcategories: main work / contribution, main skills / talents; the personal experience information form includes several subcategories of the character's life events.

[0060] Furthermore, in step S3, before the user starts a conversation with a digital character, the user selects a character that he wants to talk to from multiple digital characters, which can be achieved by clicking on a character's avatar or name, or scanning a character's QR code; after the user selects a character, the conversation module is turned on, and the user sends relevant questions or requests or inquiries through voice or text. If it is voice, it is automatically converted into text through voice recognition.

[0061] Furthermore, in step S3, the user's question is answered in the first-person manner of the character, including: for different types of questions, corresponding answer words are added before returning the found information; if the returned information includes photos or videos, the photos or videos are played.

[0062] A system for establishing and communicating with a digital character, the system comprising:

[0063] Digital character database establishment module: analyze the non-chat record data and chat record data of each character respectively, and construct the character information automatic classification result table respectively. Each character in the character information automatic classification result table corresponds to multiple forms, and each form is divided into several subcategories; after merging the character information automatic classification result table constructed by the non-chat record data and the chat record data, a digital character database is formed;

[0064] Era database building module: Create a chronological table framework for each region of the world in chronological order, and construct an automatic classification result table of era information, including years, major events, and major figures; use figures as clues to establish a connection between the digital character database and the era database;

[0065] Dialogue module: The user selects a digital character and distinguishes whether the user's question is a personal question or a non-personal question; for personal questions, the digital character database corresponding to the character is first activated. When the digital character database cannot answer, the era database is activated; for non-personal questions, the era database is activated; and the answer to the user is generated through database keyword matching.

[0066] Furthermore, all modules are implanted into a computer software, and the computer software is installed on a computer with a touch screen. The computer is placed in the scene required by the user, and the user selects and clicks the character avatar on the touch screen to interact.

[0067] Furthermore, the digital character database, era database and dialogue module are all placed in the cloud, and the database and dialogue module are accessed through the mobile terminal mobile phone APP as the front end. The user selects and clicks the character to be talked to through the mobile terminal mobile phone APP, and the mobile terminal can be connected to the projector.

[0068] The beneficial effects of the present invention are as follows: The present invention changes the previous practice of only using text introductions, statue portraits and other non-interactive media to understand the situation of various figures. When the system of the present invention is placed near various museums, memorials, monuments, statues, etc., it can enhance the interaction between users and figures, allowing historical figures to interact with users vividly, and increase users' interest in learning and understanding relevant information about the figures. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 is a system block diagram for establishing a digital character and communicating with the digital character in an embodiment of the present invention;

[0070] Figure 2 is a flow chart of a method for establishing a digital character database in an embodiment of the present invention;

[0071] Figure 3 This is the dialogue module processing flow in the embodiment of the present invention. DETAILED DESCRIPTION

[0072] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0073] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0074] The present invention discloses a method and system for establishing a digital character and having a simple conversation with the digital character through a certain method. The digital character can be an ordinary person, a celebrity including ancient and modern historical figures, heroes, or contemporary figures. The system of the present invention includes a digital character database, an era database, and a conversation module. The present invention discloses a method for establishing a digital character database, an era database, and a conversation module. The implementation method is as follows: Figure 1 The system of the present invention is not an expert system. The system of the present invention mainly focuses on the life story of the character and answers the user's questions about the character's life experience and main activities from the perspective of the character himself, rather than answering certain professional questions like an expert system. The specific implementation process of this embodiment is as follows:

[0075] S1: Building a digital character database

[0076] Collect character information, organize and analyze this information and establish a digital character database. There are two ways to collect and organize character information. The first is to manually organize the information of a certain character into the character database; the second is to automatically organize the information of the character into the character database through intelligent algorithms. Intelligent algorithms have the following two different implementation methods based on different data sources (non-chat record information and chat record information), such as Figure 2 shown.

[0077] S11: Method for analyzing the non-chat record data of the character (such as the Internet, dictionaries, resumes, archives, etc.) and establishing a character database:

[0078] Step 1: Through various sources of non-chat records, including the Internet, newspapers, self-narration, books and other media, as well as personal information such as resumes or files with the permission of the person (if the person is dead, then the permission of his relatives is required). Search for information about a person X in this information, including text description (Text-X) / photo (PX) / video (VX) / audio (AX) and other information. For different applications of this system, the source of the data can be emphasized. For example, if the system uses resumes to create digital characters, it will focus more on the source of text description information.

[0079] Step 2: The system user sets multiple forms for the digital character (such as basic information form, work information form, personal experience information form, etc.), and then divides each form into subcategories (see Table 1). For example, the basic information form may include the character's name, gender, date of birth, education level, marital status, etc.

[0080] Table 1: Results of automatic classification of character information (taking character 1 as an example)

[0081]

[0082] Step 3: Use text classification (including keyword recognition) technology to automatically classify the character's text description (Text-X) into the various subcategories in Table 1.

[0083] Step 4: Use computer vision technology to analyze and classify photos (PX) and videos (VX), and use supervised or unsupervised learning methods to automatically classify the original files of photos (PX) and videos (VX) into the various subcategories in Table 1. Due to the influence of the clarity of the original videos and photos and the number of user classifications, the following two cases are processed:

[0084] Case 1: If the classification error rate (assuming there are N videos, M of which are classified into the wrong sub-category, error rate = M / N) during the classification process is greater than 10% (this threshold standard can be adjusted by the user according to the actual system needs), the original files of the photo (PX) and video (VX) are automatically classified into the various forms of the digital person in Table 1 instead of the sub-category (see Table 1). If the error rate of automatic classification into the form is still greater than 10% (this threshold standard can be adjusted by the user according to the actual system needs), the original files of the photo (PX) or video (VX) are not classified (classified into the form or sub-category), and the original files are placed in the photo / video row of a person that cannot be classified in Table 1. Or sub-classification is performed manually.

[0085] Case 2: If the classification error rate is less than or equal to 10% during the classification process (this threshold standard can be adjusted by the user according to the actual system needs), the original file of the photo (PX) / video (VX) is automatically classified into the various subcategories of the digital character in Table 1 (see Table 1).

[0086] Step 5: Analysis of audio files:

[0087] 1) Use audio analysis technology to classify audio (AX) into speech audio and non-speech audio (such as music, etc.), and keep the original audio file.

[0088] 2) For voice audio, the system will use voice recognition technology to convert the information in the audio (AX) into text, and add the content of the converted text to each subcategory in Table 1 according to step 3 (see Table 1). According to the classification results of the recognized text (for example, a certain voice segment is related to the birth time of a person), the original audio file is automatically classified into the corresponding subcategory in Table 1 (for example, the audio subcategory corresponding to the birth time).

[0089] 3) For non-speech audio, it will not be automatically classified, and the original audio file will be placed in the audio row of a certain person that cannot be classified in Table 1. Or it can be manually classified into a small category.

[0090] Step 6: Repeat steps 1 to 5 to integrate the 2nd, 3rd, 4th, ... characters into one table 1.

[0091] S12: Method for analyzing the chat records of the character and establishing a character database:

[0092] For ordinary people who are not celebrities, there may not be as much information available as for celebrities, so a digital character of the character can be established from the chat records. A lot of information is generated during the chat between character X and other people (Y1, Y2, Y3...). Under the premise that the information is true, the conversation content can be analyzed and a character database can be established according to the following method.

[0093] Step 1: Correctly identify the party X is in the chat record, and mark what X said as XS (X's Saying, including text / voice / photo / short video), the person talking to X is Yi, and what Y said is marked as YS (Y's Saying, including text / photo / voice / short video).

[0094] Step 2: Analyze the text information in the chat content and run the following steps in sequence:

[0095] 1) Apply convolutional neural network (CNN) (see https: / / arxiv.org / pdf / 1408.5882.pdf for details) or other deep learning algorithms to classify the content XS said by digital person according to the categories (subcategories) (such as gender, age, career contribution, etc.) (a large number of samples are needed to train CNN or other deep learning algorithms).

[0096] 2) Analyze YS: In a round of dialogue, if YS says YS, as long as X’s reply is positive, a convolutional neural network (CNN) is applied to YS (see

[0097] https: / / arxiv.org / pdf / 1408.5882.pdf) or other deep learning algorithms to classify digital characters according to their categories (such as gender, age, career contributions, etc.) (a large number of samples are needed to train CNN or other deep learning algorithms).

[0098] Step 3: Analyze non-text information such as photos, videos, and audio in the chat content, specifically:

[0099] 1) According to the method in step 4 and step 5 of S11, the content XS said by X is summarized into the character information automatic classification result table 1;

[0100] 2) In a round of dialogue, for a non-text file sent by Y, as long as X's reply is affirmative, the content YS said by Y is summarized into the character information automatic classification result table 1 according to the method in steps 4 and 5 of S11.

[0101] Step 4: Repeat steps 1 to 3 to integrate each character into Table 1.

[0102] After merging the character information automatic classification result table constructed from the non-chat record data and the chat record data, a digital character database is formed.

[0103] S2: Establishing the era database

[0104] Because many of the user's requests / queries / questions (here we collectively call them questions) are related to the era in which the characters live, we answer the questions by building an era database.

[0105] According to the era and region, a database of various regions (including China and foreign countries) under the development of the era is compiled. This database includes the era of the character, the region where the event occurred, the main historical background, modern and current events. When the character database related to the character cannot answer a certain question, the era database will be activated to answer the question.

[0106] Step 1: First, establish a chronological table framework for each region of the world until today, including the years, major events, major figures, etc. (see Table 2);

[0107] Table 2: Automatic classification results of era information

[0108]

[0109]

[0110] Step 2: According to the method of establishing a digital character database (including keyword recognition technology, computer vision and speech recognition technology), with the year as the starting clue, the system automatically searches the Internet or various electronic dictionaries / dictionaries to obtain information (including text and non-text information), and automatically classifies it into various subcategories in Table 2;

[0111] Step 3: Use characters as clues to link the character database and the era database; this includes the relationship between characters and eras, as well as the relationship between characters.

[0112] S3: Establishing the Dialogue Module

[0113] Due to the variability and ambiguity of the user's natural language, the dialogue module is divided into four sub-modules. The first sub-module distinguishes whether the questions (including queries, questions or requests) from the user are questions related to the person's personal information (personal questions) or questions unrelated to the person's personal information (impersonal questions). The second sub-module is used to handle personal questions, the third sub-module is used to handle impersonal questions, and the fourth sub-module is used to generate answers to the user (see Figure 3 ).

[0114] Preparation before the conversation: Before the user starts the conversation with the digital character, the user first opens the system, and then selects a character he wants to talk to from multiple digital characters (the character's name is usually indicated below the character). This selection can be either clicking on a character's avatar or name, or scanning the character's QR code (see the implementation of the system). After the user clicks on this character, the conversation module is turned on (the user sends relevant questions, requests or queries in voice or text, and if it is voice, it is automatically converted into text through voice recognition).

[0115] Submodule 1: Module for distinguishing between impersonal and personal issues

[0116] Artificial neural networks are used to distinguish whether it is a personal question or a non-personal question. If it is distinguished as a personal question, it enters the personal question answering module (submodule 2); if it is distinguished as a non-personal question, it enters the non-personal question answering module (submodule 3).

[0117] Submodule 2: Personal question answering module, the specific implementation includes the following steps:

[0118] Step 1: according to the user's selection in the interface or voice recognition, all rows corresponding to the character in the digital character database are activated (such as a total of 9 rows of data for character 1 in Table 1);

[0119] Step 2: Segment the words in the text information line activated by the character in the digital character database to obtain "data keywords";

[0120] Step 3: Extract keywords from the user's question. If no keywords are extracted, enter the non-personal question answering module;

[0121] Step 4: Generate synonyms for the keywords in steps 1-3;

[0122] Step 5: According to the question keywords and their synonyms, search for matching data keywords and their synonyms in the activation row of the character in the digital character database;

[0123] 1) If no data keywords or synonyms are matched, ask and guide the user, "Hello, can you change the wording or ask me about my personal experience?" and return to submodule 1;

[0124] 2) If there is only one matching data keyword including its synonyms, then find a grid to which the keyword belongs, and send the content of the activated column corresponding to the grid where the matching data keyword is located, including text / photo / video, to the answer generation module;

[0125] 3) If there are multiple matching data keywords including their synonyms:

[0126] 3.1) If these matched data keywords including their synonyms belong to the same grid in Table 1, the content of the activated column corresponding to the grid including text / photo / video is sent to the answer generation module;

[0127] 3.2) If these matched data keywords including their synonyms do not belong to the same grid in Table 1 but are different columns in the same form, first find the column names corresponding to these grids, then generate a query to the user: "Hello, do you want to know my XXX (column name 1), YYY (column name 2), or XXX (column name 3)?", and send the content of the grid corresponding to the column selected by the user, including text / photo / video, to the answer generation module according to the user's answer;

[0128] 3.3) If these matched data keywords including their synonyms do not belong to the same grid but to different forms, first find the form names corresponding to these grids, and then generate a sentence to ask the user: "Hello, do you want to know my XXX (form name 1), YYY (form name 2), or XXX (form name 3)?" After the user selects a form, 3.1) and 3.2) are run in this form, and the content of the corresponding grid of the column selected by the user, including text / photo / video, is sent to the answer generation module according to the user's answer.

[0129] Submodule 3: Non-personal question answering module, the specific implementation includes the following steps:

[0130] Step 1: Based on the user's selection in the interface or voice recognition, activate the row corresponding to the person in the era database to find the era and region where the person lived;

[0131] Step 2: According to the era and region where the person lives, find the relevant data of the era and region in the activation era database, and then segment the text to obtain "data keywords";

[0132] Step 3: Extract keywords from the user's question. If keywords are extracted, proceed to step 4. If keywords are not extracted, search the entire content of the question on the Internet (such as Baidu, Wikipedia, etc.) to obtain relevant information, and send the information to submodule 4.

[0133] Step 4: Extract the keywords from step 1 to step 3 and generate synonyms of these keywords;

[0134] Step 5: Based on the question keywords and their synonyms, search and try to match the data keywords and their synonyms of the era database obtained in step 2;

[0135] 1) If a "data keyword" is matched, the content of the grid where the data keyword of the era database is located is sent to submodule 4;

[0136] 2) If no "data keyword" is matched, the network (such as Baidu Wikipedia and other websites) is searched according to the keywords in the user's question to obtain relevant information, and the information is sent to submodule 4;

[0137] Submodule 4: Answer generation module

[0138] After obtaining information from submodules 2 and 3, submodule 4 will answer the user's question in the first-person perspective of the character. The operation steps are as follows:

[0139] Step 1: For different types of questions, such as why questions, the system adds corresponding answer words before returning the found information. For example, for why questions, the system will add "because..."

[0140] Step 2: If the returned information also includes photos or videos, play the photos / videos.

[0141] In addition, you can use existing speech synthesis technology to simulate human speaking voices, or use PhotoToVideo technology to convert a photo of a person into an animation / short video that can speak and include different emotions.

[0142] The system of the present invention has three implementation modes:

[0143] 1) All modules are implanted into a computer software, which is installed on a computer with a touch screen. The computer is then placed as a hardware at the location where the user needs it (such as a museum, science and technology museum, etc.). The user selects and clicks on the avatar on the touch screen to interact;

[0144] 2) The database and the dialogue module are placed in the cloud, and the database and the dialogue module are accessed through the mobile terminal mobile phone APP as the front end. The user can select and click the person to talk to through the mobile terminal mobile phone APP, and the mobile terminal can be connected to a projector, etc., so that the presented picture is larger.

[0145] 3) Combine the text / picture description of the system and the character, and place the system's QR code next to the character's introduction (such as in scenic spots, tourist spots, museums, science and technology museums, tombstones, etc.). Users can download the APP after scanning the QR code with their mobile phones. The QR code also contains the character's information, so that users can interact with the character through the second implementation method.

[0146] The above is only a preferred embodiment of the present invention. Although the present invention has been disclosed as a preferred embodiment, it is not intended to limit the present invention. Any technician familiar with the art can make many possible changes and modifications to the technical solution of the present invention by using the above disclosed methods and technical contents without departing from the scope of the technical solution of the present invention, or modify it into an equivalent embodiment of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of protection of the technical solution of the present invention.

Claims

1. A method for creating a digital character and communicating with the digital character, characterized in that: The method comprises the following steps: S1: Establishing a digital character database: Analyze the non-chat record data and chat record data of each character respectively, and construct a character information automatic classification result table respectively. In the character information automatic classification result table, each character corresponds to multiple tables, and each table is divided into several subcategories; after merging the character information automatic classification result tables constructed by the non-chat record data and the chat record data, a digital character database is formed; S2: Establish an era database: Create a chronological table framework for each region of the world in chronological order, and construct an automatic classification result table of era information, including the year, major events that occurred, and major figures; use the figures as clues to establish a connection between the digital character database and the era database; S3: Establishing a dialogue module: The user selects a digital character, and distinguishes whether the user's question is a personal question or a non-personal question; for personal questions, first activate the digital character database corresponding to the character, and when the digital character database cannot answer, activate the era database; for non-personal questions, activate the era database; generate an answer to the user through database keyword matching; the dialogue module includes 4 sub-modules: Submodule 1: Module for distinguishing between impersonal and personal issues Use artificial neural network to distinguish whether it is a personal problem or a non-personal problem. If it is a personal problem, enter submodule 2; if it is a non-personal problem, enter submodule 3; Submodule 2: Personal question answering module, the specific implementation includes the following steps: Step 1: According to the character selected by the user, activate all rows corresponding to the character in the digital character database; Step 2: Segment the words in the text information line activated by the character in the digital character database to obtain "data keywords"; Step 3: Extract keywords from the user's question. If no keywords are extracted, enter submodule 3; Step 4: Generate synonyms of keywords; Step 5: According to the question keywords and their synonyms, search for matching data keywords and their synonyms in the activation row of the character in the digital character database; 1) If no data keyword or synonym is matched, the user is asked and guided to change the wording, or other questions are asked, and the submodule 1 is returned; 2) If there is only one matching data keyword including its synonyms, then find a grid to which the keyword belongs, and send the content of the activated column corresponding to the grid where the matching data keyword is located to submodule 4; 3) If there are multiple matching data keywords including their synonyms, the following operations are performed: 3.1) If the matched data keywords including their synonyms belong to the same grid in the character information automatic classification result table, the content of the activated column corresponding to the grid is sent to submodule 4; 3.2) If these matched data keywords including their synonyms do not belong to the same grid in the result table of the automatic classification of personal information and are different columns of the same form, first find the column names corresponding to these grids, then ask the user which column to select specifically, and send the contents of the grid corresponding to the column selected by the user to submodule 4 according to the user's answer; 3.3) If these matched data keywords including their synonyms do not belong to the same grid but to different forms, first find the form names corresponding to these grids, then ask the user which form to select. After the user selects the form, 3.1) and 3.2) are executed again in this form, and the content of the grid corresponding to the column selected by the user is sent to submodule 4 according to the user's answer; Submodule 3: Non-personal question answering module, the specific implementation includes the following steps: Step 1: Activate the row corresponding to the person selected by the user in the era database and find the era and region where the person lived; Step 2: According to the era and region where the person lives, find the relevant data of the era and region in the activation era database, and then segment the text to obtain "data keywords"; Step 3: Extract keywords from the user's question. If keywords are extracted, proceed to step 4. If keywords are not extracted, search the entire content of the question on the Internet to obtain relevant information, and send the information to submodule 4. Step 4: Generate synonyms of keywords; Step 5: Based on the question keywords and their synonyms, search and try to match the data keywords and their synonyms of the era database obtained in step 2; 1) If a "data keyword" is matched, the content of the grid where the data keyword of the era database is located is sent to submodule 4; 2) If no "data keyword" is matched, the network is searched for relevant information based on the keyword in the user's question, and the information is sent to submodule 4; Submodule 4: answer generation module, which answers the user's questions based on the information obtained from submodules 2 and 3.

2. A method for creating and communicating with a digital character according to claim 1, characterized in that: In step S1, the materials of each character are sorted into a digital character database. There are two different implementation methods according to different data sources: S11: Analysis of the non-chat record data of the character and establishment of character database: Step 1: Find information about person X from various sources other than chat records, including text descriptions, photos, videos, and audio information; Step 2: Set up multiple forms for characters, divide each form into subcategories, and create a table for automatically classifying character information; Step 3: Use text classification technology to automatically classify the character's text description into various subcategories in the character information automatic classification result table; Step 4: Use computer vision technology to analyze and classify photos and videos, and automatically classify the original files of photos and videos into various subcategories in the character information automatic classification result table, and process them in the following two situations: Case 1: If the classification error rate during the sub-classification process is greater than the threshold, the original files of the photos and videos are automatically classified into the various forms of the person in the person information automatic classification result table instead of the sub-classification; if the error rate of automatic classification into the form is still greater than the threshold, the original files of the photos or videos are not classified, and the original files are placed in the photo / video row of a person that cannot be classified in the person information automatic classification result table, or they are manually classified into sub-classifications; Case 2: If the classification error rate is less than or equal to the threshold during the classification process, the original photo or video file is automatically classified into each subcategory of the person in the person information automatic classification result table; Step 5: Analyze the audio file: 1) Use audio analysis technology to divide the audio into voice audio and non-voice audio, and retain the original audio file; 2) For the speech audio, the information in the audio is converted into text using speech recognition technology, and the content of the converted text is added to each subcategory of the character information automatic classification result table according to step 3; according to the classification result of the recognized text, the original audio file is automatically classified into the corresponding subcategory of the character information automatic classification result table; 3) For non-speech audio, it will not be automatically classified. The original audio file will be placed in the audio row of a certain person that cannot be classified in the character information automatic classification result table, or manually classified into a sub-category; Step 6: Repeat steps 1 to 5 to integrate each character into the character information automatic classification result table; S12: Analyze the chat records of the character and establish a character database: Step 1: Identify which party a person X is in the chat record, mark what X said as XS, and what Y who talked to X said as YS; Step 2: Analyze the text information in the chat content, specifically: 1) Classify the content XS said by X according to the category of digital characters; 2) In a round of dialogue, for the content YS said by Y, as long as X's reply is affirmative, the content YS said by Y is classified according to the category of digital characters; Step 3: Analyze the non-text information such as photos, videos and audio in the chat content, specifically: 1) According to the method in steps 4 and 5 of S11, the content XS said by X is summarized into the character information automatic classification result table; 2) In a round of dialogue, for a non-text file sent by Y, as long as X's reply is affirmative, the content YS said by Y is summarized into the character information automatic classification result table according to the method in steps 4 and 5 of S11; Step 4: Repeat steps 1 to 3 to integrate each character into the character information automatic classification result table.

3. A method for creating and communicating with a digital character according to claim 1, characterized in that: In step S2, establishing the era database specifically includes the following sub-steps: Step 1: Create a chronological table framework for each region of the world in chronological order, and construct an automatic classification result table of era information, including the year, major events, and major figures; Step 2: According to the method of establishing a digital character database, using the era as the starting clue, automatically search the Internet or various electronic dictionaries / encyclopedias to obtain information, and automatically classify it into various subcategories of the era information automatic classification result table; Step 3: Use characters as clues to link the character database and the era database.

4. A method for creating and communicating with a digital character according to claim 1, characterized in that: In step S1, character information is collected, sorted and analyzed, and a digital character database is established. The character information collection and sorting includes two methods. The first method is to manually sort the character's materials into the character database; the second method is to automatically sort the character's materials into the character database through an intelligent algorithm. The intelligent algorithm sorts the materials based on two different data sources: non-chat record materials and chat record materials.

5. A method for creating and communicating with a digital character according to claim 1, characterized in that: In step S1, the multiple forms of the character information automatic classification result table include a basic information form, a work information form, and a personal experience information form; the basic information form includes the following subcategories: the character's name, gender, birth time, education level, and marital status; the work information form includes the following subcategories: main work / contribution, main skills / talents; the personal experience information form includes several subcategories of the character's life events.

6. A method for creating and communicating with a digital character according to claim 1, characterized in that: In step S3, before the user starts a conversation with a digital character, the user selects a character that he wants to talk to from multiple digital characters, which can be achieved by clicking on a character's avatar or name, or scanning a character's QR code; after the user selects a character, the conversation module is turned on, and the user sends relevant questions or requests or inquiries through voice or text. If it is voice, it is automatically converted into text through voice recognition.

7. A method for creating and communicating with a digital character according to claim 1, characterized in that: In step S3, the user's question is answered in the first person of the character, including: for different types of questions, the corresponding answer words are added before returning the found information; if the returned information includes photos or videos, the photos or videos are played.

8. A system for creating and communicating with a digital character, characterized in that: The system is implemented based on the method according to any one of claims 1 to 7, and the system comprises: Digital character database establishment module: analyze the non-chat record data and chat record data of each character respectively, and construct the character information automatic classification result table respectively. Each character in the character information automatic classification result table corresponds to multiple forms, and each form is divided into several subcategories; after merging the character information automatic classification result table constructed by the non-chat record data and the chat record data, a digital character database is formed; Era database building module: Create a chronological table framework for each region of the world in chronological order, and construct an automatic classification result table of era information, including years, major events, and major figures; use figures as clues to establish a connection between the digital character database and the era database; Dialogue module: The user selects a digital character and distinguishes whether the user's question is a personal question or an impersonal question; for personal questions, the digital character database corresponding to the character is first activated. When the digital character database cannot answer, the era database is activated; for impersonal questions, the era database is activated; and the answer to the user is generated through database keyword matching.

9. The system according to claim 8, characterized in that All modules are implanted into a computer software, and the computer software is installed on a computer with a touch screen. The computer is placed in the scene required by the user, and the user selects and clicks the character portrait on the touch screen to interact; or, the digital character database, era database and dialogue module are all placed in the cloud, and the database and dialogue module are accessed through the mobile terminal mobile phone APP as the front end. The user selects and clicks the character to be talked to through the mobile terminal mobile phone APP, and the mobile terminal can be connected to the projector.

Images