Natural language processing and dialogue system based on artificial intelligence

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a natural language processing and dialogue system based on artificial intelligence, which comprises a database module, a word library arrangement module, a dialogue module, a language recognition module, a language processing module and a content output module, a user can firstly upload different language data into the database module, and the language data in the data module is sorted through the lexicon sorting module, so that the artificial intelligence system can have massive language data, convenience is brought to the use of workers, and when the user uses the dialogue module, the user can conveniently use the dialogue module. The language recognition module can firstly perform basic recognition on dialogue languages on the dialogue module, analyze the dialogue of the user through the language processing module, perform basic feedback through the content output module, understand semantics of the dialogue according to the dialogue subsequently input by the user, collect feedback of the user at the same time, and send the feedback to the user processing module. And updating the database module.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a natural language processing and dialogue system based on artificial intelligence. Background Art

[0002] Artificial intelligence is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. With the popularization of the Internet and the development of computer technology, artificial interaction technology has gradually emerged. Among them, intelligent dialogue systems have received widespread attention. Intelligent dialogue between people and computers has greatly facilitated people's work, life and study.

[0003] At present, most of the artificial intelligence dialogue systems in the prior art import the vocabulary into the system by the user, and when the user uses the dialogue system, the corresponding information is searched from the massive vocabulary based on keywords and fed back to the display terminal.

[0004] However, using the above method, the artificial intelligence dialogue system only analyzes an independent sentence of the user and cannot connect the relationship between the contexts, which can easily cause dialogue obstacles, and thus there is a problem of low language processing accuracy. Summary of the invention

[0005] The purpose of the present invention is to provide a natural language processing and dialogue system based on artificial intelligence, which can improve the language processing ability of artificial intelligence.

[0006] To achieve the above-mentioned purpose, the present invention provides a natural language processing and dialogue system based on artificial intelligence, including a database module, a vocabulary library arrangement module, a dialogue module, a language recognition module, a language processing module and a content output module;

[0007] The database module, the word library arrangement module, the dialogue module, the language recognition module, the language processing module and the content output module are connected in sequence;

[0008] The database module is used to store massive vocabulary data;

[0009] The word library sorting module is used to classify and sort the language data in the database;

[0010] The dialogue module is used for the user to input dialogue content into the system;

[0011] The language recognition module is used to perform basic recognition on the dialogue content used for input and perform preprocessing;

[0012] The language processing module is used to perform text analysis on the user's conversation content;

[0013] The content output module is used for artificial intelligence to process the user's conversation content and then feed it back to the user.

[0014] Wherein, the database module includes an import unit, a storage unit and a prompt unit, and the import unit, the storage unit and the prompt unit are connected in sequence;

[0015] The import unit is used for users to import different language vocabulary data into the artificial intelligence system;

[0016] The storage unit is used to encrypt and store the imported data;

[0017] The prompting unit prompts the user of the remaining storage space after the user has used the artificial intelligence for a long time.

[0018] Wherein, the word library arrangement module includes an arrangement unit, a classification unit and a proofreading unit, and the arrangement unit, the classification unit and the proofreading unit are connected in sequence;

[0019] The collating unit is used to sort the language data input into the system according to a predetermined rule;

[0020] The classification unit is used to classify the sorted language data according to usage requirements;

[0021] The proofreading unit is used to proofread the classified data to avoid errors.

[0022] Wherein, the dialogue module includes a dialogue input unit and a display unit, and the input unit and the display unit are connected in sequence;

[0023] The input unit is used for the user to input the dialogue into the system;

[0024] The display unit is used for the user to observe the dialogue interface with the artificial intelligence.

[0025] Wherein, the language recognition module includes a basic recognition unit and a preprocessing unit, and the basic recognition unit and the preprocessing unit are connected in sequence;

[0026] The basic recognition unit is used to perform text recognition on the dialogue language input by the user;

[0027] The pre-processing unit is used to format the input dialogue.

[0028] Wherein, the language processing module includes a feature extraction unit and a text analysis unit, and the feature extraction unit and the text analysis unit are connected in sequence;

[0029] The feature extraction unit is used to extract keywords from the user input dialogue;

[0030] The text analysis unit is used to perform lexical analysis on the user's conversation.

[0031] Wherein, the content output module includes a content combing unit and a content output unit, and the content combing unit and the content output unit are connected in sequence;

[0032] The content combing unit is used to organize the analysis results of the text analysis unit;

[0033] The content output unit is used to output the sorted data to the display unit.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. By setting the database module, the vocabulary library sorting module, the dialogue module, the language recognition module, the language processing module and the content output module, the user can first upload different language data to the database module, and sort the language data in the data module through the vocabulary library sorting module, so that the artificial intelligence system can have massive language data, which is convenient for the staff to use. When the user uses the dialogue module, the language recognition module can first perform basic recognition of the dialogue language on the dialogue module, and analyze the user's dialogue through the language processing module, and make basic feedback through the content output module, and understand the semantics of the dialogue input by the user subsequently, and at the same time, collect user feedback and complete the update of the database module.

[0036] 2. By setting up the local backup unit and the cloud backup unit, all data of the artificial intelligence system can be regularly backed up to its storage system server and cloud server to prevent data corruption and accidental loss from causing abnormal usage.

[0037] 3. By setting the notification unit and the recovery unit, it is possible to push notifications to users when data in the artificial intelligence system is accidentally lost, and the lost data can be manually restored by the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0039] Figure 1 It is a schematic diagram of the overall structure of the first embodiment of the present invention.

[0040] Figure 2 It is another schematic structural diagram of the first embodiment of the present invention.

[0041] Figure 3 It is a schematic diagram of the overall structure of the second embodiment of the present invention.

[0042] Figure 4 It is another schematic structural diagram of the second embodiment of the present invention.

[0043] Figure 5 It is a schematic diagram of the overall structure of the third embodiment of the present invention.

[0044] Figure 6 It is another schematic structural diagram of the third embodiment of the present invention.

[0045] 101-database module, 102-lexicon sorting module, 103-dialogue module, 104-language recognition module, 105-language processing module, 106-content output module, 107-import unit, 108-storage unit, 109-prompt unit, 110-sorting unit, 111-classification unit, 112-proofreading unit, 113-dialogue input unit, 114-display unit, 115-basic recognition unit, 116-preprocessing unit, 117-feature extraction unit, 118-text analysis unit, 119-content combing unit, 120-content output unit, 201-data backup module, 202-local backup unit, 203-cloud backup unit, 301-data recovery module, 302-notification unit, 303-recovery unit. DETAILED DESCRIPTION

[0046] The first embodiment of the present application is:

[0047] See also Figure 1-2 , Figure 1 is a schematic diagram of the overall structure of the first embodiment of the present invention, Figure 2 It is another schematic structural diagram of the first embodiment of the present invention.

[0048] The artificial intelligence-based natural language processing and dialogue system of the present invention comprises a database module 101, a vocabulary library arrangement module 102, a dialogue module 103, a language recognition module 104, a language processing module 105 and a content output module 106; the database module 101 comprises an import unit 107, a storage unit and a prompt unit 109, the vocabulary library arrangement module 102 comprises an arrangement unit 110, a classification unit 111 and a proofreading unit 112, the dialogue module 103 comprises a dialogue input unit 113 and a display unit 114, the language recognition module 104 comprises a basic recognition unit 115 and a preprocessing unit 116, the language processing module 105 comprises a feature extraction unit 117 and a text analysis unit 118, and the content output module 106 comprises a content combing unit 119 and a content output unit 120. The above-mentioned solution solves the problem that the existing artificial intelligence system only analyzes an independent sentence of the user, cannot connect the relationship between the context, easily causes dialogue obstacles, and thus has the problem of low language processing accuracy.

[0049] According to this specific implementation, the database module 101, the word library arrangement module 102, the dialogue module 103, the language recognition module 104, the language processing module 105 and the content output module 106 are connected in sequence;

[0050] The database module 101 is used to store massive vocabulary data.

[0051] The word library sorting module 102 is used to classify and sort the language data in the database;

[0052] The dialogue module 103 is used for the user to input the dialogue content into the system;

[0053] The language recognition module 104 is used to perform basic recognition on the dialogue content used for input and perform pre-processing;

[0054] The language processing module 105 is used to perform text analysis on the user's conversation content;

[0055] The content output module 106 is used for artificial intelligence to process the user's conversation content and then feed it back to the user;

[0056] The database module 101 is used to store different language data to increase the amount of information in artificial intelligence; the word library sorting module 102 will sort the language data in the database, including removing duplicate, erroneous or non-standard language entries to ensure the accuracy and completeness of the data. The sorted data will be classified, such as by subject, context or frequency of use, and through sorting and classification, the module can optimize the retrieval algorithm so that users can quickly find the required language information when querying, thereby improving the system's response speed and user experience; the dialogue module 103 can provide a user-friendly interactive interface so that users can easily communicate with the system, receive feedback from the system, and receive text, voice or other forms of information provided by users through various input methods; the language recognition module 104 receives user input from the dialogue module 103, which can be in text or voice form. For text input, it may include The processing module 105 may include operations such as removing irrelevant characters, correcting spelling errors, and word segmentation. For voice input, it may include steps such as denoising, echo cancellation, endpoint detection, and voice signal enhancement. The language processing module 105 first performs natural language understanding on the user's text input, which includes parsing the grammatical structure of the text, recognizing the meaning of words, determining the subject-predicate-object structure of the sentence, analyzing the user's input content to identify the user's intention, and considering the contextual information of the conversation so as to provide a coherent and relevant response. The content output module 106 formats the analysis results of the language processing module 105 and converts them into a form suitable for users to read or listen to. If the system encounters problems or uncertainties when processing the user's conversation content, the content output module 106 will be responsible for conveying this information in a user-friendly manner, which may include providing error messages, suggestions or solutions, and will use the user's historical data or preferences to customize personalized responses to enhance the user experience.

[0057] Wherein, the import unit 107, the storage unit 108 and the prompt unit 109 are connected in sequence;

[0058] The import unit 107 is used for the user to import different language vocabulary data into the artificial intelligence system;

[0059] The storage unit 108 is used to encrypt and store the imported data;

[0060] The prompting unit 109 prompts the user about the remaining storage space after the user has used the artificial intelligence for a long time;

[0061] The import unit 107 first ensures that the language vocabulary data provided by the user conforms to the data format required by the system. Before the data is imported, the import unit 107 verifies the data to ensure the accuracy and completeness of the data. Once the data passes the verification, the import unit 107 imports the data into the relevant storage structure of the system; the storage unit 108 uses an encryption algorithm to encrypt the imported data, and the encrypted data is stored in a secure file system; the prompt unit 109 regularly monitors the amount of space used in the storage unit 108 and compares it with the total storage space to determine the proportion or actual size of the remaining space. The user can set one or more thresholds. When the remaining amount of storage space reaches these thresholds, the prompt unit 109 will trigger a notification mechanism. Once the threshold is triggered, the prompt unit 109 will send a notification to the user through the user interface, email, SMS or other appropriate communication channels.

[0062] Secondly, the arranging unit 110, the classification unit 111 and the proofreading unit 112 are connected in sequence;

[0063] The arranging unit 110 is used to arrange the language data input into the system according to a predetermined rule;

[0064] The classification unit 111 is used to classify the sorted language data according to usage requirements;

[0065] The proofreading unit 112 is used to proofread the classified data to avoid errors;

[0066] The sorting unit 110 will standardize the language data to make it conform to a predetermined format or standard, merge or integrate similar or related data to reduce redundancy and improve data uniformity, and verify the sorted data to ensure that it conforms to predetermined rules and quality standards. According to the predefined classification standards, the classification unit 111 will automatically assign the language data to corresponding categories and label the data of each category accordingly to facilitate rapid retrieval and call when needed. As data accumulates and needs change, the classification unit 111 may need to dynamically adjust the classification standards to ensure that the classification of data always meets the latest usage needs, and provide data statistics functions to help users understand the data volume, usage frequency and change trends of each category; the proofreading unit 112 will use natural language processing technology, grammar checking tools and custom rule sets to automatically check errors in the classified data, and provide tools or interfaces to allow professional proofreaders to review and correct the results of automatic proofreading, and record the errors found and the corrections made. This information can be used to improve the proofreading algorithm and rules. At the same time, the system may provide a feedback mechanism to let users or data providers understand the problems found during the proofreading process.

[0067] Again, the input unit and the display unit 114 are connected in sequence;

[0068] The input unit is used for the user to input the dialogue into the system;

[0069] The display unit 114 is used for the user to observe the dialogue interface with the artificial intelligence;

[0070] The input unit provides a user interface that allows the user to input the conversation content into the artificial intelligence system through a keyboard, touch screen, voice or other input device, and formats the user's input to ensure that the data meets the system processing requirements; the display unit 114 provides a graphical user interface for displaying the conversation interface of the artificial intelligence system, supports multiple output modes, including text, images, audio and video, to adapt to different user needs and scenarios, and can provide real-time display of the conversation content between the user and the artificial intelligence, including the user's input and the system's response.

[0071] In addition, the basic recognition unit 115 and the pre-processing unit 116 are connected in sequence;

[0072] The basic recognition unit 115 is used to perform text recognition on the dialogue language input by the user;

[0073] The pre-processing unit 116 is used to format the input dialogue;

[0074] The basic recognition unit 115 receives the conversation language input by the user through the input device. For text input, the basic recognition unit 115 will directly use the input text data, and for voice input, the basic recognition unit 115 uses automatic speech recognition (ASR) technology to convert the voice signal into text and detect the language type of the user input so that the system can use the correct language model and resources for subsequent processing; the preprocessing unit 116 will preprocess the recognized text, such as removing redundant spaces, normalizing punctuation marks, converting uppercase and lowercase letters, etc., and errors may occur during the text recognition process. The preprocessing unit 116 will use an error correction algorithm to improve the accuracy of text recognition and format the processed text into a unified format within the system so that the subsequent language processing module 105 can process it correctly.

[0075] At the same time, the feature extraction unit 117 and the text analysis unit 118 are connected in sequence;

[0076] The feature extraction unit 117 is used to extract keywords from the user input dialogue;

[0077] The text analysis unit 118 is used to perform lexical analysis on the user's conversation;

[0078] The feature extraction unit 117 will perform an in-depth analysis of the text input by the user, including lexical analysis, grammatical analysis and semantic analysis, and will not only identify keywords but also consider the context of these keywords in the sentence to more accurately understand the user's intentions; the text analysis unit 118 will perform part-of-speech tagging on each word after segmentation to determine whether each word is a noun, verb, adjective, adverb, etc., analyze the frequency of occurrence of the word in the text, help identify important concepts and themes, and conduct a deeper semantic analysis, such as identifying the relationship between the upper and lower levels, synonymous relationships, etc. between words, in order to understand the deeper meaning of the words. In some cases, the text analysis unit 118 will also perform sentiment analysis to identify the sentiment tendency and emotional color in the text.

[0079] Then, the content combing unit 119 and the content output unit 120 are connected in sequence;

[0080] The content combing unit 119 is used to organize the analysis results of the text analysis unit 118;

[0081] The content output unit 120 is used to output the data sorted in the content sorting unit 119 to the display unit 114;

[0082] The content combing unit 119 receives the analysis results from the text analysis unit 118, and integrates the results of the text analysis to ensure that all relevant information is considered and organized together, structures the integrated information, and considers the context of the analysis results to ensure that the output content is contextually coherent and appropriate; the content output unit 120 further prepares and formats the data according to the characteristics and requirements of the display unit 114, integrates the prepared data into the user interface, and updates the output content in real time to reflect the latest data and processing results.

[0083] In the artificial intelligence-based natural language processing and dialogue system described in this embodiment, the user can first upload different language data to the database module 101, and organize the language data in the data module through the vocabulary organization module 102, so that the artificial intelligence system can have massive language data, which is convenient for the staff to use. When the user uses the dialogue module 103, the language recognition module 104 can first perform basic recognition of the dialogue language on the dialogue module 103, and analyze the user's dialogue through the language processing module 105, and make basic feedback through the content output module 106, and understand the semantics of the dialogue input by the user subsequently, and at the same time, collect user feedback to complete the update of the database module 101.

[0084] The second embodiment of the present application is:

[0085] Based on the first embodiment, please refer to Figure 3-4 , Figure 3 is a schematic diagram of the overall structure of the second embodiment of the present invention, Figure 4 It is another schematic structural diagram of the second embodiment of the present invention.

[0086] The artificial intelligence-based natural language processing and dialogue system of this embodiment further includes a data backup module 201 , and the data backup module 201 includes a local backup unit 202 and a cloud backup unit 203 .

[0087] Wherein, the data backup module 201 includes a local backup unit 202 and a cloud backup unit 203, and the local backup unit 202 and the cloud backup unit 203 are connected in sequence;

[0088] The local backup unit 202 is used to regularly back up the data in the artificial intelligence system to its storage system server;

[0089] The cloud backup unit 203 is used to regularly back up all data of the artificial intelligence system to a cloud server;

[0090] The local backup unit 202 will monitor data changes within the artificial intelligence system, including newly added, modified or deleted data. According to the backup strategy, the local backup unit 202 will select the data that needs to be backed up and copy the selected data to the local storage system server. After the backup is completed, the local backup unit 202 will verify the backed up data to ensure the integrity and accuracy of the data, and record the time of each backup, the amount of backed up data, the backup results and other information for subsequent audits and queries; the cloud backup unit 203 will monitor data changes within the artificial intelligence system, including newly added, modified or deleted data. According to the backup strategy, the cloud backup unit 203 will select the data that needs to be backed up and copy the selected data to the local storage system server.

[0091] The artificial intelligence-based natural language processing and dialogue system described in this embodiment can regularly back up all data of the artificial intelligence system to its storage system server and cloud server to prevent data corruption and accidental loss from causing abnormal usage.

[0092] The third embodiment of the present application is:

[0093] Based on the second embodiment, please refer to Figure 5-6 , Figure 5 is a schematic diagram of the overall structure of the third embodiment of the present invention, Figure 6 It is another schematic structural diagram of the third embodiment of the present invention.

[0094] The artificial intelligence-based natural language processing and dialogue system of this embodiment further includes a data recovery module 301 , and the data recovery module 301 includes a notification unit 302 and a recovery unit 303 .

[0095] Wherein, the notification unit 302 and the recovery unit 303 are connected in sequence;

[0096] The notification unit 302 is used to send a notification to the user when data is lost;

[0097] The recovery unit 303 is used for users to manually recover lost data.

[0098] The notification unit 302 monitors the data in the artificial intelligence system. When data is lost, the notification unit 302 generates notification content, sends the generated notification to the user in a predetermined manner, and records information such as the time of each notification, the object to which it is sent, and the content of the notification for subsequent inquiries; the recovery unit 303 provides a user interface, allowing the user to easily access the recovery function and select a method for recovering data. Through the interface, the user selects the type of data to be recovered, such as documents, pictures, and videos, and selects the source of the recovered data, including the local backup unit 202 and the cloud backup unit 203 recovery points. After performing the recovery operation, the recovery unit 303 starts the recovery process and recovers the data from the backup source to the specified location.

[0099] The artificial intelligence-based natural language processing and dialogue system described in this embodiment can push notifications to users when data in the artificial intelligence system is accidentally lost, and the lost data can be restored manually by the user.

[0100] What is disclosed above is only one or more preferred embodiments of the present application, and cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A natural language processing and dialogue system based on artificial intelligence, characterized in that: It includes database module, vocabulary library arrangement module, dialogue module, language recognition module, language processing module and content output module; The database module, the word library arrangement module, the dialogue module, the language recognition module, the language processing module and the content output module are connected in sequence; The database module is used to store massive vocabulary data; The word library sorting module is used to classify and sort the language data in the database; The dialogue module is used for the user to input dialogue content into the system; The language recognition module is used to perform basic recognition on the dialogue content used for input and perform preprocessing; The language processing module is used to perform text analysis on the user's conversation content; The content output module is used for artificial intelligence to process the user's conversation content and then feed it back to the user.

2. The natural language processing and dialogue system based on artificial intelligence as claimed in claim 1, characterized in that: The database module includes an import unit, a storage unit and a prompt unit, and the import unit, the storage unit and the prompt unit are connected in sequence; The import unit is used for users to import different language vocabulary data into the artificial intelligence system; The storage unit is used to encrypt and store the imported data; The prompting unit prompts the user of the remaining storage space after the user has used the artificial intelligence for a long time.

3. The natural language processing and dialogue system based on artificial intelligence as claimed in claim 2, characterized in that: The word library arrangement module includes an arrangement unit, a classification unit and a proofreading unit, and the arrangement unit, the classification unit and the proofreading unit are connected in sequence; The collating unit is used to sort the language data input into the system according to a predetermined rule; The classification unit is used to classify the sorted language data according to usage requirements; The proofreading unit is used to proofread the classified data to avoid errors.

4. The natural language processing and dialogue system based on artificial intelligence as claimed in claim 3, characterized in that: The dialogue module includes a dialogue input unit and a display unit, wherein the input unit and the display unit are connected in sequence; The input unit is used for the user to input the dialogue into the system; The display unit is used for the user to observe the dialogue interface with the artificial intelligence.

5. The natural language processing and dialogue system based on artificial intelligence as claimed in claim 4, characterized in that: The language recognition module includes a basic recognition unit and a preprocessing unit, and the basic recognition unit and the preprocessing unit are connected in sequence; The basic recognition unit is used to perform text recognition on the dialogue language input by the user; The pre-processing unit is used to format the input dialogue.

6. The natural language processing and dialogue system based on artificial intelligence as claimed in claim 5, characterized in that: The language processing module includes a feature extraction unit and a text analysis unit, and the feature extraction unit and the text analysis unit are connected in sequence; The feature extraction unit is used to extract keywords from the user input dialogue; The text analysis unit is used to perform lexical analysis on the user's conversation.

7. The natural language processing and dialogue system based on artificial intelligence as claimed in claim 6, characterized in that: The content output module includes a content combing unit and a content output unit, and the content combing unit and the content output unit are connected in sequence; The content combing unit is used to organize the analysis results of the text analysis unit; The content output unit is used to output the sorted data to the display unit.

8. The natural language processing and dialogue system based on artificial intelligence as claimed in claim 7, characterized in that: The artificial intelligence-based natural language processing and dialogue system further includes a data backup module, and the data backup module is connected to the database unit; The data backup module includes a local backup unit and a cloud backup unit, and the data backup module includes a local backup unit and a cloud backup unit, and the local backup unit and the cloud backup unit are connected in sequence; The local backup unit is used to regularly back up the data in the artificial intelligence system to its storage system server; The cloud backup unit is used to regularly back up all data of the artificial intelligence system to the cloud server.

Images