A user interaction method and device based on knowledge base

Abstract

The present invention provides a user interaction method and device based on a knowledge base, the method comprising: providing a first entrance; receiving first information input by a user; obtaining second information related to the current scene; processing the first information and the second information, and inputting them into a pre-created knowledge base; matching according to preset rules in the knowledge base; displaying matching result information in a predetermined manner; receiving user interaction operations on the displayed information, giving feedback results, and obtaining user input information and environmental information by pre-building a complete knowledge base, and performing accurate matching in the knowledge base to provide users with more accurate adaptation content. The present invention improves the accuracy of data information push and improves user experience.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and in particular relates to a user interaction method and device based on a knowledge base. Background Art

[0002] With the advancement of science and technology, people's channels for obtaining information are becoming more and more diversified, and the way of communication and learning has also changed dramatically. It has become common to obtain learning information through online networks. Students in remote mountainous areas with poor educational resources can also use certain platforms to obtain learning tutorials and materials. However, the current online learning systems generally have low intelligence, fixed information formats, limited stored data, and untimely updates.

[0003] When using traditional learning systems, the usage methods are relatively limited. For example, users can only watch online courses that are broadcast regularly, and relevant materials are relatively lacking, or the materials seen by users are all the same, and the data are not associated with each other, resulting in high search costs, inconvenience in use, poor learning effects, and difficulty in meeting the growing diversified learning needs. How to provide a fast and effective intelligent user interaction system to provide users with complete learning materials and conveniently and accurately obtain information is an urgent problem to be solved at this stage. Summary of the invention

[0004] The present invention aims to solve the above-mentioned problems and provide a user interaction method and device based on a knowledge base to achieve the acquisition of complete relevant data and information, and to quickly, efficiently and accurately match the data required by the user from the constructed knowledge database according to the information collected by the client, thereby achieving accurate data and information recommendations to improve the user experience.

[0005] To solve the above problems, the technical solution adopted by the present invention is as follows:

[0006] A user interaction method based on a knowledge base, comprising: providing a first entrance; receiving first information input by a user; obtaining second information related to a current scene; inputting the first information and the second information into a pre-created knowledge base after processing; matching in the knowledge base according to preset rules; displaying matching result information in a predetermined manner; receiving user interaction operations on the displayed information and providing feedback results.

[0007] Preferably, the first information includes at least one of text information and audio information, and the text information is input directly and / or through options. Processing the first information includes: performing semantic analysis on the text information, performing audio recognition processing on the audio information, and obtaining the numbers, keywords and their corresponding synonymous information contained therein.

[0008] Preferably, the second information includes at least one of location information, environmental information, weather information, time information, facial information, and fingerprint information; processing the second information includes: identifying the environmental information to determine whether the user is indoors or outdoors, identifying facial information, and obtaining the user's emotional attributes.

[0009] Furthermore, matching is performed based on the first information and the second information. When a matching result exists, multiple matching results are displayed to the user in different regions, and full information of the selected result is displayed based on the user's selection.

[0010] The method may include tracking the user's operation records on the displayed content, analyzing the operation records, obtaining multi-dimensional analysis results, and updating the corresponding content stored in the knowledge base.

[0011] Preferably, the creation of a knowledge base can be achieved through the following steps: the creation of a knowledge base specifically includes: obtaining an open interface for a data source, reading data, and organizing and classifying it; for video files, obtaining key frame images, extracting audio information, analyzing the video frames and audio, and obtaining main attribute labels; for pictures and documents, obtaining text information through character recognition technology, and for audio information, voice recognition is required to determine the corresponding text; for text information, semantic analysis is performed to obtain keywords, and when it is a foreign language, translation is performed based on features; after obtaining the text information, the ending punctuation is identified, and several sentences are obtained after paragraph division, the sentences are cut, auxiliary words and numerical information are ignored, the part-of-speech tags are performed on each word, the position of each word is marked, the offset information of each word relative to the beginning and end of the paragraph is recorded, the dependency relationship between the words is obtained, a syntax tree is constructed, the terminology vocabulary is matched, and the matched words are used as the paragraph. The top node, when there are multiple matching results, the word with the highest frequency will be used as the top node of this part, and the other words will be arranged downward according to the dependency relationship, the semantic reasoning relationship between paragraphs will be obtained, and the same or similar word information between paragraphs will be counted, and the tree structure of the paragraphs will be associated according to the semantic reasoning relationship and statistical information to form one or more mesh topological structures, and the reasoning model will be used to infer and measure the main attribute labels, node contents, or corresponding similar semantic words corresponding to the topological structures obtained from different types of files to obtain the reasoning relationship; for the part where there is no reasoning relationship, matching and association are performed through clustering technology to obtain the network topological structure of multiple fields; a mesh graph database is established, and the obtained results are stored in the graph database to form a knowledge base. Optionally, the node stores multiple attribute information for words, including at least part of speech, whether it is a term, the frequency of occurrence in this paragraph, the offset at the beginning of the paragraph, and the offset at the end of the paragraph.

[0012] The present invention also provides a user interaction device based on a knowledge base, comprising: an input module, a receiving module, a processing module and an acquisition module connected in sequence, wherein the processing module is also connected in sequence to a display module and an interaction module. The input module is used to provide a first entry for a user to input information; the receiving module is used to receive the first information input by the user; the acquisition module is used to obtain the second information related to the current scene; the processing module is used to process the first information and the second information, input them into a pre-created knowledge base, and match them in the knowledge base according to preset rules; the display module is used to display the matching result information in a predetermined manner; the interaction module is used to receive the user's interactive operation on the displayed information and give feedback results.

[0013] The present invention also provides a user interaction system based on a knowledge base, the system comprising at least: a client and a server connected to the client in communication; the client is used to provide a first entrance, receive first information input by a user, obtain second information related to the current scene, transmit the first information and the second information to the server, receive information from the server for display, and receive user interaction information; the server is used to process the first information and the second information, input them into a pre-created knowledge base, match them in the knowledge base according to preset rules, convert the matching results into a predetermined method, receive the client's interactive operations, obtain feedback results, and transmit the information to be displayed to the client.

[0014] The present invention also provides a computer-readable storage medium, which stores a computer program. When the program is processed, the user interaction method based on the knowledge base is implemented.

[0015] The present invention provides an entry for users to input information and obtains information related to the scene to obtain complete information about the user himself and the scene environment around the user. It obtains data from the data source in advance, and uses natural language processing and other means to obtain the relationship between the data to build a network structure, which is stored in the knowledge database to form a complete database. According to the information collected in the early stage, the corresponding content is searched in the database to achieve more accurate content push, thereby improving the accuracy of data information push and improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The figure is a flow chart of a method according to an embodiment of the present invention.

[0017] Figure 2 A schematic diagram of the contents stored in the knowledge base.

[0018] Figure 3 This is an example of implementing expression recognition based on user face images.

[0019] Figure 4 The figure is an architectural diagram of a device corresponding to the method. DETAILED DESCRIPTION

[0020] The flow chart and block diagram in the accompanying drawings of the present invention show the possible architecture, function and operation of the method and system according to various embodiments of the present disclosure. It should be noted that each box in the flow chart or block diagram can represent a module, a program segment, or a part of the code, and the module, the program segment, or a part of the code may include one or more executable instructions for realizing the logical functions specified in each embodiment. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the flow chart and / or block diagram, and the combination of the boxes in the flow chart and / or block diagram can be implemented using a dedicated hardware-based system that performs a specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0021] The terms "including", "comprising" and similar terms used herein should be understood as open terms, i.e., "including but not limited to", indicating that other contents may also be included. In the present disclosure, the term "based on" means "based at least in part on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment".

[0022] The present application provides a user interaction method based on a knowledge base, which constructs a knowledge database storing a large amount of data information in advance and is arranged on a server side or a local terminal. When a user uses the functional software of the terminal, the terminal obtains the user's relevant information and matches it in the database in the hope of obtaining the data content that best matches the user's needs, and pushes the data content to the user, thereby improving the efficiency and accuracy of information acquisition.

[0023] Exemplarily, the method relies on corresponding system devices to be implemented and can be used in the field of online learning to expand the convenience of information acquisition for a large number of user groups, so that a wide range of high-quality learning materials can be obtained without leaving home. Existing libraries, publishing houses and schools have rich learning materials, such as textbooks, tutorials, documents, video / audio materials, etc. The system extracts data content from various institutions through open interfaces, and data can also be actively uploaded to the system by all parties, such as recorded courseware, test question analysis, theoretical explanation content, etc.

[0024] Figure 1A flow chart of an embodiment of the method is shown. Before querying, various types of data information are first obtained, and the data can be classified by type through extensions, such as video, picture, document / text, audio, etc., and the author, year, subject, field and other related data tags of the relevant files are sorted and stored accordingly.

[0025] For video data, use the video image recognition algorithm to first remove the beginning and useless tail frames of the video. For the remaining part, use the trained deep learning model to identify the video frames, obtain the scene information of the video frames, and obtain the scene of the video, such as outdoor or indoor; perform cyclic matching on the previous and next video frames, and when the difference information of adjacent frames is less than the first threshold, put the frame into the constructed set, and use the mean of the features of the video frame as the set feature. When there is a feature mutation between the previous and next frames, and the difference between the feature of the next frame and the set feature exceeds the second threshold, mark the first position; identify the size of the sound source of the video, obtain the sound pop curve information, mark the singular value points of the curve, and obtain the corresponding second position information; extract audio information from the video, perform recognition processing on the extracted audio, obtain the corresponding text, and when the text is in a foreign language, use the intelligent translation system to translate it Translate into Chinese, map the text and video by time and store them to form matching subtitle information; for the extracted text content, use natural language processing technology to perform semantic analysis, segment the sentences, use the trained Bert sequence annotation model to annotate the text, extract named entity information, count the frequency of entity occurrence, and sort them in descending order, match the named entities with the previously marked positions, and when there are entities corresponding to the marked positions and the entity frequency is not less than the third threshold, use these entities as the main attribute labels of the video file, otherwise match the named entities with the term corpus, use the matched entities with higher frequency as the main attribute labels, and match and infer the labels of multiple videos to obtain the associated topological structure of multiple labels and multiple videos;

[0026] Text information can be processed directly. The audio data needs to be trimmed and then speech recognition is performed to obtain text information. For document data, character recognition technologies such as OCR are used to identify and extract the text information contained therein. Semantic analysis is performed on the text information to obtain keywords. When it is a foreign language, it is translated according to the features. The ending punctuation is identified and several sentences are obtained after paragraph division. The sentences are cut, and auxiliary words and numerical information are ignored. The part-of-speech tag is performed on each word, and the position of each word is marked. The offset information of each word relative to the beginning and end of the paragraph is recorded to obtain the dependency relationship between the words, build a syntax tree, match the terminology library, and use the matched words as the top node of the paragraph. When there are multiple matching results, the word with the highest frequency is taken as the top node of the part. When there is no match, the word with the highest frequency is taken as the top node, and other words are arranged downward according to the dependency relationship. The word attribute information is stored in the node in the form of a multi-tuple, which may include part of speech, whether it is a term, paragraph frequency, paragraph head offset, and paragraph tail offset. There is a progressive or parallel relationship between theoretical knowledge. Based on the complete learning corpus, the reasoning model is trained; the semantic reasoning relationship between paragraphs is obtained, and the same or similar word information between paragraphs is counted. According to the semantic reasoning relationship and statistical information, the tree structure of the paragraph is associated to form one or more mesh topological structures;

[0027] The inference model is used to infer the main attribute labels, node contents, or corresponding similar semantic words corresponding to the topological structures obtained from different types of files to obtain the inference relationship; for the parts where there is no inference relationship, clustering technology is used to match and associate them, thereby obtaining the network topology structure of multiple fields. A network graph database is established, and the obtained results are stored in the graph database to form a knowledge base, which can be a knowledge graph. Figure 2 An example of a constructed knowledge network is shown.

[0028] The method provides a user with an input entry to receive the first information input by the user, which can be in the form of one or more options, or directly input text, voice or video, or a combination of multiple methods, such as image search. The content input by the user is identified, and field information that can represent the user's needs is extracted. Processing the first information includes: semantic analysis of text information, audio recognition processing of audio information, and obtaining the numbers, keywords and their corresponding synonyms contained therein.

[0029] Optionally, before the user uses this method, a login authentication is required, such as an account password, face, fingerprint, etc. The terminal can also obtain secondary information such as the surrounding weather, location, environment, etc. The facial information can also be recognized to obtain the user's emotional attributes. Under different emotional states, the characteristics of the facial features are quite different. For example, in the case of a smiling face, the mouth opens and closes more, the corners of the mouth are raised, and the distance between the corners of the mouth and the eyes is closer. Figure 3 An example of smiling and not smiling is shown. The two corners of the eyes and two points on the upper and lower edges are selected as eye feature points, the corners of the mouth and three feature points on the upper and lower edges are selected as mouth feature points, and two feature points on both ends of the eyebrows are selected. The positions of each feature point under different expressions are marked.

[0030] Training the recognition model, when inputting the user's face image, using SIFT and other technologies to extract the face feature image, extract the corner points, locate the position of each corner point, obtain the offset information relative to the face, extract the corresponding eyebrow, eye, and mouth feature points, obtain the mouth opening and closing situation, eyebrow bending information, and eye dynamic information, input the recognition model, and realize expression recognition. Provide personalized matching content for users, such as identifying the user's age information, and giving priority to matching the user with age-appropriate learning or entertainment content. When it is recognized that the user is in an outdoor environment, it will give priority to matching content related to outdoor sports.

[0031] Match the first information and the second information. When there are matching results, multiple matching results are displayed to the user in different regions, and the full information of the selected results is displayed according to the user's choice. After processing the first information and the second information, input them into the pre-created knowledge base; extract information that can represent the user's real needs, and expand its synonymous / near-meaning information. Optionally, the user's age, expression, and environment can be used as references to match in the knowledge base. After obtaining the matching results, organize them, use the keywords as titles, and display them in different regions according to the file type. Display summary information in the region to provide users with jump entries, such as jumping to the previous / next item, the introduction of a term, etc. Receive user interaction operations on the displayed information and give feedback results. When the user selects a result, jump to the corresponding details interface.

[0032] Track the user's operation records for displayed content, analyze the operation records, obtain multi-dimensional analysis results, and update the corresponding content stored in the knowledge base. Count the user's learning time, habits and other information, analyze the learned content to determine the user's focus content, and use it as a reference for the next matching or intelligent recommendation. According to the user's actual usage logic, optimize the topological relationship of the knowledge network.

[0033] In one embodiment, the present invention provides a user interaction device based on a knowledge base. Figure 4The framework diagram of the device is shown, which specifically includes: an input module, a receiving module, a processing module and an acquisition module connected in sequence, wherein the processing module is also connected in sequence to a display module and an interaction module. The input module is used to provide a first entrance for users to input information; the receiving module is used to receive the first information input by the user; the acquisition module is used to obtain the second information related to the current scene; the processing module is used to process the first information and the second information, input them into a pre-created knowledge base, and match them in the knowledge base according to preset rules; the display module is used to display the matching result information in a predetermined manner; the interaction module is used to receive the user's interactive operation on the displayed information and give feedback results.

[0034] The present invention also provides a user interaction system based on a knowledge base, the system comprising at least: a client and a server connected to the client in communication; the client is used to provide a first entrance, receive first information input by a user, obtain second information related to the current scene, transmit the first information and the second information to the server, receive information from the server for display, and receive user interaction information; the server is used to process the first information and the second information, input them into a pre-created knowledge base, match them in the knowledge base according to preset rules, convert the matching results into a predetermined method, receive the client's interactive operations, obtain feedback results, and transmit the information to be displayed to the client.

[0035] The present invention also provides a computer-readable storage medium, which stores a computer program. When the program is processed, the user interaction method based on the knowledge base is implemented.

[0036] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as above in the preferred embodiment, it is not used to limit the present invention. Any technician familiar with the art can make many possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the scope of the technical solution of the present invention, or modify it into an equivalent embodiment of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of protection of the technical solution of the present invention.

Claims

1. A user interaction method based on a knowledge base, characterized in that: include: Provides first access; receiving first information input by a user; Acquire second information related to the current scene; After processing, the first information and the second information are input into a pre-created knowledge base; Matching is performed in the knowledge base according to preset rules; Display matching result information in a predetermined manner; Receive user interaction on displayed information and provide feedback results; The second information includes at least one of location information, environmental information, weather information, time information, face information, and fingerprint information; Processing the second information includes: identifying environmental information to determine whether the user is indoors or outdoors, identifying facial information, and obtaining emotional attributes of the user; Creating a knowledge base includes: Obtain the open interface of the data source, read the data, and organize and classify it; For video files, obtain key frame images, extract audio information, analyze video frames and audio, and obtain main attribute labels; For pictures and documents, character recognition technology is used to obtain text information, and for audio information, speech recognition is required to determine the corresponding text; Conduct semantic analysis on text information to obtain keywords. If the text is in a foreign language, translate it based on the features. After obtaining the text information, identify the ending punctuation, divide the paragraphs to obtain several sentences, cut the sentences, ignore the auxiliary words and numerical information, mark the part of speech for each word, mark the position of each word, record the offset information of each word relative to the beginning and end of the paragraph, obtain the dependency relationship between the words, build a grammar tree, match the terminology library, and use the matched words as the top node of the paragraph. When there are multiple matching results, the word with the highest frequency is used as the top node of the part, and the other words are arranged downward according to the dependency relationship. The semantic reasoning relationship between the paragraphs is obtained, and the same or similar word information between the paragraphs is counted. According to the semantic reasoning relationship and statistical information, the tree structure of the paragraphs is associated to form one or more mesh topological structures; The inference model is used to infer the main attribute labels, node contents, or corresponding similar semantic words corresponding to the topological structures obtained from different types of files to obtain the inference relationship; for the parts where there is no inference relationship, the clustering technology is used to match and associate them, so as to obtain the network topology structure of multiple fields; A network graph database is established and the obtained results are stored in the graph database to form a knowledge base.

2. The method according to claim 1, characterized in that: The first information includes at least one of text information and audio information, and the text information is directly input and / or input through options. Processing the first information includes: performing semantic analysis on the text information, performing audio recognition processing on the audio information, and obtaining the numbers, keywords and their corresponding synonyms contained therein.

3. The method according to claim 1, characterized in that: Matching is performed based on the first information and the second information. When a matching result exists, multiple matching results are displayed to the user in different regions, and full information of the selected result is displayed based on the user's selection.

4. The method according to claim 1, characterized in that: Track the user's operation records on the displayed content, analyze the operation records, obtain multi-dimensional analysis results, and update the corresponding content stored in the knowledge base.

5. The method according to claim 1, characterized in that: The node stores multiple attribute information about the word, including at least part of speech, whether it is a term, frequency of occurrence in this paragraph, paragraph start offset, and paragraph end offset.

6. A device using the knowledge base-based user interaction method according to claim 1, characterized in that: include: An input module, a receiving module, a processing module and an acquisition module are connected in sequence, wherein the processing module is also connected in sequence to a display module and an interaction module; wherein, An input module, used to provide a first entry for a user to input information; A receiving module, used for receiving first information input by a user; An acquisition module, used for acquiring second information related to the current scene; A processing module, used to process the first information and the second information, input them into a pre-created knowledge base, and match them in the knowledge base according to preset rules; A display module, used to display the matching result information in a predetermined manner; The interactive module is used to receive user interaction operations on displayed information and provide feedback results.

7. A user interaction system using the knowledge base-based user interaction method according to claim 1, characterized in that: The system at least comprises: a client and a server connected to the client for communication; The client is used to provide a first entry, receive first information input by a user, obtain second information related to the current scene, transmit the first information and the second information to the server, receive information from the server for display, and receive interactive information from the user; The server is used to process the first information and the second information, input them into a pre-created knowledge base, match them according to preset rules in the knowledge base, convert the matching results into a predetermined method, receive the client's interactive operation, obtain feedback results, and transmit the information to be displayed to the client.

8. A computer-readable storage medium using the knowledge base-based user interaction method according to any one of claims 1 to 5, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the knowledge base-based user interaction method according to any one of claims 1 to 5.

Images