Natural language processing system based on reinforcement learning

Abstract

The invention discloses a natural language processing system based on reinforcement learning, and relates to the technical field of language intelligent processing, the natural language processing system comprises a language library building module, an identification and authentication module, a data acquisition module, a data analysis module and an instruction execution module, the language library building module builds a language library of language information by means of a language representation model, and a plurality of execution instructions are stored; the language library building module is set to be in a continuous updating state and has a continuous learning function, the identification and authentication module is used for pre-inputting specific characteristics and identity information held by a vehicle driver, the data acquisition module is used for carrying out data identification and acquisition, and the data analysis module is in signal connection with the data acquisition module and is used for analyzing the vehicle driver. The language information extraction module is in signal connection with the language library building module and is used for extracting language information corresponding to execution instructions in the language library building module, the instruction execution module is in signal connection with the language library building module and is used for executing corresponding instructions, and simplification and accuracy of language information expression are improved through arrangement of the system.

Description

Technical Field

[0001] The present invention relates to the technical field of language intelligent processing, and in particular to a natural language processing system based on reinforcement learning. Background Art

[0002] With the rapid development of voice recognition technology, more and more vehicles will be equipped with intelligent voice recognition systems. Usually, voice recognition technology is used to implement an intelligent voice service that obeys commands and answers questions. At the same time, it can also be linked with the vehicle's navigation, audio, air-conditioning and other systems to realize the function of voice control of the vehicle system.

[0003] Currently on the market, the voice recognition processing system installed in the vehicle terminal can first recognize the set startup voice, then start and respond, and then receive the next voice command, analyze it in order and send relevant instructions.

[0004] However, in actual passenger usage scenarios, it was found that the language data between the driver and the passengers could not be recognized efficiently and accurately, and the language processing was not personalized, resulting in a poor human-computer interaction experience at the vehicle control screen terminal. Summary of the invention

[0005] The purpose of the present invention is to provide a natural language processing system based on reinforcement learning to solve the problems raised in the above background technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solution: a natural language processing system based on reinforcement learning, comprising:

[0007] A language database building module, which uses a language representation model to build a language database of language information, stores a number of execution instructions, and is set to a continuous update state and has a continuous learning function;

[0008] Identification and authentication module, used to pre-enter the unique characteristics and identity information held by the vehicle driver;

[0009] The data collection module is used to identify and collect data, including language collection, scene collection, and emotion collection and recognition, and quantify the collected data;

[0010] A data analysis module, the data analysis module is connected to the data acquisition module by signal, and is used to extract language information corresponding to the execution instruction in the language library building module;

[0011] An instruction execution module, the instruction execution module is connected to the language library building module by signal, and is used to execute corresponding instructions. The instruction execution module also includes a reinforcement learning module, and the reinforcement learning module is used to store the collected uncollected words into the language library to be learned.

[0012] The present invention further illustrates that the language database building module divides the language information into regular words and keywords, and the regular words and keywords are used in combination to form a sentence structure that can cover the language information;

[0013] The keywords are divided into effective words and bad words. The bad words are impolite words that have been temporarily collected. The bad word library can be deleted, modified and supplemented according to the driver's personal habits, thereby continuously reinforcing learning. Bad words are used to express the situational attitude of the voice communicator.

[0014] The present invention further illustrates that the data acquisition module signal connection:

[0015] A group of pressure sensors are set at each seat in the car to identify the position and obtain the occupancy status of each seat;

[0016] A group of sound collectors are sequentially arranged on one side of a group of pressure sensors for collecting language from non-driving positions.

[0017] The present invention further illustrates that the data acquisition module includes:

[0018] A language collection module, the language collection module is connected to the sound collector signal and is used to transmit the language information emitted from the non-driver's seat to the data analysis module;

[0019] A scenario acquisition module, which is used to identify bad words and make a scenario determination during driving, specifically a peaceful scenario or an extreme scenario;

[0020] The emotion recognition module is used to perform emotion recognition based on emotion recognition words and intonation in the language information to obtain the emotion information of the passengers, so that the driver can obtain the emotion data of the passengers in a timely manner.

[0021] The present invention further illustrates that the data acquisition module, in combination with the data analysis module, obtains corresponding scenario data and emotion data through natural language information;

[0022] The data analysis module is also internally provided with a filtering and noise reduction module, which is used to first filter the acquired language information to identify whether it is a conversation with the driver.

[0023] The present invention further illustrates that the operation process of the natural language processing system is as follows:

[0024] In the first step, the driver performs identity authentication, distinguishes between the driving seat and the non-driving seat, and confirms the system is turned on. From then on, the system has data collection and human-computer interaction functions;

[0025] The second step is to obtain the seat information, obtain the seat position and number of passengers through the pressure transmission data of the pressure sensor, and start the corresponding sound collector to obtain the language information of the non-driving seat in time;

[0026] The third step is to determine whether to filter and reduce noise based on the number of passengers in order to identify the conversation partner. Then, based on the language collection, the scenario data and emotion data are acquired, and language tags are established and transmitted to the driver.

[0027] In the fourth step, the system analyzes the language tags, makes corresponding instructions, and transmits them to the control screen to facilitate the driver to obtain information and authorize relevant instructions.

[0028] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the present invention adopts a data acquisition module and a data analysis module to collect and identify language, scene and emotion, and quantifies the collected data. By setting language tags, the acquisition of language information data on the driver's side is simplified and accurately determined, thereby improving the sense of interaction between passengers and drivers, as well as the driver's human-computer interaction experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0030] Figure 1 It is a schematic diagram of the language processing flow of the present invention;

[0031] Figure 2 It is a schematic diagram of the module arrangement of the present invention. DETAILED DESCRIPTION

[0032] The following is a further non-limiting detailed description of the technical solution of the present invention in conjunction with the preferred embodiments and the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] See also Figure 1 The present invention provides a technical solution: a natural language processing system based on reinforcement learning, which is arranged in a moving vehicle, usually arranged in various places in the vehicle for identification, and is connected to a central processing unit signal, including:

[0034] The language database building module uses the language representation model to build a language database of language information. The database is pre-established and stores a number of execution instructions. It uses artificial intelligence technology to realize artificial intelligence dialogue, thereby achieving the effect of human-computer interaction. The language database building module is set to a continuous update state and has a continuous learning function;

[0035] Identification and authentication module, used for authentication between humans and the system, especially for distinguishing the identities of people in the driver's seat and those not in the driver's seat;

[0036] The data collection module is used for data recognition and collection, including language collection, scene collection and emotion collection and recognition, and quantifies the collected data for subsequent filtering and analysis to obtain more accurate recognition instructions and improve the human-computer interaction experience;

[0037] The data analysis module is connected to the data acquisition module by signal, and is used to extract the language information corresponding to the execution instruction in the language library building module;

[0038] The instruction execution module is connected to the language library building module by signal to execute corresponding instructions.

[0039] Specifically, the language database building module divides language information into regular words and keywords. Regular words and keywords are used together to form a sentence structure that can cover language information. For example, in "Can you pull over to the side of the road?", "can" is identified as a regular word and does not have important language information features, while "pull over," "park," and "ma" will be identified as keywords and have important language information features. "ma" will be identified as an emotion recognition word to express the emotional state of the voice communicator.

[0040] The keywords are divided into effective words and bad words. Bad words are impolite words that have been temporarily collected. The bad word database can be deleted, modified and supplemented according to the personal habits of drivers, so as to carry out continuous reinforcement learning. Bad words are used to express the situational attitude of the voice communicator and can correspond to the corresponding situation.

[0041] The language database building module is connected to the intelligent speech recognition system signal and has intelligent functions such as recognition, dictation, screening, and dialogue.

[0042] Furthermore, the identification and authentication module is used to pre-enter the unique features and identity information held by the vehicle driver. The unique features include but are not limited to face recognition, fingerprint recognition, gesture recognition, and voice recognition, which are used to facilitate information recognition for drivers who often use the vehicle. Drivers who occasionally use the vehicle can also perform information recognition according to their usage needs. If it is used by a private car, it can improve the convenience of the driver. If it is used by an online car-hailing service, it can facilitate the recording of the driver's driving information;

[0043] The identification and authentication module can be connected to the camera signal installed in the car, and can also be connected to the control panel signal.

[0044] Furthermore, the data acquisition module is signal-connected to a group of pressure sensors and a group of sound collectors;

[0045] A group of pressure sensors are set at each seat in the car for position identification and obtaining the occupancy status of each seat. On the one hand, it can obtain the number of passengers in the car, and on the other hand, it can identify the driver's seat and non-driver's seat. A group of sound collectors are sequentially set on one side of a group of pressure sensors for language collection in the non-driver's seat. The sound collector is connected to the language database module signal through the intelligent speech recognition system.

[0046] Furthermore, the data collection module includes a language collection module, a scenario acquisition module, and an emotion recognition module;

[0047] The language collection module is connected to the sound collector signal to transmit the language information emitted from the non-driver's seat to the data analysis module;

[0048] The scenario acquisition module, under the action of the data analysis module, is used to identify bad words and make driving scenario judgments, specifically, whether the scenario is peaceful or extreme;

[0049] The emotion recognition module is used to perform emotion recognition based on the emotion recognition words and intonation in the language information to obtain the emotion information of the passengers, so that the driver can obtain the emotion data of the passengers in a timely manner;

[0050] The data collection module combines with the data analysis module to obtain corresponding situational data and emotional data through natural language information.

[0051] Furthermore, a filtering and noise reduction module is also provided inside the data analysis module, and the filtering and noise reduction module is used to first filter the acquired language information to identify whether it is a conversation with the driver;

[0052] It should be noted that when a group of pressure sensors determines that there are at least two passengers, the filtering and noise reduction module will perform dialogue object recognition. After dialogue object recognition, the language information data acquisition on the driver's side can be determined more accurately.

[0053] The command execution module is used to recognize language information and integrate the context data and emotion data it assigns to the driver, and is mostly set to the control screen.

[0054] The instruction execution module also includes a reinforcement learning module, which is used to store the collected unincluded words into a to-be-learned language library. The to-be-learned language library intelligently makes learning instructions to the driver by obtaining the number of times. After the driver agrees, the words will be entered into the primitive language library, and the primitive language library will remain in a continuous and stable learning state.

[0055] In this embodiment, the operation process of the natural language processing system is as follows:

[0056] In the first step, the driver performs identity authentication, distinguishes between the driving seat and the non-driving seat, and confirms the system is turned on. From then on, the system has data collection and human-computer interaction functions;

[0057] The second step is to obtain the seat information, obtain the seat position and number of passengers through the pressure transmission data of the pressure sensor, and start the corresponding sound collector to obtain the language information of the non-driving seat in time;

[0058] The third step is to determine whether to filter and reduce noise based on the number of passengers in order to identify the conversation partner. Then, based on the language collection, the scenario data and emotion data are acquired, and language tags are established and transmitted to the driver.

[0059] In the fourth step, the system analyzes the language tags, makes corresponding instructions, and transmits them to the control screen to facilitate the driver to obtain information and authorize relevant instructions.

[0060] Among them, in the third step, the data collection process is as follows:

[0061] A1: Under the filtering and noise reduction effect, the language information of the conversation partner is filtered out and marked as Sp is defined as the language information at position P, is the keyword composition structure, m is the total number of keywords in a single language information, i is the keyword number sequence, and the specific language meaning will be obtained. a represents the scenario data, and b represents the emotion data. Through such language tags, the driver can obtain the personalized information contained in the language of the passengers in the car in a timely, fast and accurate manner;

[0062] A2: a represents the scenario data. The values ​​of a are 0 and 1, which represent a peaceful scenario and an extreme scenario respectively. The identification method is bad word recognition. The scenario data of the corresponding seat is determined according to the frequency of occurrence of bad words within the set time T. The frequency limit is set in advance. When the actual value of the frequency of occurrence of bad words obtained is not higher than the frequency limit, a takes the value of 0, indicating that the scenario of the seat is peaceful. On the contrary, a takes the value of 1, indicating that the scenario of the seat is extreme. By displaying it on the language mark, the driver can not only obtain the keywords in time, but also obtain the scenario data revealed by the passengers, judge the situation of civilized language in the car, and make good language records and storage;

[0063] A3: b represents emotion data. Emotion recognition is performed based on the emotion recognition words and intonation in the language information to obtain the emotion information of the passenger, which is used to indicate the degree to which the passenger needs to get a response from the driver. The intonation is the recognition of a high rising tone. The emotion data b value first obtains the emotion recognition words, and sets b = X*Y, where X is the high rising tone appearance representation value within the set time. If a high rising tone appears, it is recorded as 2, otherwise it is recorded as 1, and Y is the number of emotion recognition words that appear within the set time.

[0064] For example, b=2*1=2 means that one emotion recognition word appears within the set time, and it has a high rising tone, and the emotion data b will be marked as 2; b=1*2=2 means that two emotion recognition words appear within the set time, but there is no high rising tone, and the emotion data b will also be marked as 2;

[0065] By combining the number of emotion recognition words and intonation recognition, the degree to which the passengers need a response from the driver is obtained. The higher the emotion data b, the higher the passenger's need.

[0066] By setting language markers, the driver in the driving seat can not only clearly obtain the language information expressed by the passengers, but also further understand the passengers' status and needs, thereby improving the sense of interaction between passengers and drivers, as well as the driver's human-computer interaction experience.

[0067] In the fourth step, the system will make an analysis based on the language tag. When a is 0, it means that the situation of the passenger who made the language information is peaceful. Under the driver's pre-authorization conditions, the authorization device in the car will make relevant instructions based on the language information.

[0068] For example, if the language information of the passenger at seat number 2 is "Can the air conditioner cool down?" and it is in high rising tone, the language tag is {S2: air conditioner cools down}0 2 After receiving the keyword information, the temperature inside the car will be lowered. It should be noted that this operation can be performed with the driver's prior authorization, thereby improving the riding experience of non-driving seats;

[0069] When a is 1, it means that the situation of the passenger who made the language information is extreme. For the sake of driving safety in the car, the authorized device in the car will not have the authority to execute the command. The language mark is reflected on the control screen. The driver makes his own adjustments according to the specific situation in the car to ensure driving safety.

[0070] In addition, when there are at least two passengers and conflicting language information appears within the set time, the authorization device in the car will limit the number of execution instructions based on the set number of valid authorization implementations, and the language mark will always be reflected on the control screen.

[0071] Finally, when unrecorded words appear, the system will conduct reinforcement learning, and the unrecorded words will be stored in the to-be-learned language library. The to-be-learned language library will intelligently make learning instructions to the driver by obtaining the number of times. The driver will enter it into the original language library and, if authorized, will have the instruction execution authorization, thereby continuously learning natural language.

[0072] In the description of the present invention, it is necessary to understand that the terms "up", "down", "front", "back", "left", "right", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as a limitation on the present invention.

[0073] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents, and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A natural language processing system based on reinforcement learning, connected to a central processing unit signal, characterized in that: include: A language database building module, which uses a language representation model to build a language database of language information, stores a number of execution instructions, and is set to a continuous update state and has a continuous learning function; Identification and authentication module, used to pre-enter the unique characteristics and identity information held by the vehicle driver; The data collection module is used to identify and collect data, including language collection, scene collection, and emotion collection and recognition, and quantify the collected data; A data analysis module, the data analysis module is connected to the data acquisition module by signal, and is used to extract language information corresponding to the execution instruction in the language library building module; An instruction execution module, the instruction execution module is connected to the language library building module by signal, and is used to execute corresponding instructions. The instruction execution module also includes a reinforcement learning module, and the reinforcement learning module is used to store the collected uncollected words into the language library to be learned.

2. The natural language processing system based on reinforcement learning according to claim 1, characterized in that: The language database building module divides the language information into regular words and keywords, and the regular words and keywords are used in combination to form a sentence structure that can cover the language information; The keywords are divided into effective words and bad words. The bad words are impolite words that have been temporarily collected. The bad word library can be deleted, modified and supplemented according to the driver's personal habits, thereby continuously reinforcing learning. Bad words are used to express the situational attitude of the voice communicator.

3. The natural language processing system based on reinforcement learning according to claim 2, characterized in that: The data acquisition module signal connection: A group of pressure sensors are set at each seat in the car to identify the position and obtain the occupancy status of each seat; A group of sound collectors are sequentially arranged on one side of a group of pressure sensors for collecting language from non-driving positions.

4. The natural language processing system based on reinforcement learning according to claim 3, characterized in that: The data acquisition module comprises: A language collection module, the language collection module is connected to the sound collector signal and is used to transmit the language information emitted from the non-driver's seat to the data analysis module; A scenario acquisition module, which is used to identify bad words and make a scenario determination during driving, specifically a peaceful scenario or an extreme scenario; The emotion recognition module is used to perform emotion recognition based on emotion recognition words and intonation in the language information to obtain the emotion information of the passengers, so that the driver can obtain the emotion data of the passengers in a timely manner.

5. The natural language processing system based on reinforcement learning according to claim 4, characterized in that: The data acquisition module combines with the data analysis module to obtain corresponding situation data and emotion data through natural language information; The data analysis module is also internally provided with a filtering and noise reduction module, which is used to first filter the acquired language information to identify whether it is a conversation with the driver.

6. The natural language processing system based on reinforcement learning according to claim 5, characterized in that: The operation process of the natural language processing system is as follows: In the first step, the driver performs identity authentication, distinguishes between the driving seat and the non-driving seat, and confirms the system is turned on. From then on, the system has data collection and human-computer interaction functions; The second step is to obtain the seat information, obtain the seat position and number of passengers through the pressure transmission data of the pressure sensor, and start the corresponding sound collector to obtain the language information of the non-driving seat in time; The third step is to determine whether to filter and reduce noise based on the number of passengers in order to identify the conversation partner. Then, based on the language collection, the scenario data and emotion data are acquired, and language tags are established and transmitted to the driver. In the fourth step, the system analyzes the language tags, makes corresponding instructions, and transmits them to the control screen to facilitate the driver to obtain information and authorize relevant instructions.

7. The natural language processing system based on reinforcement learning according to claim 6, characterized in that: Under the filtering and noise reduction effect, the language information of the dialogue object is filtered out, and the language is marked as: Sp is defined as the language information at position P, is the keyword composition structure, m is the total number of keywords in a single language information, i is the sequence number of keywords, and the specific language meaning will be obtained. a represents the situational data, and b represents the emotional data. Such language markers can assist the driver to obtain the personalized information contained in the language of the passengers in the car in a timely, fast and accurate manner.

8. The natural language processing system based on reinforcement learning according to claim 7, characterized in that: The value of a is 0 or 1, which respectively represents a peaceful situation or an extreme situation. The identification method is bad word recognition. The situation data of the corresponding seat is determined according to the frequency of occurrence of bad words within a set time T. The frequency limit is set in advance. When the actual value of the frequency of occurrence of bad words is not higher than the frequency limit, a takes the value of 0, indicating that the situation of the seat is peaceful. Otherwise, a takes the value of 1, indicating that the situation of the seat is extreme.

9. The natural language processing system based on reinforcement learning according to claim 8, characterized in that: The b is represented as emotion data, and emotion recognition is performed based on the emotion recognition words and intonation in the language information to obtain the emotion information of the passenger, which is used to indicate the degree to which the passenger needs to get a response from the driver. The intonation is a high-rising tone recognition. The emotion data b value first obtains the emotion recognition words; Set b=X*Y, where X is the value indicating the occurrence of high rising tone within the set time. The occurrence of high rising tone is recorded as 2, otherwise it is recorded as 1, and Y is the number of emotion recognition words that appear within the set time.

Images