Natural language analysis method and device

Abstract

The present application provides a natural language parsing method and device. The method includes: receiving current voice data input by a user, determining the candidate scene corresponding to the current displayed content and the weight of the candidate scene, sending a parsing request to a server, the parsing request includes the candidate scene corresponding to the current displayed content, the weight of the candidate scene and the text corresponding to the current voice data, receiving user intention information sent by the server, and performing operations corresponding to the user intention information. Thus, the accuracy of semantic parsing can be improved and the user experience can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a natural language parsing method and device. Background Art

[0002] Natural language is the crystallization of human wisdom. Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence, and is one of the most difficult problems in the field of artificial intelligence. By utilizing natural language processing, natural language communication between people and electronic devices can be achieved.

[0003] In the existing natural language analysis method, by receiving the current voice data input by the user, semantic analysis is performed according to the text corresponding to the current voice data or according to the text corresponding to the current voice data and the text corresponding to the previous voice data, and the analysis result is obtained, and the user intention is determined according to the analysis result. For example, the text corresponding to the current voice data input is "I want to watch Zhang San's movie", and the analysis result is "I want to watch Zhang San's movie".

[0004] However, the above method is not accurate in parsing text that indicates ambiguous intent or parsing text corresponding to the last voice data when there is no current voice data, or parsing when the scene of the text parsing corresponding to the last voice data is inconsistent with the scene corresponding to the currently displayed content, resulting in a problem that the semantic parsing result deviates from the user's intent. For example, the user browses movies through a remote control or touch screen → the user inputs "latest" by voice → outputs a non-latest movie intention. For another example, the user inputs "I want to watch a movie" by voice → the user switches to QQ Music through a remote control or touch screen → the user inputs "latest" by voice → "latest movie". In both scenarios, the semantic parsing results are inconsistent with the user's intent. Summary of the invention

[0005] The present application provides a natural language parsing method and device to solve the problem of low semantic parsing accuracy.

[0006] In a first aspect, the present application provides a display device, including:

[0007] A display for displaying images and a user interface;

[0008] Controller for:

[0009] Receive current voice data input by the user;

[0010] Determining candidate scenes corresponding to the currently displayed content and weights of the candidate scenes;

[0011] Sending a parsing request to a server, the parsing request including the candidate scenes corresponding to the currently displayed content, the weights of the candidate scenes, and the text corresponding to the current voice data, so that the server performs semantic parsing on the text corresponding to the current voice data according to the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes, obtains a semantic parsing result, and determines user intent information according to the semantic parsing result;

[0012] Receive the user intention information sent by the server, and perform an operation corresponding to the user intention information.

[0013] In some possible implementations, the controller is used to:

[0014] According to any one of the type of the currently playing program, the currently opened application and the currently displayed hybrid application interface, the candidate scene corresponding to the currently displayed content and the weight of the candidate scene are determined.

[0015] In some possible implementations, the controller is used to:

[0016] If the user interface displays a currently playing program, determining a scene corresponding to the type of the currently playing program as a candidate scene corresponding to the currently displayed content, and determining a maximum value of a weight in a preset weight range as a weight of the scene corresponding to the type of the currently playing program;

[0017] If the user interface displays a currently opened application, the scene corresponding to the currently opened application is determined as the candidate scene corresponding to the currently displayed content, and the maximum weight value in the preset weight range is determined as the weight of the scene corresponding to the currently opened application;

[0018] If the user interface displays a hybrid application interface, the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes are determined according to the interface elements of the hybrid application interface.

[0019] In some possible implementations, the controller is configured to:

[0020] When rendering the interface elements of the hybrid application interface, configuring a scene label and a weight for each interface element according to the position and size of the interface element and a preset weight configuration range;

[0021] Perform normalization calculation according to the weight configured for each interface element to obtain the normalized weight of each interface element;

[0022] According to the scene labels, the scene labels and normalized weights of all interface elements of the hybrid application interface are summed and normalized to obtain a normalized weight of each scene label;

[0023] The scene label whose normalized weight is greater than a preset threshold is determined as the candidate scene corresponding to the currently displayed content, and the normalized weight of the corresponding scene label is determined as the weight of the candidate scene.

[0024] In a second aspect, the present application provides a natural language parsing method, comprising:

[0025] Receive current voice data input by the user;

[0026] Determining candidate scenes corresponding to the currently displayed content and weights of the candidate scenes;

[0027] Sending a parsing request to a server, the parsing request including a candidate scene corresponding to the currently displayed content, a weight of the candidate scene, and a text corresponding to the current voice data;

[0028] Receive the user intention information sent by the server, and perform the operation corresponding to the user intention information.

[0029] In some possible implementations, determining the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes includes:

[0030] According to any one of the type of the currently playing program, the currently opened application and the currently displayed hybrid application interface, the candidate scene corresponding to the currently displayed content and the weight of the candidate scene are determined.

[0031] In some possible implementations, determining the candidate scene corresponding to the currently displayed content and the weight of the candidate scene according to any one of the type of the currently played program, the currently opened application, and the currently displayed hybrid application interface includes:

[0032] If the user interface displays a currently playing program, determining a scene corresponding to the type of the currently playing program as a candidate scene corresponding to the currently displayed content, and determining a maximum value of a weight in a preset weight range as a weight of the scene corresponding to the type of the currently playing program;

[0033] If the user interface displays a currently opened application, the scene corresponding to the currently opened application is determined as the candidate scene corresponding to the currently displayed content, and the maximum weight value in the preset weight range is determined as the weight of the scene corresponding to the currently opened application;

[0034] If the user interface displays a hybrid application interface, the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes are determined according to the interface elements of the hybrid application interface.

[0035] In some possible implementations, determining the candidate scene corresponding to the currently displayed content and the weight of the candidate scene according to the interface elements of the hybrid application interface includes:

[0036] When rendering the interface elements of the hybrid application interface, configuring a scene label and a weight for each interface element according to the position and size of the interface element and a preset weight configuration range;

[0037] Perform normalization calculation according to the weight configured for each interface element to obtain the normalized weight of each interface element;

[0038] According to the scene labels, the scene labels and normalized weights of all interface elements of the hybrid application interface are summed and normalized to obtain a normalized weight of each scene label;

[0039] The scene label whose normalized weight is greater than a preset threshold is determined as the candidate scene corresponding to the currently displayed content, and the normalized weight of the corresponding scene label is determined as the weight of the candidate scene.

[0040] In a third aspect, the present application provides a natural language parsing method, comprising:

[0041] Receiving a parsing request sent by a display device, the parsing request including a candidate scene corresponding to a currently displayed content, a weight of the candidate scene, and text corresponding to current voice data, the current voice data being input to the display device by a user;

[0042] Performing semantic analysis on the text corresponding to the current voice data according to the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes to obtain a semantic analysis result, and determining user intention information according to the semantic analysis result;

[0043] The user intention information is sent to the display device.

[0044] In some possible implementations, before performing semantic parsing on the text corresponding to the current voice data according to the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes to obtain the semantic parsing result, the method further includes:

[0045] Determining that there is no text corresponding to the previous voice data of the current voice data;

[0046] If it is determined that there is text corresponding to the previous voice data of the current voice data, semantic analysis is performed based on the scene of the text analysis corresponding to the previous voice data of the current voice data, the candidate scene corresponding to the currently displayed content, the weight of the candidate scene and the text corresponding to the current voice data to obtain a semantic analysis result.

[0047] In some possible implementations, the scene of the text parsing corresponding to the previous voice data of the current voice data and the candidate scene corresponding to the currently displayed content, the weight of the candidate scene and the text corresponding to the current voice data are semantically parsed to obtain a semantic parsing result, including:

[0048] If the similarity between the scene of text analysis corresponding to the previous voice data of the current voice data and the candidate scene corresponding to the currently displayed content is greater than or equal to the similarity threshold, semantic analysis is performed on the text corresponding to the current voice data according to the scene of text analysis corresponding to the previous voice data of the current voice data to obtain a semantic analysis result;

[0049] If the similarity between the scene of the text analysis corresponding to the previous voice data of the current voice data and the candidate scene corresponding to the currently displayed content is less than the similarity threshold, the text corresponding to the current voice data is semantically analyzed according to the candidate scene corresponding to the currently displayed content and the weight of the candidate scene to obtain a semantic analysis result.

[0050] In some possible implementations, performing semantic parsing on the text corresponding to the current voice data according to the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes to obtain a semantic parsing result includes:

[0051] Performing word segmentation annotation on the text corresponding to the current voice data to obtain word segmentation annotation information, wherein the word segmentation annotation information includes at least one word and an attribute annotation corresponding to each word;

[0052] Perform domain intent location according to the word segmentation annotation information to determine whether the first domain intent is located;

[0053] If the first domain intention is located, determining the first domain intention as the semantic parsing result;

[0054] If the first domain intent is not located, the domain intent is located according to the candidate scenes corresponding to the currently displayed content, the weights of the candidate scenes and the word segmentation annotation information, and the located second domain intent is determined as the semantic parsing result.

[0055] In some possible implementations, the performing domain intent location according to the candidate scenes corresponding to the currently displayed content, the weights of the candidate scenes, and the word segmentation annotation information includes:

[0056] For each candidate scene corresponding to the currently displayed content, determining a related word list for each candidate scene according to a preset mapping relationship between the scene and the related word list, wherein the related word list includes at least one related word and an attribute label corresponding to each related word;

[0057] Combining the related word list of each candidate scene with the word segmentation annotation information respectively to obtain combined word segmentation annotation information corresponding to each candidate scene, each candidate scene corresponding to at least one combined word segmentation annotation information;

[0058] Perform domain intent location according to the combined word segmentation annotation information corresponding to each of the candidate scenarios to obtain at least one domain intent and weight;

[0059] The second domain intention is determined based on the at least one domain intention and weight and the weight of the candidate scenario.

[0060] In some possible implementations, determining the second domain intent according to the at least one domain intent and the weight and the weight of the candidate scenario includes:

[0061] Calculating a target weight of each domain intent according to a weight of each domain intent in the at least one domain intent and a weight of each candidate scenario;

[0062] The domain intention with the largest target weight is determined as the second domain intention.

[0063] In some possible implementations, the performing domain intent location according to the word segmentation tag information to determine whether the first domain intent is located includes:

[0064] The word segmentation annotation information is matched with a preset rule template to determine whether it matches the first domain intent, and the preset rule template includes attribute annotation, regular expression and weight; or, the word segmentation annotation information is matched with a preset dependency syntactic relationship and weight to determine whether it matches the first domain intent, and the preset dependency syntactic relationship is configured according to a dependency syntax tree.

[0065] In a fourth aspect, the present application provides a natural language parsing device, comprising:

[0066] A receiving module, used for receiving current voice data input by a user;

[0067] A determination module, used to determine the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes;

[0068] A sending module, configured to send a parsing request to a server, wherein the parsing request includes a candidate scene corresponding to the currently displayed content, a weight of the candidate scene, and a text corresponding to the current voice data;

[0069] The receiving module is further used to: receive user intention information sent by the server, and display content corresponding to the user intention information on the user interface.

[0070] In some possible implementations, the determining module is used to:

[0071] According to any one of the type of the currently playing program, the currently opened application and the currently displayed hybrid application interface, the candidate scene corresponding to the currently displayed content and the weight of the candidate scene are determined.

[0072] In some possible implementations, the determining module is used to:

[0073] If the user interface displays a currently playing program, determining a scene corresponding to the type of the currently playing program as a candidate scene corresponding to the currently displayed content, and determining a maximum value of a weight in a preset weight range as a weight of the scene corresponding to the type of the currently playing program;

[0074] If the user interface displays a currently opened application, the scene corresponding to the currently opened application is determined as the candidate scene corresponding to the currently displayed content, and the maximum weight value in the preset weight range is determined as the weight of the scene corresponding to the currently opened application;

[0075] If the user interface displays a hybrid application interface, the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes are determined according to the interface elements of the hybrid application interface.

[0076] In some possible implementations, the determining module is used to:

[0077] When rendering the interface elements of the hybrid application interface, configuring a scene label and a weight for each interface element according to the position and size of the interface element and a preset weight configuration range;

[0078] Perform normalization calculation according to the weight configured for each interface element to obtain the normalized weight of each interface element;

[0079] According to the scene labels, the scene labels and normalized weights of all interface elements of the hybrid application interface are summed and normalized to obtain a normalized weight of each scene label;

[0080] The scene label whose normalized weight is greater than a preset threshold is determined as the candidate scene corresponding to the currently displayed content, and the normalized weight of the corresponding scene label is determined as the weight of the candidate scene.

[0081] In a fifth aspect, the present application provides a natural language parsing device, comprising:

[0082] A receiving module, configured to receive a parsing request sent by a display device, wherein the parsing request includes a candidate scene corresponding to a currently displayed content, a weight of the candidate scene, and text corresponding to current voice data, wherein the current voice data is input by a user to the display device;

[0083] A semantic analysis module, used to perform semantic analysis on the text corresponding to the current voice data according to the candidate scenes corresponding to the current displayed content and the weights of the candidate scenes, obtain a semantic analysis result, and determine user intention information according to the semantic analysis result;

[0084] A sending module is used to send the user intention information to the display device.

[0085] In some possible implementations, the semantic parsing module is further used to:

[0086] Before performing semantic parsing on the text corresponding to the current voice data according to the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes and obtaining a semantic parsing result, determining that there is no text corresponding to the previous voice data of the current voice data;

[0087] If it is determined that there is text corresponding to the previous voice data of the current voice data, semantic analysis is performed based on the scene of the text analysis corresponding to the previous voice data of the current voice data, the candidate scene corresponding to the currently displayed content, the weight of the candidate scene and the text corresponding to the current voice data to obtain a semantic analysis result.

[0088] In some possible implementations, the semantic parsing module includes:

[0089] A first parsing unit is used to perform semantic parsing on the text corresponding to the previous voice data of the current voice data according to the scene of the text parsing corresponding to the previous voice data of the current voice data, and obtain a semantic parsing result when the similarity between the scene of the text parsing corresponding to the previous voice data of the current voice data and the candidate scene corresponding to the currently displayed content is greater than or equal to a similarity threshold;

[0090] The second parsing unit is used to perform semantic parsing on the text corresponding to the current voice data according to the candidate scene corresponding to the current display content and the weight of the candidate scene when the similarity between the scene of text parsing corresponding to the previous voice data of the current voice data and the candidate scene corresponding to the current display content is less than the similarity threshold, so as to obtain a semantic parsing result.

[0091] In some possible implementations, the second parsing unit is used to:

[0092] Performing word segmentation annotation on the text corresponding to the current voice data to obtain word segmentation annotation information, wherein the word segmentation annotation information includes at least one word and an attribute annotation corresponding to each word;

[0093] Perform domain intent location according to the word segmentation annotation information to determine whether the first domain intent is located;

[0094] If the first domain intention is located, determining the first domain intention as the semantic parsing result;

[0095] If the first domain intent is not located, the domain intent is located according to the candidate scenes corresponding to the currently displayed content, the weights of the candidate scenes and the word segmentation annotation information, and the located second domain intent is determined as the semantic parsing result.

[0096] In some possible implementations, the second parsing unit is used to:

[0097] For each candidate scene corresponding to the currently displayed content, determining a related word list for each candidate scene according to a preset mapping relationship between the scene and the related word list, wherein the related word list includes at least one related word and an attribute label corresponding to each related word;

[0098] Combining the related word list of each candidate scene with the word segmentation annotation information respectively to obtain combined word segmentation annotation information corresponding to each candidate scene, each candidate scene corresponding to at least one combined word segmentation annotation information;

[0099] Perform domain intent location according to the combined word segmentation annotation information corresponding to each of the candidate scenarios to obtain at least one domain intent and weight;

[0100] The second domain intention is determined based on the at least one domain intention and weight and the weight of the candidate scenario.

[0101] In some possible implementations, the second parsing unit is used to:

[0102] Calculating a target weight of each domain intent according to a weight of each domain intent in the at least one domain intent and a weight of each candidate scenario;

[0103] The domain intention with the largest target weight is determined as the second domain intention.

[0104] In some possible implementations, the second parsing unit is used to:

[0105] The word segmentation annotation information is matched with a preset rule template to determine whether it matches the first domain intent, and the preset rule template includes attribute annotation, regular expression and weight; or, the word segmentation annotation information is matched with a preset dependency syntactic relationship and weight to determine whether it matches the first domain intent, and the preset dependency syntactic relationship is configured according to a dependency syntax tree.

[0106] In a sixth aspect, the present application provides a server, including:

[0107] Processor; and

[0108] A memory, configured to store executable instructions of the processor;

[0109] Wherein, the processor is configured to execute the natural language parsing method described in any one of the third aspect and each possible design of the third aspect by executing the executable instructions.

[0110] In the seventh aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the natural language parsing method described in any one of the third aspect and the possible designs of the third aspect or the fourth aspect and the possible designs of the fourth aspect.

[0111] The natural language parsing method and device provided in the present application, while the display device recognizes the text corresponding to the voice data input by the user, determines the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes, and then carries the text corresponding to the voice data input by the user and the determined candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes in a parsing request and sends it to a server, so that when the server parses the text, it performs semantic parsing on the text corresponding to the current voice data according to the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes to obtain a semantic parsing result, and determines the user intent information according to the semantic parsing result, and finally sends the user intent information to the display device, so that when the server parses the text, it can parse the text in combination with the candidate scenes corresponding to the currently displayed content, instead of only performing semantic parsing based on the text, or performing semantic parsing based on the text and the text corresponding to the previous voice data, thereby improving the accuracy of semantic parsing and improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0112] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0113] Figure 1 A schematic diagram of an operation scenario between a display device and a control device in an embodiment provided in the present application;

[0114] Figure 2 A hardware configuration block diagram of a display device 200 in an embodiment provided in this application;

[0115] Figure 3 A configuration block diagram of a control device 1001 in an embodiment provided in this application;

[0116] Figure 4 A schematic diagram of the software system of the display device provided for this application;

[0117] Figure 5 A schematic diagram of the application programs that can be provided by the display device provided in this application;

[0118] Figure 6 A schematic diagram of an application of a display device in a voice interaction scenario;

[0119] Figure 7 A flow chart showing the application of the display device in a voice interaction scenario;

[0120] Figure 8 A schematic diagram of an application of a display device in a voice interaction scenario;

[0121] Fig. 9 Another flowchart of applying a display device to a voice interaction scenario;

[0122] Fig.10 Issue a schematic diagram of the identification model to the supplier of the identification model;

[0123] Fig.11 A schematic diagram of a process for the server 400 to obtain a recognition model;

[0124] Fig.12 A schematic diagram of a process for a server to update a recognition model;

[0125] Fig.13 An interactive flow chart of an embodiment of a natural language parsing method provided in an embodiment of the present application;

[0126] Fig.14 A schematic diagram of a hybrid application interface provided in an embodiment of the present application;

[0127] Fig.15 A schematic diagram of configuring scene labels and weights for interface elements of a hybrid application interface provided in an embodiment of the present application;

[0128] Fig.16A flowchart of an embodiment of a natural language parsing method provided in an embodiment of the present application;

[0129] Fig.17 A flowchart of an embodiment of a natural language parsing method provided in an embodiment of the present application;

[0130] Fig.18 A flowchart of an embodiment of a natural language parsing method provided in an embodiment of the present application;

[0131] Fig.19 A schematic diagram of the structure of a natural language parsing device provided in an embodiment of the present application;

[0132] Fig. 20 A schematic diagram of the structure of a natural language parsing device provided in an embodiment of the present application;

[0133] Fig.21 A schematic diagram of the structure of a natural language parsing device provided in an embodiment of the present application;

[0134] Fig. 22 A schematic diagram of the hardware structure of the display device provided for this application;

[0135] Fig.23 A schematic diagram of the hardware structure of the server provided for this application. DETAILED DESCRIPTION

[0136] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0137] Based on the exemplary embodiments described in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the claims attached to this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of the disclosure can also constitute a complete implementation method separately.

[0138] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0139] The terms "first" and "second" and the like in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise indicated. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented in an order other than those given in the diagrams or descriptions of the embodiments of this application.

[0140] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.

[0141] The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0142] The term "remote control" as used in this application refers to a component of an electronic device (such as the display device disclosed in this application), which can usually wirelessly control the electronic device within a short range. It is generally connected to the electronic device using infrared and / or radio frequency (RF) signals and / or Bluetooth, and may also include functional modules such as WiFi, wireless USB, Bluetooth, motion sensors, etc. For example: a handheld touch remote control replaces most of the physical built-in hard keys in a general remote control device with a user interface in a touch screen.

[0143] The term "gesture" used in this application refers to a user's behavior of expressing an intended idea, action, purpose, or result through a change in hand shape or hand movement.

[0144] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device in an embodiment provided by the present application. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through a mobile terminal 1002 and a control apparatus 1001 .

[0145] In some embodiments, the control device 1001 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, etc., to control the display device 200 wirelessly or in a wired manner. The user may input user commands through buttons on the remote controller, voice input, control panel input, etc. to control the display device 200. For example, the user may input corresponding control commands through the volume plus and minus keys, channel control keys, up / down / left / right movement keys, voice input keys, menu keys, power on / off keys, etc. on the remote controller to realize the function of controlling the display device 200.

[0146] In some embodiments, a mobile terminal, a tablet computer, a computer, a laptop computer, and other smart devices may also be used to control the display device 200. For example, an application running on the smart device is used to control the display device 200. The application may be configured to provide the user with various controls in an intuitive user interface (UI) on a screen associated with the smart device.

[0147] In some embodiments, the mobile terminal 1002 can install software applications with the display device 200, and realize connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. For example, it is possible to establish a control instruction protocol with the mobile terminal 1002 and synchronize the remote control keyboard to the mobile terminal 1002, and realize the function of controlling the display device 200 by controlling the user interface on the mobile terminal 1002. It is also possible to transmit the audio and video content displayed on the mobile terminal 1002 to the display device 200 to realize the synchronous display function.

[0148] like Figure 1 It is also shown that the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactions to the display device 200. For example, the display device 200 receives software program updates or accesses a remotely stored digital media library by sending and receiving information and interacting with an electronic program guide (EPG). The server 400 may be a cluster or multiple clusters, and may include one or more types of servers. Other network service content such as video on demand and advertising services is provided through the server 400.

[0149] The display device 200 may be a liquid crystal display, an OLED display, or a projection display device. The specific display device type, size, and resolution are not limited. It is understood by those skilled in the art that the display device 200 may be modified in performance and configuration as required.

[0150] In addition to providing the broadcast receiving television function, the display device 200 may also provide an intelligent network television function with computer support functions, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0151] Figure 2 A hardware configuration block diagram of a display device 200 in an embodiment provided in the present application.

[0152] In some embodiments, the display device 200 includes at least one of a controller 250, a tuner and demodulator 210, a communicator 220, a detector 230, an input / output interface 255, a display 275, an audio output interface 285, a memory 260, a power supply 290, a user interface 265 and an external device interface 240.

[0153] In some embodiments, the display 275 is a component for receiving an image signal outputted from the processor and displaying video content and images as well as a menu control interface.

[0154] In some embodiments, the display 275 includes a display screen component for presenting images and a driving component for driving image display.

[0155] In some embodiments, the displayed video content may be from broadcast television content, or may be various broadcast signals received through wired or wireless communication protocols, or may be from various image content sent from a network server via a network communication protocol.

[0156] In some embodiments, the display 275 is used to present a user interface generated in the display device 200 and used to control the display device 200 .

[0157] In some embodiments, depending on the type of display 275, a driving component for driving the display is also included.

[0158] In some embodiments, the display 275 is a projection display, and may also include a projection device and a projection screen.

[0159] In some embodiments, the communicator 220 is a component for communicating with an external device or an external server according to various communication protocol types. For example, the communicator may include at least one of a Wifi chip, a Bluetooth communication protocol chip, a wired Ethernet communication protocol chip, other network communication protocol chips or a near field communication protocol chip, and an infrared receiver.

[0160] In some embodiments, the display device 200 may establish control signal and data signal transmission and reception with the external control device 1001 or the content providing device through the communicator 220 .

[0161] In some embodiments, the user interface 265 may be used to receive infrared control signals from the control device 1001 (eg, an infrared remote controller, etc.).

[0162] In some embodiments, the detector 230 is a signal used by the display device 200 to collect external environment or interact with the outside.

[0163] In some embodiments, the detector 230 includes a light receiver, a sensor for collecting ambient light intensity, and can adaptively display parameter changes by collecting ambient light.

[0164] In some embodiments, the image collector 232 in the detector 230, such as a camera, a camera, etc., can be used to collect external environment scenes, as well as to collect user attributes or gestures for interacting with the user. It can adaptively change display parameters and recognize user gestures to realize the function of interacting with the user.

[0165] In some embodiments, the detector 230 may also include a temperature sensor or the like, such as by sensing the ambient temperature.

[0166] In some embodiments, the display device 200 can adaptively adjust the display color temperature of the image. For example, when the temperature is high, the display device 200 can be adjusted to display a cooler color temperature of the image, or when the temperature is low, the display device 200 can be adjusted to display a warmer color temperature of the image.

[0167] In some embodiments, the detector 230 may also include a sound collector 231, such as a microphone, which can be used to collect voice data. When the user speaks a command by voice, the microphone can collect voice data including the command spoken by the user. Exemplarily, the sound collector 231 can collect voice signals including control commands of the user to control the display device 200, or collect environmental sounds for identifying the type of environmental scene, so that the display device 200 can adaptively adapt to environmental noise.

[0168] In some embodiments, Figure 2 As shown, the input / output interface 255 is configured to perform data transmission between the controller 250 and other external devices or other controllers 250, such as receiving video signal data and audio signal data, or command instruction data from external devices.

[0169] In some embodiments, the external device interface 240 may include, but is not limited to, any one or more interfaces such as a high-definition multimedia interface HDMI interface, an analog or digital high-definition component input interface, a composite video input interface, a USB input interface, an RGB port, etc. It is also possible that the above multiple interfaces form a composite input / output interface.

[0170] In some embodiments, Figure 2 As shown, the tuner-demodulator 210 is configured to receive broadcast television signals through wired or wireless reception, and can perform modulation and demodulation processing such as amplification, mixing and resonance, and demodulate audio and video signals from multiple wireless or wired broadcast television signals. The audio and video signals may include television audio and video signals carried in the television channel frequency selected by the user, as well as EPG data signals.

[0171] In some embodiments, the frequency demodulated by the tuner-demodulator 210 is controlled by the controller 250, which can send a control signal based on the user's selection to make the modem respond to the TV signal frequency selected by the user and modulate and demodulate the TV signal carried by the frequency.

[0172] In some embodiments, the broadcast television signal can be classified into terrestrial broadcast signals, cable broadcast signals, satellite broadcast signals, or Internet broadcast signals according to different broadcasting formats of the television signal. Or it can be classified into digital modulation signals, analog modulation signals, etc. according to different modulation types. Or it can be classified into digital signals, analog signals, etc. according to different signal types.

[0173] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box. In this way, the set-top box outputs the TV audio and video signals modulated and demodulated from the received broadcast TV signals to the main device, and the main device receives the audio and video signals through the first input / output interface.

[0174] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 can control the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 275, the controller 250 can perform operations related to the object selected by the user command.

[0175] In some embodiments, the object may be any one of the selectable objects, such as a hyperlink or an icon. Operations related to the selected object may include, for example, displaying a page, document, image, etc. connected to the hyperlink, or executing a program corresponding to the icon. User commands for selecting UI objects may be commands inputted through various input devices (e.g., a mouse, keyboard, touchpad, etc.) connected to the display device 200 or voice commands corresponding to voices spoken by the user.

[0176] like Figure 2As shown, the controller 250 includes at least one of a random access memory 251 (Random Access Memory, RAM), a read-only memory 252 (Read-Only Memory, ROM), a video processor 270, an audio processor 280, other processors 253 (for example: a graphics processor (Graphics Processing Unit, GPU), a central processing unit 254 (Central Processing Unit, CPU), a communication interface (Communication Interface), and a communication bus 256 (Bus). Among them, the communication bus connects various components.

[0177] In some embodiments, RAM 251 is used to store temporary data for the operating system or other running programs.

[0178] In some embodiments, ROM 252 is used to store various system startup instructions.

[0179] In some embodiments, ROM 252 is used to store a basic input / output system, called Basic Input Output System (BIOS), which is used to complete the power-on self-test of the system, the initialization of each functional module in the system, the basic input / output driver of the system, and the booting operating system.

[0180] In some embodiments, when a power-on signal is received, the display device 200 starts to power on, and the CPU runs the system startup instruction in the ROM 252, and copies the temporary data of the operating system stored in the memory to the RAM 251, so as to start or run the operating system. After the operating system is started, the CPU copies the temporary data of various application programs in the memory to the RAM 251, so as to start or run various application programs.

[0181] In some embodiments, the CPU processor 254 is used to execute operating system and application instructions stored in the memory, and to execute various applications, data and content according to various interactive instructions received from external input, so as to finally display and play various audio and video contents.

[0182] In some exemplary embodiments, the CPU processor 254 may include multiple processors. The multiple processors may include a main processor and one or more sub-processors. The main processor is used to perform some operations of the display device 200 in the pre-power-on mode and / or the operation of displaying the picture in the normal mode. The one or more sub-processors are used for an operation in the standby mode and the like.

[0183] In some embodiments, the graphics processor 253 is used to generate various graphic objects, such as icons, operation menus, and user input command display graphics, etc. It includes an operator that performs operations by receiving various interactive instructions input by the user and displays various objects according to display attributes. It also includes a renderer that renders various objects obtained based on the operator, and the rendered objects are used to be displayed on the display.

[0184] In some embodiments, the video processor 270 is configured to receive an external video signal and perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, image synthesis, etc. according to the standard codec protocol of the input signal, and obtain a signal that can be directly displayed or played on the display device 200.

[0185] In some embodiments, the video processor 270 includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like.

[0186] Among them, the demultiplexing module is used to demultiplex the input audio and video data stream. For example, if MPEG-2 is input, the demultiplexing module demultiplexes it into video signals and audio signals.

[0187] The video decoding module is used to process the demultiplexed video signal, including decoding and scaling.

[0188] The image synthesis module, such as an image synthesizer, is used to superimpose and mix the GUI signal generated by the graphics generator according to the user input or itself with the scaled video image to generate an image signal for display.

[0189] The frame rate conversion module is used to convert the input video frame rate, such as converting the 60Hz frame rate to 120Hz frame rate or 240Hz frame rate. The usual format is implemented by interpolation.

[0190] The display formatting module is used to receive the frame rate converted video output signal and change the signal to conform to the display format signal, such as outputting RGB data signal.

[0191] In some embodiments, the graphics processor 253 can be integrated with the video processor or can be separately configured. When integrated, it can perform processing of graphic signals output to the display. When separately configured, it can perform different functions respectively, such as a GPU+FRC (Frame Rate Conversion) architecture.

[0192] In some embodiments, the audio processor 280 is used to receive an external audio signal, and perform decompression and decoding, as well as noise reduction, digital-to-analog conversion, and amplification processing according to the standard codec protocol of the input signal to obtain a sound signal that can be played in a speaker.

[0193] In some embodiments, the video processor 270 may include one or more chips. The audio processor may also include one or more chips.

[0194] In some embodiments, the video processor 270 and the audio processor 280 may be separate chips, or may be integrated with the controller into one or more chips.

[0195] In some embodiments, the audio output receives the sound signal output by the audio processor 280 under the control of the controller 250, such as the speaker 286, and in addition to the speaker carried by the display device 200 itself, can be output to the external audio output terminal of the generating device of the external device, such as an external audio interface or a headphone interface, etc., and can also include a short-range communication module in the communication interface, for example, a Bluetooth module for Bluetooth speaker sound output.

[0196] The power supply 290, under the control of the controller 250, uses the power input from the external power supply to provide power supply support for the display device 200. The power supply 290 may include a built-in power supply circuit installed inside the display device 200, or may be an external power supply installed on the display device 200 to provide a power interface for an external power supply in the display device 200.

[0197] The user interface 265 is used to receive a user input signal, and then send the received user input signal to the controller 250. The user input signal may be a remote control signal received by an infrared receiver, and may receive various user control signals through a network communication module.

[0198] In some embodiments, the user inputs a user command through the control device 1001 or the mobile terminal 1002 , and the user input interface responds to the user input through the controller 250 according to the user input.

[0199] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 275, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through a sensor.

[0200] In some embodiments, the "user interface" is a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be an interface element such as an icon, window, control, etc. displayed on the display screen of an electronic device, where the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, and other visual interface elements.

[0201] The memory 260 includes storing various software modules for driving the display device 200. For example, the various software modules stored in the first memory include at least one of a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules.

[0202] The basic module is a bottom-level software module used for signal communication between various hardware in the display device 200 and sending processing and control signals to the upper-level modules. The detection module is a management module used to collect various information from various sensors or user input interfaces, and perform digital-to-analog conversion and analysis management.

[0203] For example, the speech recognition module includes a speech analysis module and a speech data database module. The display control module is a module used to control the display to display image content, and can be used to play multimedia image content and UI interface information. The communication module is a module used to perform control and data communication with external devices. The browser module is a module used to perform data communication between browsing servers. The service module is a module used to provide various services and various applications. At the same time, the memory 260 is also used to store and receive external data and user data, images of various items in various user interfaces, and visual effects of focus objects.

[0204] Figure 3 FIG. 1 is a block diagram of a configuration of a control device 1001 in an embodiment provided in the present application. Figure 3 As shown, the control device 1001 includes a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0205] The control device 1001 is configured to control the display device 200, and can receive the user's input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200. For example, the user operates the channel increase and decrease keys on the control device 1001, and the display device 200 responds to the channel increase and decrease operations.

[0206] In some embodiments, the control device 1001 may be a smart device, for example, the control device 1001 may be installed with various applications for controlling the display device 200 according to user needs.

[0207] In some embodiments, Figure 1 As shown, after installing the application for controlling the display device 200, the mobile terminal 1002 or other intelligent electronic device can play a similar function to the control device 1001. For example, the user can install the application to realize the function of the physical button of the control device 1001 by using various function keys or virtual buttons of the graphical user interface provided on the mobile terminal 1002 or other intelligent electronic device.

[0208] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller is used to control the operation and operation of the control device 1001, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0209] The communication interface 130, under the control of the controller 110, realizes the communication of control signals and data signals with the display device 200. For example, the received user input signal is sent to the display device 200. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0210] The user input / output interface 140 includes at least one of other input interfaces such as a microphone 141, a touch panel 142, a sensor 143, and a button 144. For example, a user can input a user command through voice, touch, gesture, pressing, and other actions. The input interface converts the received analog signal into a digital signal, and the digital signal into a corresponding command signal, and sends it to the display device 200.

[0211] The output interface includes an interface for sending received user commands to the display device 200. In some embodiments, it can be an infrared interface or a radio frequency interface. For example, in the case of an infrared signal interface, the user input command needs to be converted into an infrared control signal according to an infrared control protocol, and then sent to the display device 200 via an infrared sending module. For another example, in the case of a radio frequency signal interface, the user input command needs to be converted into a digital signal, and then modulated according to a radio frequency control signal modulation protocol, and then sent to the display device 200 via a radio frequency sending terminal.

[0212] In some embodiments, the control device 1001 includes at least one of a communication interface 130 and an input / output interface 140. The control device 1001 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, and can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0213] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 1001 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0214] The power supply 180 is used to provide operating power support for each component of the control device 1001 under the control of the controller, and can be a battery and related control circuits.

[0215] In some embodiments, the system may include a kernel, a command parser (shell), a file system, and an application. The kernel, shell, and file system together form the basic operating system structure, which allows users to manage files, run programs, and use the system. After power-on, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, etc., runs and maintains virtual memory, schedulers, signals, and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. After startup, the application is compiled into machine code to form a process.

[0216] Figure 4 For the software system diagram of the display device provided in this application, see Figure 4 In some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0217] In some embodiments, at least one application is running in the application layer, and these applications may be window programs, system settings programs, clock programs, camera applications, etc. provided by the operating system; or they may be applications developed by third-party developers, such as hi-view programs, karaoke programs, magic mirror programs, etc. In specific implementation, the application packages in the application layer are not limited to the above examples, and may actually include other application packages, which are not limited in the embodiments of the present application.

[0218] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Applications can access system resources and obtain system services during execution through the API interface.

[0219] like Figure 4 As shown, in the embodiment of the present application, the application framework layer includes managers, content providers, etc., wherein the manager includes at least one of the following modules: an activity manager (ActivityManager) is used to interact with all activities running in the system; a location manager (Location Manager) is used to provide system services or applications with access to system location services; a package manager (Package Manager) is used to retrieve various information related to the application package currently installed on the device; a notification manager (NotificationManager) is used to control the display and clearing of notification messages; a window manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0220] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit of the application (including switching the user interface currently displayed in the display window to the system desktop), opening, and backing (including switching the user interface currently displayed in the display window to the parent user interface of the currently displayed user interface), etc.

[0221] In some embodiments, the window manager is used to manage all window programs, such as obtaining the display size, determining whether there is a status bar, locking the screen, capturing the screen, controlling display window changes (for example, shrinking the display window, shaking the display, distorting the display, etc.), etc.

[0222] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0223] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, touch sensor, pressure sensor, etc.), etc.

[0224] In some embodiments, the core layer further includes a power driver module for performing power management.

[0225] In some embodiments, Figure 4 The software programs and / or modules corresponding to the software architecture in Figure 2 or Figure 3 in the first memory or the second memory shown.

[0226] In some embodiments, taking the magic mirror application (photographing application) as an example, when the remote control receiving device receives the input operation of the remote control, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the input operation into a raw input event (including the value of the input operation, the timestamp of the input operation, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer, identifies the control corresponding to the input event according to the current position of the focus, and the input operation is a confirmation operation, and the control corresponding to the confirmation operation is the control of the magic mirror application icon. The magic mirror application calls the interface of the application framework layer to start the magic mirror application, and then starts the camera driver by calling the kernel layer to realize the capture of static images or videos through the camera.

[0227] In some embodiments, for a display device with a touch function, taking a split-screen operation as an example, the display device receives an input operation (such as a split-screen operation) performed by a user on the display screen, and the kernel layer can generate a corresponding input event according to the input operation, and report the event to the application framework layer. The activity manager of the application framework layer sets the window mode (such as a multi-window mode) and the window position and size corresponding to the input operation. The window management of the application framework layer draws the window according to the settings of the activity manager, and then sends the drawn window data to the display driver of the kernel layer, and the display driver displays the corresponding application interface in different display areas of the display screen.

[0228] In some embodiments, Figure 5 A schematic diagram of the application programs that can be provided by the display device provided in this application, such as Figure 5 As shown in , the application layer includes at least one application that can display a corresponding icon control in the display, such as: a live TV application icon control, a video on demand application icon control, a media center application icon control, an application center icon control, a game application icon control, etc.

[0229] In some embodiments, the live TV application can provide live TV through different signal sources. For example, the live TV application can use input from cable TV, wireless broadcast, satellite service, or other types of live TV services to provide TV signals. And, the live TV application can display video of the live TV signal on the display device 200.

[0230] In some embodiments, the video on demand application can provide videos from different storage sources. Unlike the live TV application, the video on demand provides video display from certain storage sources. For example, the video on demand can come from the server side of the cloud storage, from the local hard disk storage containing the stored video programs.

[0231] In some embodiments, the media center application can provide an application for playing various multimedia contents. For example, the media center can be a service provided by a user to access various images or audios through the media center application, which is different from live TV or video on demand.

[0232] In some embodiments, the application center can provide and store various applications. The application can be a game, an application, or some other application related to a computer system or other device but can be run on a smart TV. The application center can obtain these applications from different sources, store them in a local storage, and then run them on the display device 200.

[0233] More specifically, in some embodiments, any of the display devices 200 described above in the present application may have a voice interaction function to improve the intelligence level of the display device 200 and improve the user experience of the display device 200.

[0234] In some embodiments, Figure 6 The present invention is a schematic diagram of an application of a display device in a voice interaction scenario, wherein user 1 can speak the instructions that he wants the display device 200 to execute through voice, and the display device 200 can collect voice data in real time, and recognize the instructions of user 1 included in the voice data, and directly execute the instructions after recognizing the instructions of user 1. During the whole process, user 1 does not actually operate the display device 200 or other devices, but simply speaks the instructions.

[0235] In some embodiments, when Figure 2 The display device 200 shown is used in Figure 6 In the scenario shown, the display device 200 can collect voice data in real time through its sound collector 231, and then the sound collector 231 sends the collected voice data to the controller 250, and finally the controller 250 recognizes the instructions included in the voice data.

[0236] In some embodiments, Figure 7 This is a flow chart of the application of the display device in the voice interaction scenario, which can be Figure 6 The display device in the scene shown executes, specifically, in S11, the sound collector 231 in the display device 200 collects voice data in the surrounding environment of the display device 200 in real time, and sends the collected voice data to the controller 250 for recognition.

[0237] In some embodiments, Figure 7 In S12 shown, after receiving the voice data, the controller 250 recognizes the instructions included in the voice data. For example, if the voice data includes the instruction "increase brightness" issued by user 1, the controller 250 can execute the recognized instruction after recognizing the instruction included in the voice data to control the display 275 to increase the brightness. It can be understood that in this case, the controller 250 recognizes each received voice data, and there may be a situation where there is no instruction in the recognized voice data.

[0238] In other embodiments, the model based on instruction recognition is large and the operation efficiency is low. It can also be stipulated that user 1 adds keywords before saying the instruction, for example, "ABCD", then the user needs to say the instruction "ABCD, increase the brightness", so that in Figure 7 In S12 shown, after receiving the voice data, the controller 250 first identifies whether there is a keyword "ABCD" in each voice data. After identifying the keyword, it uses the instruction recognition model to identify the specific instruction corresponding to "increase brightness" in the voice data.

[0239] In some embodiments, after receiving the voice data, the controller 250 may further denoise the voice data, including removing echoes and environmental noise, process the voice data into clean voice data, and recognize the processed voice data.

[0240] In some embodiments, Figure 7 Another application diagram of a display device in a voice interaction scenario is shown in FIG. 2 , in which the display device 200 can be connected to the server 400 via the Internet. After the display device 200 collects voice data, the voice data can be sent to the server 400 via the Internet. The server 400 recognizes the instructions included in the voice data and sends the recognized instructions back to the display device 200, so that the display device 200 can directly execute the received instructions. This scenario is similar to the following scenario: Figure 6 Compared with the scenario shown, the requirement on the computing power of the display device 200 is reduced, and a larger recognition model can be set on the server 400 to further improve the accuracy of command recognition in voice data.

[0241] In some embodiments, when Figure 2 The display device 200 shown is used in Figure 6 In the scenario shown, the display device 200 can collect voice data in real time through its sound collector 231. Subsequently, the sound collector 231 sends the collected voice data to the controller 250. The controller 250 sends the voice data to the server 400 through the communicator 220. After the server 400 recognizes the instructions included in the voice data, the display device 200 receives the instructions sent by the server 400 through the communicator 220, and finally the controller 250 executes the received instructions.

[0242] Figure 8 A schematic diagram of an application of a display device in a voice interaction scenario. In some embodiments, Fig. 9 Another flowchart of the application of the display device in the voice interaction scenario can be as follows Figure 8 The device in the scenario shown is executed, wherein in S21, the sound collector 231 in the display device 200 collects voice data in the surrounding environment of the display device 200 in real time, and sends the collected voice data to the controller 250, and the controller 250 further sends the voice data to the server 400 through the communicator 220 in S22, and the server recognizes the instructions included in the voice data in S23. Subsequently, the server 400 sends the recognized instructions back to the display device 200 in S24, and correspondingly, the display device 200 receives the instructions through the communicator 220 and sends them to the controller 250, and finally the controller 250 can directly execute the received instructions.

[0243] In some embodiments, Figure 7In S23, after receiving the voice data, the server 400 recognizes the instruction included in the voice data. For example, the voice data includes the instruction "increase brightness" issued by user 1. However, since the model of instruction recognition is relatively large and the server 400 recognizes each received voice data, it may happen that there is no instruction in the recognized voice data. Therefore, in order to reduce invalid recognition by the server 400 and reduce the amount of communication interaction data between the display device 200 and the server 400, in the specific implementation, it can also be stipulated that user 1 adds keywords before speaking the instructions, for example, "ABCD", then the user needs to say the instruction "ABCD, increase brightness". Subsequently, the controller 250 of the display device 200 first uses a keyword recognition model with a smaller model and lower computational complexity in S22 to identify whether the keyword "ABCD" exists in the voice data. If the keyword is not recognized in the voice data currently being processed by the controller 250, the controller 250 will not send the voice data to the server 400; if the keyword is recognized in the voice data currently being processed by the controller 250, the controller 250 will send all the voice data, or the part after the keyword in the voice data to the server 400, and the server 400 will recognize the received voice data. Since the voice data received by the controller 250 includes keywords at this time, the voice data sent to the server 400 for recognition is more likely to include user instructions, thereby reducing invalid recognition calculations of the server 400 and reducing invalid communications between the display device 200 and the server 400.

[0244] In some embodiments, in order to allow the display device 200 to have Figure 6 In a specific scenario shown in FIG. 1 , the recognition function of the command in the voice data, or the display device 200 can have the following functions: Figure 6 or Figure 8 In a specific scenario shown, for the recognition function of keywords in voice data, as a supplier of the voice interaction function of the display device 200, it is also necessary to produce a machine learning model that can be used to recognize instructions or keywords, such as textcnn, transform and other deep learning models. These models are stored in the display device 200 and used by the display device 200 when performing recognition.

[0245] In some embodiments, Fig.10 A schematic diagram of a recognition model supplier sending a recognition model, wherein after the server 400 set by the supplier obtains the recognition model (which may be a command recognition model or a keyword recognition model), the recognition model may be sent to each display device 200. Fig.10The process shown may be performed when the display device 200 is produced, and the server 400 sends the recognition model to each display device 200; or, after the display device 200 begins to be used, the server 400 sends the recognition model to the display device 200 via the Internet.

[0246] In some embodiments, the server 400 may obtain a recognition model by collecting voice data and learning based on a machine learning model. Fig.11 A flow chart of obtaining a recognition model for the server 400, wherein, in S31, each display device (taking display device 1-display device N, a total of N as an example) collects voice data 1-N, and sends the collected voice data 1-N to the server 400 in S32. Subsequently, in S33, the supplier's staff can manually annotate each voice data and the instructions or keywords included in the voice data, and then send the voice data itself and the annotation information corresponding to the voice data as data into the machine learning model, which is learned by the server. When the learned recognition model is used in the subsequent use, when a voice data to be recognized is input, the recognition model compares the voice data with the learned voice data, and outputs the probability of each annotation information. Finally, the annotation information corresponding to the maximum probability can be used as the recognition result of the voice data to be recognized. In S34, the server 400 can send the calculated recognition model to each display device.

[0247] In some embodiments, the server 400 may not use Fig.11 In the illustrated embodiment, the recognition model is calculated by the voice data actually collected by the display devices 1-N. Instead, the staff can directly input different voice data and the annotation information of each voice data, and send the calculated recognition model to each display device.

[0248] In some embodiments, Fig.11 The display devices 1-N shown collecting voice data and sending them to the server and the server calculating the recognition model and sending it to the display devices 1-N can be two independent processes, that is, the server receives the voice data collected by N display devices in S32, and the server can send the trained recognition model to another N display devices in S34. In these two processes, the N display devices can be the same or different, or they can also be partially the same.

[0249] In some embodiments, since the number of samples used in obtaining the recognition model is limited, the recognition model set by the display device 200 cannot achieve 100% accurate recognition. Therefore, the supplier can also collect the voice data collected by each display device 200 during actual use at any time through the server 400, and update the recognition model that has been recognized based on the collected voice data, so as to further improve the recognition accuracy of the recognition model.

[0250] For example, Fig.12 A flow chart of updating the recognition model by the server. It can be understood that when executing Fig.12 Prior to the embodiment shown, each display device is configured as follows: Fig.10 The recognition model is set in the manner shown in FIG. Fig.12 In S31, each display device (taking display device 1-display device N, N in total as an example) collects voice data 1-N, and sends the collected voice data 1-N to the server 400 in S32. Subsequently, in S33, the supplier's staff can manually annotate each voice data and the instructions or keywords included in the voice data, and then send the voice data itself and the annotation information corresponding to the voice data as data into the machine learning model, and the server updates the already calculated recognition model according to the received new voice data, and in S34, the server 400 can resend the updated recognition model to each display device 200, so that each display device 200 can be updated using the updated recognition model. Among them, for any one of the N display devices, since the new learning model uses the voice data collected by this display device 200, it can effectively improve the accuracy of the subsequent recognition of the collected voice data by this display device 200.

[0251] In some embodiments, Fig.12 Each display device shown can send voice data to the server after receiving it, or send the voice data collected within a fixed time period to the server after the end of the time period, or send a certain amount of voice data to the server after it is collected, or send the received voice to the server according to the instructions of the user of the display device or the instructions of the server staff.

[0252] In some embodiments, Fig.12The N display devices shown can send voice data to the server at the same agreed time, and the server can update the recognition model according to the N voice data received; alternatively, the N display devices can also send voice data to the server separately, and the server can start updating the recognition model according to the received voice data after the number of voice data received is greater than N.

[0253] Before formally introducing the embodiments of the present application, the problems existing in the prior art are first introduced in conjunction with the accompanying drawings and the application scenarios of the present application.

[0254] The natural language parsing method provided in this application can be applied to Figure 1 In the scenario shown. Figure 1 As shown, the display device 200 communicates with the server 400 through the network, and the user can operate the display device 200 through the mobile terminal 1002 or the control device 1001. In the embodiment of the present application, the user can input voice data (instructions) to the display device 200 through the mobile terminal 1002 or the control device 1001. After receiving the voice data input by the user, the display device 100 recognizes the voice data, and after recognizing it as text, sends a parsing request including the text to the server 400, and the server 400 performs semantic parsing on the text to obtain a semantic parsing result, and determines the user intention information according to the semantic parsing result, and finally sends the user intention information to the display device 200, and the display device 200 performs the operation corresponding to the user intention information. For example, the text corresponding to the voice data is "I want to listen to Zhang San's songs", and the user intention information determined by the server 400 is the relevant information of Zhang San's songs, such as links and related pictures of Zhang San's songs, and the display device displays or plays according to the relevant information of Zhang San's songs.

[0255] In the existing natural language parsing methods, when parsing the text corresponding to the voice data, semantic parsing is only performed based on the text corresponding to the current voice data, or semantic parsing is performed based on the text corresponding to the current voice data and the text corresponding to the previous voice data. This method can only parse text with clear intentions, and the parsing accuracy for text with ambiguous intentions is not high; and, for text that does not correspond to the previous voice data, it can only be parsed based on the text corresponding to the current voice data, and the parsing accuracy for text with ambiguous intentions is also not high; and, when the scene of the text parsing corresponding to the previous voice data is inconsistent with the scene corresponding to the currently displayed content, the accuracy of the semantic parsing based on the text corresponding to the current voice data and the text corresponding to the previous voice data is also not high. This causes the semantic parsing result to deviate from the user's intention, and the user experience is not good.

[0256] To solve this problem, the present application provides a natural language parsing method and device. In the present application, it is considered that the text with ambiguous intention may be the text corresponding to the voice data input by the user in a certain scenario. Omitting the scenario when the user inputs the voice data may cause the intention to be ambiguous. Therefore, in the present application, the display device determines the candidate scene corresponding to the current display content (that is, infers the scene where the user may currently be) and the weight of the candidate scene while identifying the text corresponding to the voice data input by the user. Then, the text corresponding to the voice data input by the user and the determined candidate scene corresponding to the current display content and the weight of the candidate scene are carried in the parsing request and sent to the server. When parsing the text, the server can parse the text in combination with the scene where the user may currently be, instead of only performing semantic parsing based on the text, or performing semantic parsing based on the text and the text corresponding to the last voice data. For example: the user is currently watching and browsing a movie, and the user inputs the voice data "the latest", the server can parse the user's intention as "the latest movie", thereby improving the accuracy of semantic parsing and improving user experience.

[0257] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0258] Fig.13 The interactive flow chart of the natural language parsing method embodiment provided in the present application is as follows: Fig.13 As shown, the method of this embodiment may include:

[0259] S101. A display device receives current voice data input by a user.

[0260] Specifically, for example, users can Figure 1 The mobile terminal or control device shown in the figure inputs voice data (command) to the display device. After receiving the voice data input by the user, the display device recognizes the voice data and recognizes it as text.

[0261] S102: The display device determines candidate scenes corresponding to currently displayed content and weights of the candidate scenes.

[0262] In the embodiment of the present application, the scene is a scene corresponding to the business divided by the content displayed by the display device. The scene can be pre-set according to the business of different existing display devices, and can also be updated according to the newly added business. For example, the scene in the embodiment of the present application may include: movies, TV series, variety shows, display device control (such as TV control), music, applications, encyclopedia, education, games, shopping, weather, recipes, English and sports, etc. The above scenes are only examples, and are not limited to the above scenes in actual application.

[0263] Specifically, after receiving the voice data input by the user, the display device recognizes the voice data as text, and also determines the candidate scenes corresponding to the current display content and the weights of the candidate scenes.

[0264] As an implementable manner, the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes may be determined according to any one of the type of the currently played program, the currently opened application, and the currently displayed hybrid application interface.

[0265] Among them, the scene corresponding to the currently displayed content can be directly determined according to the type of the currently played program and the currently opened application. The type of the currently played program can include music, movies, variety shows, and weather, etc., and accordingly, the scene corresponding to the currently displayed content can be music, movies, variety shows, and weather. The currently opened application can include music applications (APP), shopping applications, weather applications, education applications, film and television applications, and control applications, etc., and accordingly, the scene corresponding to the currently displayed content can be music, shopping, weather, education, film and television, and control.

[0266] The hybrid application interface currently displayed is a hybrid application interface of a display device. For example, a display device is a TV. The homepage entered after the TV is turned on is a hybrid application interface. Fig.14 A display diagram of a hybrid application interface provided in an embodiment of the present application is shown as follows: Fig.14 As shown, the content displayed in the hybrid application interface includes movies, variety shows, music, advertising space, recommendations and others. When a user selects a certain content, he can enter the corresponding application or play the corresponding content. It is understandable that if the hybrid application interface is currently displayed, the scene selected by the user is uncertain, that is, the candidate scene. At this time, the weight of the candidate scene needs to be determined so that the server can perform semantic analysis based on the weight of the candidate scene and the candidate scene.

[0267] As an implementable manner, S102 may be:

[0268] S1021. If the user interface displays a currently playing program, determine the scene corresponding to the type of the currently playing program as the candidate scene corresponding to the currently displayed content, and determine the maximum weight value in the preset weight range as the weight of the scene corresponding to the type of the currently playing program.

[0269] S1022. If the user interface displays a currently opened application, determine the scene corresponding to the currently opened application as the candidate scene corresponding to the currently displayed content, and determine the maximum weight value in the preset weight range as the weight of the scene corresponding to the currently opened application.

[0270] Specifically, the scene corresponding to the currently displayed content can be directly determined according to the type of the currently played program and the currently opened application. Therefore, when determining the weights of the candidate scenes in these two situations, the maximum weight value in the preset weight range can be determined as the weight of the scene corresponding to the type of the currently played program. For example, the preset weight range is 0 to 1, the maximum weight value is 1, and the minimum weight value is 0.

[0271] S1023: If the user interface displays a hybrid application interface, determine the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes according to the interface elements of the hybrid application interface.

[0272] Specifically, if the user interface displays a hybrid application interface, that is, the current display content is a hybrid application interface, then the scene corresponding to the current display content is uncertain, that is, a candidate scene. Fig.14 Taking the hybrid application interface shown as an example, the content displayed by the hybrid application interface includes movies, variety shows, music, recently watched, recommended and others. Accordingly, the interface elements of the hybrid application interface are movies, variety shows, music, recently watched, recommended and others. In this case, as an implementable method, the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes are determined according to the interface elements of the hybrid application interface, which can be specifically:

[0273] First, when rendering the interface elements of the hybrid application interface, a scene label and a weight are configured for each interface element according to the position and size of the interface element and a preset weight configuration range.

[0274] Specifically, when the display device renders the interface elements of the hybrid application interface, for each interface element, the scene label and weight can be configured according to the position and size and the preset weight configuration range. Fig.15 A schematic diagram of configuring scene labels and weights for interface elements of a hybrid application interface provided in an embodiment of the present application is shown in FIG. Fig.15As shown, the hybrid application interface has a total of 18 interface elements, and the preset weight configuration range is, for example, 1-5 points (the lowest is 1 point and the highest is 5 points). The weight corresponding to the scene label is configured according to the position and size of the interface element. The scene label can be pre-set according to the position. For example, the scene label of the interface element located in the center of the hybrid application interface and with the largest area is Movie 1, and its weight is the highest score of 5 points. The scene labels and weights of the three interface elements on the right side of the hybrid application interface are Movie 2: 4 points, Variety Show 1: 4 points and Music: 4 points. The four interface elements on the left side of the hybrid application interface are Movie 2: 4 points, Variety Show 1: 4 points and Music: 4 points. The scene labels and weights of the interface elements are Variety Show 2: 2 points, Variety Show 3: 2 points, Variety Show 4: 2 points, and Variety Show 5: 2 points. The scene labels and weights of the four smaller interface elements located in the center of the hybrid application interface are Movie 3: 3 points, Movie 4: 3 points, Movie 5: 3 points, and Movie 6: 3 points. Among the five interface elements located at the top of the hybrid application interface, the scene labels and weights of two of the interface elements are Other: 1 point, Recommended: 1 point. The interface element located at the bottom of the hybrid application interface is the ad slot. Since users generally do not click on the ad slot, the ad slot has no scene label and weight.

[0275] Next, a normalization calculation is performed according to the weight configured for each interface element to obtain the normalized weight of each interface element.

[0276] Specifically, the normalization calculation is performed according to the weight of each interface element configuration, and the normalization calculation can be performed according to the following formula to obtain the normalized weight X of each interface element: C :

[0277]

[0278] Among them, X is the weight of the interface element configuration, X max and X min are respectively the highest score and the lowest score in the preset weight configuration range, where the preset weight configuration range is, for example, 1-5 points, and X max and X min 5 points and 1 point respectively. Fig.15 Taking the scene labels and weights shown in as an example, the weights configured for each interface element are normalized and calculated. The normalized weights of each interface element are calculated by the above formula, as shown in Table 1 below:

[0279] Table 1 Normalized weight of each interface element

[0280]

[0281]

[0282] Then, according to the scene labels, the scene labels and normalized weights of all interface elements of the hybrid application interface are summed and normalized to obtain the normalized weight of each scene label.

[0283] Specifically, the scene labels of different interface elements can be the same, which can be specifically the sum of the normalized weights of the interface elements with the same scene label. For example, all interface elements corresponding to scene label t are t1, t2, ... tn, and the normalized weights of interface elements t1, t2, ... tn are g1, g2, ... gn respectively. The normalized weights of scene label t are summed to be g1+g2+...+gn. For example, the scene labels and normalized weights of the interface elements shown in Table 1 are summed and normalized, and the normalization is also performed using the above formula According to the normalized weights after summation in Table 2, X max and X min They are 3.75 and 0 respectively. Through calculation, the normalized weight of each scene label is shown in Table 2 below:

[0284] Table 2 Normalized weights of each scene label

[0285] Scene Tags Normalized weight sum Normalized weight other 0 0 recommend 0 0 Movie 3.75 1 Variety Show 1.75 0.47 music 0.75 0.2

[0286] Finally, the scene label whose normalized weight is greater than a preset threshold is determined as the candidate scene corresponding to the currently displayed content, and the normalized weight of the corresponding scene label is determined as the weight of the candidate scene.

[0287] For example, the preset threshold is 0.5, and the scene label with a normalized weight greater than 0.5 is determined as the candidate scene corresponding to the currently displayed content, and the normalized weight of the corresponding scene label is determined as its weight. As shown in Table 2, the scene label with a normalized weight greater than 0.5 is 1, and the movie is determined as the candidate scene corresponding to the currently displayed content. The normalized weight of the movie is 1, and the weight of the candidate scene "movie" is 1.

[0288] S103: The display device sends a parsing request to the server, where the parsing request includes candidate scenes corresponding to the current display content, weights of the candidate scenes, and text corresponding to the current voice data.

[0289] Specifically, after obtaining the text corresponding to the current voice data and the candidate scenes corresponding to the current display content and the weights of the candidate scenes through S101 and S102, the display device sends a parsing request to the server.

[0290] In this embodiment, while the display device recognizes the text corresponding to the voice data input by the user, it determines the candidate scenes corresponding to the currently displayed content (that is, it infers the scene where the user may currently be) and the weights of the candidate scenes, and then carries the text corresponding to the voice data input by the user and the determined candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes in a parsing request and sends it to the server. When the server parses the text, it can parse the text in combination with the scene where the user may currently be, instead of performing semantic parsing only based on the text, or performing semantic parsing based on the text and the text corresponding to the previous voice data, thereby improving the accuracy of semantic parsing and improving user experience.

[0291] S104. The server receives the parsing request sent by the display device, performs semantic parsing on the text corresponding to the current voice data according to the candidate scenes corresponding to the current display content and the weights of the candidate scenes, obtains a semantic parsing result, and determines the user intention information according to the semantic parsing result.

[0292] Specifically, after the server receives the parsing request sent by the display device, before the text corresponding to the current voice data is semantically parsed according to the candidate scenes corresponding to the current displayed content and the weights of the candidate scenes, it is optional to determine that there is no text corresponding to the previous voice data of the current voice data. That is, when it is determined that there is no previous text, the text corresponding to the current voice data is semantically parsed according to the candidate scenes corresponding to the current displayed content and the weights of the candidate scenes. If it is determined that there is text corresponding to the previous voice data of the current voice data, that is, when it is determined that there is a previous text, the scene of the text parsing corresponding to the previous voice data of the current voice data is semantically parsed with the candidate scenes corresponding to the current displayed content, the weights of the candidate scenes and the text corresponding to the current voice data to obtain a semantic parsing result. If it is determined that there is a previous text (that is, the text corresponding to the previous voice data of the current voice data), the scene of the above text parsing is combined with the candidate scenes corresponding to the current displayed content, the weights of the candidate scenes and the text corresponding to the current voice data for semantic parsing. Since the above text is the voice data input by the user, the accuracy of the intention parsing can be improved. Therefore, this embodiment can realize semantic analysis of the text corresponding to the current voice data according to the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes when there is no text corresponding to the previous voice data, which can improve the accuracy of intent analysis.

[0293] As an implementable method, semantic analysis is performed based on the scene of text analysis corresponding to the previous voice data of the current voice data, the candidate scene corresponding to the current display content, the weight of the candidate scene and the text corresponding to the current voice data to obtain a semantic analysis result, which can be:

[0294] If the similarity between the scene of text analysis corresponding to the previous voice data of the current voice data and the candidate scene corresponding to the currently displayed content is greater than or equal to the similarity threshold, semantic analysis is performed on the text corresponding to the current voice data according to the scene of text analysis corresponding to the previous voice data of the current voice data to obtain a semantic analysis result.

[0295] If the similarity between the scene of the text analysis corresponding to the previous voice data of the current voice data and the candidate scene corresponding to the currently displayed content is less than the similarity threshold, the text corresponding to the current voice data is semantically analyzed according to the candidate scene corresponding to the currently displayed content and the weight of the candidate scene to obtain a semantic analysis result.

[0296] Specifically, the scene of text analysis corresponding to the last speech data of the current speech data is the scene of the previous analysis. First, the similarity between the scene of the previous analysis and the candidate scene corresponding to the current display content is compared. If the similarity between the two is greater than or equal to the similarity threshold, that is, the scene of text analysis corresponding to the last speech data is consistent with the scene corresponding to the current display content, then the text corresponding to the current speech data is semantically analyzed according to the scene of text analysis corresponding to the last speech data of the current speech data to obtain a semantic analysis result. If the similarity between the two is less than the similarity threshold, that is, the scene of text analysis corresponding to the last speech data is inconsistent with the scene corresponding to the current display content, then the text corresponding to the current speech data is semantically analyzed according to the candidate scene corresponding to the current display content and the weight of the candidate scene to obtain a semantic analysis result. By semantically analyzing the text corresponding to the current speech data according to the candidate scene corresponding to the current display content and the weight of the candidate scene when the scene of text analysis corresponding to the last speech data is inconsistent with the scene corresponding to the current display content, the accuracy of semantic analysis can be improved and the probability of deviation between the semantic analysis result and the user's intention can be reduced.

[0297] It should be noted that the candidate scene corresponding to the currently displayed content may be one or more. If not, the processing flow is the same for each candidate scene, which is to first compare the similarity between the two, and then determine the basis for semantic analysis based on the similarity and the similarity threshold and perform semantic analysis.

[0298] The following Table 3 is an example of the correspondence between the candidate scenes corresponding to the currently displayed content, the text corresponding to the current voice data, and the semantic analysis results in the embodiment of the present application. The natural language parsing method provided in the embodiment of the present application can perform semantic analysis on the text corresponding to the current voice data in combination with the candidate scenes corresponding to the currently displayed content, thereby improving the accuracy of the semantic analysis and improving the user experience.

[0299] Table 3. Examples of semantic parsing results combined with scenes

[0300]

[0301] After obtaining the semantic analysis result, the user intent information is determined based on the semantic analysis result. For example, if the semantic analysis result is "the latest movie", the user intent can be determined based on the semantic analysis result. The server is also required to obtain the latest movie information from the resource library as user intent information, such as movie links and pictures.

[0302] S105. The server sends the user intention information to the display device.

[0303] S106: The display device receives the user intention information sent by the server, and executes an operation corresponding to the user intention information.

[0304] Specifically, after receiving the user intention information sent by the server, the display device executes the operation corresponding to the user intention information, for example, the user intention information may be displayed on the display interface, such as playing a movie or music, displaying a picture, etc.

[0305] The natural language parsing method provided in this embodiment determines the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes while the display device recognizes the text corresponding to the voice data input by the user, and then carries the text corresponding to the voice data input by the user and the determined candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes in a parsing request and sends it to the server, so that when the server parses the text, it performs semantic parsing on the text corresponding to the current voice data according to the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes to obtain a semantic parsing result, and determines the user intention information according to the semantic parsing result, so that when the server parses the text, it can parse the text in combination with the candidate scenes corresponding to the currently displayed content, instead of only performing semantic parsing based on the text, or performing semantic parsing based on the text and the text corresponding to the previous voice data, thereby improving the accuracy of semantic parsing and improving user experience.

[0306] Fig.16 A flowchart of a natural language parsing method embodiment provided in an embodiment of the present application, such as Fig.16 As shown, the method of this embodiment is Fig.13 Based on the method shown, optional, Fig.13 In the embodiment shown in S104, semantic parsing is performed on the text corresponding to the current voice data according to the candidate scenes corresponding to the current display content and the weights of the candidate scenes to obtain a semantic parsing result. This can be achieved through the following steps, and the executor of this embodiment can be a server.

[0307] S1041. Perform word segmentation annotation on the text corresponding to the current voice data to obtain word segmentation annotation information, where the word segmentation annotation information includes at least one word and an attribute annotation corresponding to each word.

[0308] Specifically, word segmentation tagging involves word segmentation and part-of-speech tagging. A word is the smallest meaningful language component that can act independently. Word segmentation is the first step in natural language processing. Unlike English, where each word is separated by spaces or punctuation marks, it is difficult to define the boundaries of words in Chinese. The current mainstream word segmentation is based on three categories: rule-based, statistical, and understanding. In the embodiment of the present application, rule-based word segmentation is adopted, based on the vocabulary, and a forward maximum matching algorithm is used to perform word segmentation. For example: "I want to watch Peppa Pig's cartoons", the word segmentation is completed as: I want to watch Peppa Pig's cartoons.

[0309] Among them, part-of-speech tagging is a classification method that uses the characteristics of words as the basis for dividing word classes. In the embodiment of the present application, a word segmentation tagging method based on a word library and based on rules can be used. For example, the text is "I want to watch Peppa Pig's cartoons", and the word segmentation tagging information obtained after word segmentation tagging is: {I want-I want [funcwordintention]}, {watch-watch [actionWatch, funcwordexcl]}, {Peppa Pig-Peppa Pig [title, eduKeys]}, {of-of [funcwordstructaux]}, {cartoon-animation [rname, musicType, mtype, eduRname]}. Among them, "I want", "watch", "Peppa Pig", "of" and "cartoon" are words in the word segmentation tagging information, and "I want [funcwordintention]", "watch [actionWatch, funcwordexcl]", "Peppa Pig [title, eduKeys]", "of [funcwordstructaux]", "animation [rname, musicType, mtype, eduRname]" are attribute tags corresponding to each word in the word segmentation tagging information.

[0310] S1042: Locate the domain intent according to the word segmentation annotation information to determine whether the first domain intent is located.

[0311] Specifically, after obtaining the word segmentation annotation information through word segmentation annotation, the domain intent is first located according to the word segmentation annotation information to determine whether the first domain intent is located. If so, execute S1043, and if not, execute S1044.

[0312] In this embodiment, optionally, S1042 has the following two implementable methods:

[0313] Method 1: Match the word segmentation annotation information with a preset rule template to determine whether it matches the first domain intention, wherein the preset rule template includes annotations, regular expressions, and weights.

[0314] Among them, the preset rule template can be a rule-based text logic processing. Through big data analysis, texts that are frequently used by users and have certain rules are abstracted and extracted, and the preset rule template is written.

[0315] For example, the following four rule templates are:

[0316] (1) Movies of "cast", 1.0 - Film field

[0317] (2) Buy a flight ticket from "districtcn" to "districtcn", 1.0 - Ticketing field

[0318] (3) Weather in districtcn, 1.0 - Weather field

[0319] (4) Encyclopedia《.*》, 0.9--Encyclopedia field

[0320] The domain intents corresponding to the four rule templates are respectively the movie domain, ticket domain, weather domain, and encyclopedia domain. In rule template (1), "cast" is a label, "》 movie is a regular expression, and 1.0 is the weight. In rule template (2), "districtcn" and "districtcn" are labels, buying a ticket from "》 to "》" is a regular expression, and 1.0 is the weight. In rule template (3), "districtcn" is a label, "》 weather is a regular expression, and 1.0 is the weight. In rule template (4), ".*" is a label, encyclopedia "》" is a regular expression, and 0.9 is the weight.

[0321] Specifically, each word in the word segmentation annotation information and the attribute annotation corresponding to the word are matched with the preset rule template to determine whether the corresponding domain intent is matched. If matched, the matched domain intent is used as the first domain intent.

[0322] Method 2: Match the word segmentation annotation information with the preset dependency syntactic relationship and weight to determine whether it matches the first domain intention. The preset dependency syntactic relationship is configured according to the dependency syntactic tree.

[0323] Specifically, the syntactic structure essentially contains the relationship between words and word pairs. This kind of relationship is called dependency relations. Among them, a dependency relation connects two words, one is the core word (head) and the other is the modifier (dependant). A typical representation of the result of dependency syntactic analysis is the dependency syntactic tree. Dependency relations can include: subject-predicate relationship, verb-object relationship, indirect object relationship, prepositional object, auxiliary modifier, attributive-predicate relationship, adverbial-predicate relationship, verb-complement relationship, parallel relationship, preposition-object relationship, left-attached relationship, right-attached relationship, independent structure and core relationship, etc.

[0324] In this embodiment, the core words corresponding to each field and the dependency relationship between other words and the core words can be compiled based on empirical knowledge, for example:

[0325] First determine the core words in each field, taking the film and television field and the music field as examples:

[0326] Core words in the film and television field: rname (video category), title (video name)

[0327] Core words in the music field: musicKey (music keyword), musicName (song name)

[0328] Next, configure the dependencies and weights of the current syntax, for example:

[0329] Modifier + core word + weight:

[0330] {center:”rname”,index:[cast,rname],weight:1.0},

[0331] {center:”musicName”,index:[singer,musicName],weight:1.0}

[0332] Verb + core word:

[0333] {center:”rname”,index:[watch,rname],weight:1.0},

[0334] {center:”musicName”,index:[listen,musicName],weight:1.0}

[0335] Specifically, match the word segmentation annotation information with the preset dependency syntactic relations and weights. For example, perform word segmentation annotation on "张三的" to obtain the word segmentation annotation information: [{张三-张三[cast, singer, videoRoleNameFeeble]}, {的-的[funcwordstructaux]}]. Match the words and attribute annotations in the word segmentation annotation information: [{张三-张三[cast, singer, videoRoleNameFeeble]}, {的-的[funcwordstructaux]}] with the above preset dependency syntactic relations and weights, and no domain core words can be found.

[0336] S1043. If the first domain intention is located, determine the first domain intention as the semantic parsing result.

[0337] For example, if the first domain intention located is "最新的电影" (the latest movie), then determine "最新的电影" (the latest movie) as the semantic parsing result.

[0338] S1044. If the first domain intention is not located, perform domain intention location on the word segmentation annotation information according to the candidate scenarios corresponding to the current display content and the weights of the candidate scenarios, and determine the located second domain intention as the semantic parsing result.

[0339] Specifically, performing domain intention location on the word segmentation annotation information according to the candidate scenarios corresponding to the current display content and the weights of the candidate scenarios can be as follows:

[0340] First, for each candidate scenario corresponding to the current display content, determine the relevant word list of each candidate scenario according to the mapping relationship between the preset scenarios and the relevant word lists. The relevant word list includes at least one relevant word and the attribute annotation corresponding to each relevant word.

[0341] Among them, in this embodiment, in this implementation manner, it is necessary to preset the mapping relationship between the scenarios and the relevant word lists. The relevant word list includes at least one relevant word and the attribute annotation corresponding to each relevant word. The following Table 4 is an example of the mapping relationship between a scenario and a relevant word list:

[0342] Table 4 Mapping Relationship between Scenarios and Relevant Word Lists

[0343] Scenario Related words Attribute annotation movie Movie rname,programType,programRname movie Film and Television rname,musicType,SLLO,SLLOALONE music song musicKey music music musicKey,appTag edu educate Mtype,programType,eduKeyWords,vodChann

[0344] Next, combine the relevant word list of each candidate scenario with the word segmentation annotation information respectively to obtain the combined word segmentation annotation information corresponding to each candidate scenario. Each candidate scenario corresponds to at least one combined word segmentation annotation information.

[0345] For example, the candidate scenarios are movie and music. The candidate scenarios movie and music respectively correspond to two lists of related words. The lists of related words for the candidate scenarios movie and music are as follows:

[0346] movie - movie[rname, programType, programRname]

[0347] film and television - film and television[rname, musicType, SILO, SILOALONE]

[0348] song - song[musicKey]

[0349] music - music[musicKey, appTag]

[0350] For each candidate scenario, the list of related words for the candidate scenario is combined with the word segmentation annotation information to obtain the combined word segmentation annotation information corresponding to the candidate scenario. In this embodiment, still taking the word segmentation annotation information as [{Zhang San - Zhang San[cast, singer, videoRoleNameFeeble]}, {de - de[funcwordstructaux]}] as an example, for the candidate scenario movie, the list of related words: movie - movie[rname, programType, programRname] is combined with the word segmentation annotation information to obtain the combined word segmentation annotation information corresponding to the candidate scenario movie: [{Zhang San - Zhang San[cast, singer, videoRoleNameFeeble]}, {de - de[funcwordstructaux]}, movie - movie[rname, programType, programRname]}]. The list of related words: film and television - film and television[rname, musicType, SILO, SILOALONE] is combined with the word segmentation annotation information to obtain the combined word segmentation annotation information corresponding to the candidate scenario movie: [{Zhang San - Zhang San[cast, singer, videoRoleNameFeeble]}, {de - de[funcwordstructaux]}, film and television - film and television[rname, musicType, SILO, SILOALONE]}].

[0351] For the candidate scenario music, combine the related word list: song [musicKey] with the word segmentation annotation information to obtain the combined word segmentation annotation information corresponding to the candidate scenario music: [{Zhang San - Zhang San [cast, singer, videoRoleNameFeeble]}, {of - of [funcwordstructaux]}, {song - song [musicKey]}]. Combine the related word list: music - music [musicKey, appTag] with the word segmentation annotation information to obtain the combined word segmentation annotation information corresponding to the candidate scenario music: [{Zhang San - Zhang San [cast, singer, videoRoleNameFeeble]}, {of - of [funcwordstructaux]}, {music - music [musicKey, appTag]}].

[0352] Next, perform domain intention localization according to the combined word segmentation annotation information corresponding to each candidate scenario to obtain at least one domain intention and weight.

[0353] Specifically, performing domain intention localization according to the combined word segmentation annotation information corresponding to each candidate scenario is the same as the specific method of performing domain intention localization according to the word segmentation annotation information above. Optionally, there are two implementable methods, which can be specifically referred to as Method 1 and Method 2 above and will not be elaborated here. The only difference is that the input for performing domain intention localization is different. When performing domain intention localization according to the combined word segmentation annotation information, since the related word list corresponding to the scenario is added, the localization of the domain intention is more accurate, and at least one domain intention and weight can be located.

[0354] Finally, determine the second domain intention according to at least one domain intention and weight and the weight of the candidate scenario.

[0355] Specifically, obtain at least one domain intention and weight, and the weight of each candidate scenario is known. Determine the second domain intention according to at least one domain intention and weight and the weight of each candidate scenario. Specifically, it can be to calculate the final localization weight of each domain intention. The localization weight can be the product of the weight of the domain intention and the weight of the candidate scenario. After calculating the localization weight, it can be to take the domain intention with the largest weight as the second domain intention. For example, if the second domain intention is film and television, the server will further perform film and television search, and the film and television search result is the user intention information, and this user intention information will be sent to the display device for display.

[0356] The natural language parsing method provided in this embodiment performs semantic parsing on the text corresponding to the current voice data according to the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes. The text corresponding to the current voice data is first segmented and annotated to obtain segmentation and annotation information. Then, domain intent positioning is performed according to the segmentation and annotation information to determine whether the first domain intent is positioned. If the first domain intent is positioned, it indicates that the text corresponding to the current voice data is a text with clear intent and the domain intent can be directly positioned. The first domain intent is determined as the semantic parsing result. If the first domain intent is not positioned, it indicates that the text corresponding to the current voice data is a text with ambiguous intent and the domain intent cannot be directly positioned. The domain intent positioning is performed according to the candidate scenes corresponding to the currently displayed content, the weights of the candidate scenes and the segmentation and annotation information. The positioned second domain intent is determined as the semantic parsing result. That is, the domain intent positioning is performed on the segmentation and annotation information in combination with the candidate scenes corresponding to the currently displayed content. This can improve the accuracy of semantic parsing.

[0357] A specific embodiment is used below. Fig.13 and Fig.16 The technical solution of the method embodiment shown is described in detail.

[0358] Fig.17 A flowchart of a natural language parsing method embodiment provided in an embodiment of the present application, such as Fig.17 As shown, the method of this embodiment may include:

[0359] S201, the user inputs current voice data.

[0360] S202: The display device receives current voice data input by the user, and determines the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes according to any one of the type of the currently played program, the currently opened application, and the currently displayed hybrid application interface.

[0361] S203: The display device sends a parsing request to the server, where the parsing request includes candidate scenes corresponding to the current display content, weights of the candidate scenes, and text corresponding to the current voice data.

[0362] In this embodiment and below Fig.18 In the illustrated embodiment, taking the candidate scenes corresponding to the currently displayed content as movie and music as an example, the weights of the candidate scenes corresponding to the currently displayed content are, for example: [{scene:movie, weight:0.98}, {scene:music, weight:0.6}], where the weights are 0.98 and 0.6 respectively.

[0363] S204: The server receives the parsing request sent by the display device, and determines whether there is text corresponding to the previous voice data of the current voice data.

[0364] If not, execute S205, and if so, execute S206.

[0365] S205. The server performs semantic analysis on the text corresponding to the current voice data according to the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes, obtains a semantic analysis result, and determines the user intention information according to the semantic analysis result.

[0366] S206. The server performs semantic analysis based on the scene of the text analysis corresponding to the previous voice data of the current voice data, the candidate scene corresponding to the currently displayed content, the weight of the candidate scene and the text corresponding to the current voice data, obtains the semantic analysis result, and determines the user intention information based on the semantic analysis result.

[0367] Specifically, S206 may include:

[0368] S2061. The server determines whether the similarity between the scene of text analysis corresponding to the previous voice data of the current voice data and the candidate scene corresponding to the current display content is greater than or equal to a similarity threshold.

[0369] If yes, execute S2062; if no, execute S2063.

[0370] S2062: The server performs semantic analysis on the text corresponding to the current voice data according to the scenario of the text analysis corresponding to the previous voice data of the current voice data, and obtains a semantic analysis result.

[0371] S2063. The server performs semantic analysis on the text corresponding to the current voice data according to the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes to obtain a semantic analysis result.

[0372] It should be noted that S2063 is the same as S205.

[0373] The specific process or implementation of S201-S206 in this embodiment can be found in detail. Fig.13 The description in the illustrated embodiment will not be repeated here.

[0374] The natural language parsing method provided in this embodiment, while the display device recognizes the text corresponding to the voice data input by the user, determines the candidate scene corresponding to the current display content and the weight of the candidate scene according to any one of the type of the currently playing program, the currently opened application, and the currently displayed hybrid application interface. Then, the text corresponding to the voice data input by the user, the candidate scene corresponding to the determined current display content, and the weight of the candidate scene are carried in the parsing request and sent to the server. When the server parses the text, it performs semantic parsing on the text corresponding to the current voice data according to the candidate scene corresponding to the current display content and the weight of the candidate scene to obtain a semantic parsing result, and determines the user intention information according to the semantic parsing result. Therefore, when the server parses the text, it can parse the text in combination with the candidate scene corresponding to the current display content, rather than only performing semantic parsing based on the text, or performing semantic parsing based on the text and the text corresponding to the previous voice data, thereby improving the accuracy of semantic parsing and enhancing the user experience.

[0375] The following combines Fig.18 A specific implementable manner is described in detail for the server in S205 or S2063 to perform semantic parsing on the text corresponding to the current voice data according to the candidate scene corresponding to the current display content and the weight of the candidate scene to obtain a semantic parsing result.

[0376] Fig.18 The flowchart of the embodiment of the natural language parsing method provided in this application embodiment. The execution subject of this embodiment can be a server, as Fig.18 shown, the method of this embodiment may include:

[0377] S301. Perform word segmentation and annotation on the text corresponding to the current voice data to obtain word segmentation and annotation information, where the word segmentation and annotation information includes at least one word and the attribute annotation corresponding to each word.

[0378] Specifically, in this embodiment, taking the text corresponding to the current voice data as: "Zhang San's" as an example, where Zhang San is a singer, perform word segmentation and annotation on the text "Zhang San's" corresponding to the current voice data to obtain word segmentation and annotation information: [{Zhang San-Zhang San[cast, singer, videoRoleNameFeeble]}, {'s-'s[funcwordstructaux]}].

[0379] S302. Perform domain intention positioning according to the word segmentation and annotation information to determine whether the first domain intention is located.

[0380] Since the intention of "Zhang San's" is ambiguous, the domain intention cannot be located at this time. Then, S303 is executed.

[0381] Specifically, there are two implementable ways for S302:

[0382] Method 1: Match the word segmentation annotation information with a preset rule template to determine whether the first domain intention is matched. Among them, the preset rule template includes annotation, regular expression, and weight.

[0383] For example, the preset rule templates are as follows:

[0384] (1) The movie "cast", 1.0 - Movie field

[0385] (2) Buy a ticket for "districtcn" to fly to "districtcn", 1.0 - Ticketing field

[0386] (3) The weather in "districtcn", 1.0 - Weather field

[0387] (4) Encyclopedia of ".*", 0.9 - Encyclopedia field

[0388] Match each word and the corresponding attribute annotation in the word segmentation annotation information [{Zhang San - Zhang San [cast, singer, videoRoleNameFeeble]}, {of - of [funcwordstructaux]}] with the above preset rule template. No domain intention can be matched. At this time, for example, UNKNOWN can be sent to the display device.

[0389] Method 2: Match the word segmentation annotation information with a preset dependency syntactic relationship and weight to determine whether the first domain intention is matched. The preset dependency syntactic relationship is configured according to the dependency syntactic tree.

[0390] For example, the preset dependency syntactic relationship and weight are as follows:

[0391] Modifier + Core word + Weight (weight):

[0392] {center: "rname", index: [cast, rname], weight: 1.0},

[0393] {center: "musicName", index: [singer, musicName], weight: 1.0}

[0394] Verb + Core word:

[0395] {center: "rname", index: [watch, rname], weight: 1.0},

[0396] {center:”musicName”,index:[listen,musicName],weight:1.0}

[0397] Specifically, match the word segmentation annotation information with the preset dependency syntactic relationship and weight. For example, segment and annotate "张三的" to obtain the word segmentation annotation information: [{张三-张三[cast,singer,videoRoleNameFeeble]},{的-的[funcwordstructaux]}]. Match the words and attribute annotations in the word segmentation annotation information: [{张三-张三[cast,singer,videoRoleNameFeeble]},{的-的[funcwordstructaux]}] with the above preset dependency syntactic relationship and weight. No domain core word can be found. At this time, for example, UNKNOWN can be sent to the display device.

[0398] S303. For each candidate scene corresponding to the current display content, determine the relevant word list of each candidate scene according to the mapping relationship between the preset scene and the relevant word list. The relevant word list includes at least one relevant word and the attribute annotation corresponding to each relevant word.

[0399] In this embodiment, for example, the mapping relationship between the preset scene and the relevant word list is shown in Table 4. In this embodiment, there are two candidate scenes corresponding to the current display content, [{scene:movie, weight:0.98},{scene:music, weight:0.6}]. The weights of the candidate scenes are 0.98 and 0.6 respectively.

[0400] According to Table 4, the candidate scenes movie and music respectively correspond to two relevant word lists. The relevant word lists of the candidate scenes movie and music are respectively:

[0401] 电影-电影[rname,programType,programRname]

[0402] 影视-影视[rname,musicType,SILO,SILOALONE]

[0403] 歌曲-歌曲[musicKey]

[0404] 音乐-音乐[musicKey,appTag]

[0405] S304. Combine the relevant word list of each candidate scene with the word segmentation annotation information respectively to obtain the combined word segmentation annotation information corresponding to each candidate scene. Each candidate scene corresponds to at least one combined word segmentation annotation information.

[0406] Specifically, for each candidate scenario, the relevant word list of the candidate scenario is combined with the word segmentation annotation information to obtain the combined word segmentation annotation information corresponding to the candidate scenario. In this embodiment, the word segmentation annotation information is [{Zhang San-Zhang San[cast, singer, videoRoleNameFeeble]}, {de-of[funcwordstructaux]}] as an example. For candidate scenarios movie and music, the combined word segmentation annotation information corresponding to the candidate scenarios is as follows:

[0407] (1) [{Zhang San-Zhang San[cast, singer, videoRoleNameFeeble]}, {de-of[funcwordstructaux]}, movie-movie[rname, programType, programRname]}].

[0408] (2) [{Zhang San-Zhang San[cast, singer, videoRoleNameFeeble]}, {de-of[funcwordstructaux]}, film and television-film and television[rname, musicType, SILO, SILOALONE]}].

[0409] (3) [{Zhang San-Zhang San[cast, singer, videoRoleNameFeeble]}, {de-of[funcwordstructaux]}, song-song[musicKey]}].

[0410] (4) [{Zhang San-Zhang San[cast, singer, videoRoleNameFeeble]}, {de-of[funcwordstructaux]}, music-music[musicKey, appTag]}].

[0411] S305. Perform domain intention positioning according to the combined word segmentation annotation information corresponding to each candidate scenario to obtain at least one domain intention and weight.

[0412] Specifically, perform domain intention positioning again according to the combined word segmentation annotation information corresponding to each candidate scenario. For example, for Method 1:

[0413] The preset rule templates are the following 4:

[0414] (1), The movie of 《cast》, 1.0--Movie domain

[0415] (2), Buy a flight ticket from 《districtcn》 to 《districtcn》, 1.0 - Ticketing field

[0416] (3), Weather in 《districtcn》, 1.0 - Weather field

[0417] (4), Encyclopedia 《.*》, 0.9 - Encyclopedia field

[0418] For the first combined word segmentation annotation information, traverse the rule templates. Starting from the first rule template "movie of 《cast》", extract the attribute annotation cast in the rule template, and determine whether cast exists in the first combined word segmentation annotation information. In the first combined word segmentation annotation information [{Zhang San - Zhang San [cast, singer, videoRoleNameFeeble]}, {of - of [funcwordstructaux]}, {movie - movie [rname, programType, programRname]}], cast exists. Therefore, replace the text "Zhang San's" corresponding to the current voice data with "< <cast>>" movie", the current rule template matches successfully. Therefore, the first combined word segmentation annotation information is in the film and television field, and the domain intention = movie, weight = 1.0, that is, the weight of the domain intention movie is 1.0.

[0419] The processing processes for the second, third, and fourth combined word segmentation annotation information are similar and will not be elaborated here.

[0420] For Method 2:

[0421] The preset dependency syntactic relations and weights are as follows:

[0422] (1) {center: "musicName", index: [singer, musicName], weight: 1.0}

[0423] (2) {center: "rname", index: [cast, rname], weight: 1.0}

[0424] (3) {center: "rname", index: [watch, rname], weight: 1.0}

[0425] (4) {center: "musicName", index: [listen, musicName], weight: 1.0}

[0426] For the combined word segmentation annotation information of item (1), traverse the preset dependency syntactic relations. The dependency syntactic relation of item (1) is: {center:”musicName”,index:[singer,musicName],weight:1.0}, with the core word being musicName. Recursively search in the combined word segmentation annotation information of item (1) [{Zhang San-Zhang San[cast,singer,videoRoleNameFeeble]},{of - of[funcwordstructaux]},{movie - movie[rname,programType,programRname]}] to check if there is an attribute annotation of musicName. The core word is not found, so this relation does not match. Continue to match the next combined word segmentation annotation information; The dependency syntactic relation of item (2) is: {center:”rname”,index:[cast,rname],weight:1.0}, with the core word being rname. Search in the combined word segmentation annotation information of item (1) and find the core word. Further analyze the syntax. Before the core word rname is cast. Continue to search forward for the rname word in the combined word segmentation annotation information of item (1) and find the cast annotation. So this syntactic relation matches successfully, obtaining domain intention = movie, weight = 1.0, that is, the weight of the domain intention movie is 1.0.

[0427] The processing procedures for the (2)nd, (3)rd, and (4)th combined word segmentation annotation information are similar and will not be elaborated here.

[0428] Finally, the domain intentions and weights corresponding to the 4 combined word segmentation annotation information are as follows:

[0429] (1)[{Zhang San-Zhang San[cast,singer,videoRoleNameFeeble]},{of - of[funcwordstructaux]},{movie - movie[rname,programType,programRname]}] Domain intention = movie, weight = 1.0

[0430] (2)[{Zhang San-Zhang San[cast,singer,videoRoleNameFeeble]},{of - of[funcwordstructaux]},{film and television - film and television[rname,musicType,SILO,SILOALONE]}] Domain intention = movie, weight = 1.0

[0431] (3) [{Zhang San - Zhang San [cast, singer, videoRoleNameFeeble]}, {of - of [funcwordstructaux]}, {music - music [musicKey, appTag]}] domain intent = music, weight = 1.0

[0432] (4) [{Zhang San - Zhang San [cast, singer, videoRoleNameFeeble]}, {of - of [funcwordstructaux]}, {song - song [musicKey]}] domain intent = music, weight = 1.0

[0433] S306: Determine a second domain intention based on at least one domain intention and weight and the weight of the candidate scenario.

[0434] Specifically, the final positioning weight of each domain intent can be calculated, and the positioning weight can be the product of the weight of the domain intent and the weight of the candidate scene. The calculation result is as follows:

[0435] (1) Domain intent = movie, positioning weight = 0.98

[0436] (2) Domain intent = movie, positioning weight = 0.98

[0437] (3) Domain intent = music, positioning weight = 0.6

[0438] (4) Domain intent = music, positioning weight = 0.6

[0439] After calculating the positioning weight, the domain intention with the largest weight can be used as the second domain intention. For example, if the positioning weight of film and television is the largest, the server will further perform a film and television search. The film and television search results are the user intention information, which is sent to the display device for display.

[0440] The natural language parsing method provided in this embodiment, when the server performs semantic parsing on the text corresponding to the current voice data according to the candidate scenes corresponding to the current displayed content and the weights of the candidate scenes, firstly performs word segmentation annotation on the text corresponding to the current voice data to obtain word segmentation annotation information, and then performs domain intent location according to the word segmentation annotation information to determine whether the first domain intent is located. If the first domain intent is not located, it indicates that the text corresponding to the current voice data is a text with ambiguous intent and the domain intent cannot be directly located. Then, the domain intent is located according to the candidate scenes corresponding to the current displayed content, the weights of the candidate scenes and the word segmentation annotation information. Specifically, for each candidate scene corresponding to the current displayed content, according to the mapping relationship between the preset scene and the related word list, the related word list of each candidate scene is determined, and then the related word list of each candidate scene is respectively combined with the word segmentation annotation information to obtain the combined word segmentation annotation information corresponding to each candidate scene, and then the domain intent is located according to the combined word segmentation annotation information corresponding to each candidate scene to obtain at least one domain intent and weight, and finally the second domain intent is determined according to at least one domain intent and weight and the weight of the candidate scene, and the second domain intent is determined as the semantic parsing result. By combining the candidate scenes corresponding to the currently displayed content with the word segmentation annotation information to locate the domain intent, the accuracy of semantic parsing can be improved.

[0441] Fig.19 A schematic diagram of the structure of a natural language parsing device provided in an embodiment of the present application is shown in FIG. Fig.19 As shown, the device of this embodiment may include: a receiving module 11, a determining module 12 and a sending module 13, wherein:

[0442] The receiving module 11 is used to receive the current voice data input by the user;

[0443] The determination module 12 is used to determine the candidate scenes corresponding to the current display content and the weights of the candidate scenes;

[0444] The sending module 13 is used to send a parsing request to the server, where the parsing request includes the candidate scene corresponding to the current display content, the weight of the candidate scene and the text corresponding to the current voice data;

[0445] The receiving module 11 is further used to receive user intention information sent by the server, and display content corresponding to the user intention information on the user interface.

[0446] Optionally, the determination module 12 is used to determine the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes according to any one of the type of the currently played program, the currently opened application and the currently displayed hybrid application interface.

[0447] Optionally, the determination module 12 is used for:

[0448] If the user interface displays a currently playing program, the scene corresponding to the type of the currently playing program is determined as the candidate scene corresponding to the currently displayed content, and the maximum weight value in the preset weight range is determined as the weight of the scene corresponding to the type of the currently playing program;

[0449] If the user interface displays a currently opened application, the scene corresponding to the currently opened application is determined as the candidate scene corresponding to the currently displayed content, and the maximum weight value in the preset weight range is determined as the weight of the scene corresponding to the currently opened application;

[0450] If the user interface displays a hybrid application interface, the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes are determined according to the interface elements of the hybrid application interface.

[0451] Optionally, the determination module 12 is used for:

[0452] When rendering the interface elements of the hybrid application interface, configure the scene label and weight for each interface element according to the position and size of the interface element and the preset weight configuration range;

[0453] Perform normalization calculation according to the weight configured for each interface element to obtain the normalized weight of each interface element;

[0454] According to the scene labels, the scene labels and normalized weights of all interface elements of the hybrid application interface are summed and normalized to obtain the normalized weight of each scene label;

[0455] The scene label whose normalized weight is greater than a preset threshold is determined as the candidate scene corresponding to the currently displayed content, and the normalized weight of the corresponding scene label is determined as the weight of the candidate scene.

[0456] The device provided in this embodiment can be used to execute the above method. Its implementation method and technical effect are similar, and this embodiment will not be repeated here.

[0457] Fig. 20 A schematic diagram of the structure of a natural language parsing device provided in an embodiment of the present application is shown in FIG. Fig. 20 As shown, the device of this embodiment may include: a receiving module 21, a semantic parsing module 22 and a sending module 23, wherein:

[0458] The receiving module 21 is used to receive a parsing request sent by a display device, the parsing request includes a candidate scene corresponding to the current display content, a weight of the candidate scene and a text corresponding to the current voice data, and the current voice data is input by a user to the display device;

[0459] The semantic analysis module 22 is used to perform semantic analysis on the text corresponding to the current voice data according to the candidate scenes corresponding to the current display content and the weights of the candidate scenes, obtain semantic analysis results, and determine user intention information according to the semantic analysis results;

[0460] The sending module 23 is used to send the user intention information to the display device.

[0461] Optionally, the semantic parsing module 22 is further used for:

[0462] Before performing semantic analysis on the text corresponding to the current voice data according to the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes and obtaining the semantic analysis result, determining that there is no text corresponding to the previous voice data of the current voice data;

[0463] If it is determined that there is text corresponding to the previous voice data of the current voice data, semantic analysis is performed based on the scene of the text analysis corresponding to the previous voice data of the current voice data, the candidate scene corresponding to the currently displayed content, the weight of the candidate scene and the text corresponding to the current voice data to obtain a semantic analysis result.

[0464] The device provided in this embodiment can be used to execute the above method. Its implementation method and technical effect are similar, and this embodiment will not be repeated here.

[0465] Fig.21 A schematic diagram of the structure of a natural language parsing device provided in an embodiment of the present application is shown in FIG. Fig.21 As shown, the device of this embodiment is Fig. 22 On the basis of the device shown in the figure, further, the semantic parsing module 22 may include: a first parsing unit 221 and a second parsing unit 222, wherein:

[0466] The first parsing unit 221 is used to perform semantic parsing on the text corresponding to the current voice data according to the scene of the text parsing corresponding to the previous voice data of the current voice data, and obtain a semantic parsing result when the similarity between the scene of the text parsing corresponding to the previous voice data of the current voice data and the candidate scene corresponding to the current display content is greater than or equal to the similarity threshold;

[0467] The second parsing unit 222 is used to perform semantic analysis on the text corresponding to the current voice data according to the candidate scene corresponding to the current display content and the weight of the candidate scene when the similarity between the text parsed scene corresponding to the previous voice data of the current voice data and the candidate scene corresponding to the currently displayed content is less than a similarity threshold, to obtain a semantic analysis result.

[0468] Optionally, the second parsing unit 222 is used for:

[0469] Perform word segmentation and annotation on the text corresponding to the current voice data to obtain word segmentation and annotation information, where the word segmentation and annotation information includes at least one word and an attribute annotation corresponding to each word;

[0470] Locate the domain intent based on the word segmentation and annotation information to determine whether the first domain intent has been located;

[0471] If the first domain intention is located, the first domain intention is determined as the semantic analysis result;

[0472] If the first domain intent is not located, the domain intent is located according to the candidate scenes corresponding to the currently displayed content, the weights of the candidate scenes and the word segmentation annotation information, and the located second domain intent is determined as the semantic analysis result.

[0473] Optionally, the second parsing unit 222 is used for:

[0474] For each candidate scene corresponding to the currently displayed content, determining a related word list for each candidate scene according to a preset mapping relationship between the scene and the related word list, wherein the related word list includes at least one related word and an attribute label corresponding to each related word;

[0475] Combining the related word list of each candidate scene with the word segmentation annotation information respectively to obtain combined word segmentation annotation information corresponding to each candidate scene, and each candidate scene corresponds to at least one combined word segmentation annotation information;

[0476] Based on the combined word segmentation annotation information corresponding to each candidate scene, the domain intent is located to obtain at least one domain intent and weight;

[0477] A second domain intention is determined based on at least one domain intention and weight and the weight of the candidate scenario.

[0478] Optionally, the second parsing unit 222 is used for:

[0479] Calculating a target weight of each domain intent according to a weight of each domain intent in at least one domain intent and a weight of each candidate scenario;

[0480] The domain intent with the largest target weight is determined as the second domain intent.

[0481] Optionally, the second parsing unit 222 is used for:

[0482] Match the word segmentation annotation information with a preset rule template to determine whether it matches the first domain intent, and the preset rule template includes attribute annotation, regular expression and weight; or, match the word segmentation annotation information with a preset dependency syntactic relationship and weight to determine whether it matches the first domain intent, and the preset dependency syntactic relationship is configured according to the dependency syntax tree.

[0483] In this application, the display device and the server can be divided into functional modules according to the above method examples. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the various embodiments of this application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0484] Fig. 22 This is a schematic diagram of the hardware structure of the display device provided in this application. Fig. 22 As shown, the display device is used to implement the operation corresponding to the display device in any of the above method embodiments. The display device of this embodiment may include: a display 31 and a controller 32;

[0485] The display 31 is used to display images and a user interface, and a selector in the user interface to indicate that an item in the user interface is selected;

[0486] The controller 32 is used to:

[0487] Receive current voice data input by the user, and determine the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes;

[0488] Sending a parsing request to a server, the parsing request including the candidate scene corresponding to the currently displayed content, the weight of the candidate scene, and the text corresponding to the current voice data, so that the server performs semantic parsing on the text corresponding to the current voice data according to the candidate scene corresponding to the currently displayed content and the weight of the candidate scene, obtains a semantic parsing result, and determines user intention information according to the semantic parsing result;

[0489] Receive user intent information sent by the server and perform operations corresponding to the user intent information.

[0490] Fig.23 The hardware structure diagram of the server provided for this application. Fig.23 As shown, the server is used to implement the operations corresponding to the server in any of the above method embodiments. The server in this embodiment may include:

[0491] Memory 40 and processor 41, wherein

[0492] The memory 40 is used to store processor executable instructions;

[0493] The processor 41 is configured to execute the natural language parsing method in any of the above method embodiments.

[0494] Optionally, the server of this embodiment may further include a receiver 42 and a transmitter 43 .

[0495] Optionally, the receiver 42 may be used to receive a parsing request sent by a display device, and the transmitter 43 may be used to send user intention information to the display device.

[0496] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the natural language parsing method as described in the above embodiment.

[0497] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0498] For ease of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are intended to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.< / cast>

Claims

1. A display device, characterized in that: include: A display for displaying images and a user interface; Controller for: Receive current voice data input by the user; Determining candidate scenes corresponding to the currently displayed content and weights of the candidate scenes; Sending a parsing request to a server, the parsing request including the candidate scenes corresponding to the currently displayed content, the weights of the candidate scenes, and the text corresponding to the current voice data, so that the server performs semantic parsing on the text corresponding to the current voice data according to the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes, obtains a semantic parsing result, and determines user intent information according to the semantic parsing result; receiving the user intention information sent by the server, and performing an operation corresponding to the user intention information; The controller is used to: If the user interface displays a currently playing program, determining a scene corresponding to the type of the currently playing program as a candidate scene corresponding to the currently displayed content, and determining a maximum value of a weight in a preset weight range as a weight of the scene corresponding to the type of the currently playing program; If the user interface displays a currently opened application, the scene corresponding to the currently opened application is determined as the candidate scene corresponding to the currently displayed content, and the maximum weight value in the preset weight range is determined as the weight of the scene corresponding to the currently opened application; If the user interface displays a hybrid application interface, when rendering interface elements of the hybrid application interface, a scene label and a weight are configured for each interface element according to the position and size of the interface element and a preset weight configuration range, and each interface element corresponds to a candidate scene; Perform normalization calculation according to the weight configured for each interface element to obtain the normalized weight of each interface element; According to the scene labels, the scene labels and normalized weights of all interface elements of the hybrid application interface are summed and normalized to obtain a normalized weight of each scene label; The scene label whose normalized weight is greater than a preset threshold is determined as the candidate scene corresponding to the currently displayed content, and the normalized weight of the corresponding scene label is determined as the weight of the candidate scene.

2. The device according to claim 1, characterized in that The controller is used to: According to any one of the type of the currently playing program, the currently opened application and the currently displayed hybrid application interface, the candidate scene corresponding to the currently displayed content and the weight of the candidate scene are determined.

3. A natural language parsing method, characterized in that: include: Receive current voice data input by the user; Determining candidate scenes corresponding to the currently displayed content and weights of the candidate scenes; Sending a parsing request to a server, the parsing request including a candidate scene corresponding to the currently displayed content, a weight of the candidate scene, and a text corresponding to the current voice data; Receiving user intention information sent by the server, and performing an operation corresponding to the user intention information; The determining of the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes includes: If the user interface displays a currently playing program, determining a scene corresponding to the type of the currently playing program as a candidate scene corresponding to the currently displayed content, and determining a maximum value of a weight in a preset weight range as a weight of the scene corresponding to the type of the currently playing program; If the user interface displays a currently opened application, the scene corresponding to the currently opened application is determined as the candidate scene corresponding to the currently displayed content, and the maximum weight value in the preset weight range is determined as the weight of the scene corresponding to the currently opened application; If the user interface displays a hybrid application interface, When rendering the interface elements of the hybrid application interface, a scene label and a weight are configured for each interface element according to the position and size of the interface element and a preset weight configuration range, wherein each interface element corresponds to a candidate scene; Perform normalization calculation according to the weight configured for each interface element to obtain the normalized weight of each interface element; According to the scene labels, the scene labels and normalized weights of all interface elements of the hybrid application interface are summed and normalized to obtain a normalized weight of each scene label; The scene label whose normalized weight is greater than a preset threshold is determined as the candidate scene corresponding to the currently displayed content, and the normalized weight of the corresponding scene label is determined as the weight of the candidate scene.

4. The method according to claim 3, characterized in that: The determining of the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes includes: According to any one of the type of the currently playing program, the currently opened application and the currently displayed hybrid application interface, the candidate scene corresponding to the currently displayed content and the weight of the candidate scene are determined.

5. A natural language parsing method, characterized in that: include: Receiving a parsing request sent by a display device, the parsing request including a candidate scene corresponding to a currently displayed content, a weight of the candidate scene, and text corresponding to current voice data, the current voice data being input to the display device by a user; Performing semantic analysis on the text corresponding to the current voice data according to the candidate scenes corresponding to the currently displayed content and the weights of the candidate scenes to obtain a semantic analysis result, and determining user intention information according to the semantic analysis result; sending the user intention information to the display device; The performing semantic analysis on the text corresponding to the current voice data according to the candidate scenes corresponding to the current displayed content and the weights of the candidate scenes to obtain a semantic analysis result includes: Performing word segmentation annotation on the text corresponding to the current voice data to obtain word segmentation annotation information, wherein the word segmentation annotation information includes at least one word and an attribute annotation corresponding to each word; Perform domain intent location according to the word segmentation annotation information to determine whether the first domain intent is located; If the first domain intention is located, determining the first domain intention as the semantic parsing result; If the first domain intent is not located, the domain intent is located according to the candidate scene corresponding to the currently displayed content, the weight of the candidate scene and the word segmentation annotation information, and the located second domain intent is determined as the semantic analysis result; Before performing semantic parsing on the text corresponding to the current voice data according to the candidate scenes corresponding to the current displayed content and the weights of the candidate scenes to obtain the semantic parsing result, the method further includes: Determine whether there is text corresponding to the previous voice data of the current voice data; If it is determined that there is text corresponding to the previous voice data of the current voice data, semantic analysis is performed based on the scene of the text analysis corresponding to the previous voice data of the current voice data, the candidate scene corresponding to the currently displayed content, the weight of the candidate scene and the text corresponding to the current voice data to obtain a semantic analysis result.

6. The method according to claim 5, characterized in that The semantic parsing is performed based on the scene of the text parsing corresponding to the previous voice data of the current voice data, the candidate scene corresponding to the current display content, the weight of the candidate scene and the text corresponding to the current voice data to obtain the semantic parsing result, including: If the similarity between the scene of text analysis corresponding to the previous voice data of the current voice data and the candidate scene corresponding to the currently displayed content is greater than or equal to the similarity threshold, semantic analysis is performed on the text corresponding to the current voice data according to the scene of text analysis corresponding to the previous voice data of the current voice data to obtain a semantic analysis result; If the similarity between the scene of the text analysis corresponding to the previous voice data of the current voice data and the candidate scene corresponding to the currently displayed content is less than the similarity threshold, the text corresponding to the current voice data is semantically analyzed according to the candidate scene corresponding to the currently displayed content and the weight of the candidate scene to obtain a semantic analysis result.

7. The method according to claim 6, characterized in that The performing domain intent positioning according to the candidate scenes corresponding to the currently displayed content, the weights of the candidate scenes and the word segmentation annotation information includes: For each candidate scene corresponding to the currently displayed content, determining a related word list for each candidate scene according to a preset mapping relationship between the scene and the related word list, wherein the related word list includes at least one related word and an attribute label corresponding to each related word; Combining the related word list of each candidate scene with the word segmentation annotation information respectively to obtain combined word segmentation annotation information corresponding to each candidate scene, each candidate scene corresponding to at least one combined word segmentation annotation information; Perform domain intent location according to the combined word segmentation annotation information corresponding to each of the candidate scenarios to obtain at least one domain intent and weight; The second domain intention is determined based on the at least one domain intention and weight and the weight of the candidate scenario.

8. A server, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the natural language parsing method described in any one of claims 5-7 by executing the executable instructions.

Images