Terminal device and semantic understanding method

Abstract

The application provides a terminal device and a semantic understanding method. The method can input a natural language text input by a user into an intent classification model, perform intent classification, obtain a target domain category and a target device identifier, and mount a target domain adapter corresponding to the target domain category to a base model. The natural language text input is input into the base model mounted with the target domain adapter to perform intent recognition and slot extraction, obtain original slot information, and then map the original slot information into a device instruction corresponding to the target device. If the target device is a terminal device, the device instruction is executed. If the target device is an external device, a communication device is controlled to send the device instruction to the external device. The method realizes coarse-grained classification at the domain level through the intent classification model, and realizes fine-grained intent recognition and slot extraction within the domain through the domain adapter, thereby improving the accuracy of intent recognition in a multi-business domain scenario.

Description

Technical Field

[0001] This application relates to the field of terminal equipment technology, and in particular to a terminal equipment and a semantic understanding method. Background Technology

[0002] Terminal devices refer to devices capable of performing specific functions, such as smart TVs, communication terminals, smart advertising screens, projectors, printers, scanners, etc. Taking smart TVs as an example, a smart TV is a television product based on Internet application technology, equipped with an open operating system and chip, possessing an open application platform, enabling two-way human-computer interaction, and integrating multiple functions such as audio-visual, entertainment, and data to meet diverse and personalized user needs.

[0003] The terminal device supports voice interaction, allowing users to control it via natural language commands to perform various operations such as channel switching, volume adjustment, appliance control, and media asset searching. The terminal device needs to accurately understand the user's natural language commands and translate them into executable device commands.

[0004] Existing semantic understanding solutions typically employ a single model to process natural language commands across all business domains. However, in practical applications, user natural language commands often involve multiple different business domains, such as TV control, smart home device control, and media content search. Semantic overlap and intent confusion exist between these different business domains. For example, the natural language command "turn up" might mean increasing the volume in TV control, but in air conditioner control it might mean increasing the air conditioner temperature. A single model struggles to effectively distinguish these cross-domain semantic differences, leading to decreased intent recognition accuracy and consequently affecting the correct execution of user commands by the terminal device.

[0005] Furthermore, when a user's natural language command involves multiple devices, such as simultaneously controlling a washing machine and a television, it is necessary to accurately identify the multiple devices involved and correctly route the generated device commands to the appropriate devices for execution. A single model, when handling such cross-device scenarios, is prone to causing command routing confusion due to low intent recognition accuracy, thus degrading the user experience. Summary of the Invention

[0006] This application provides a terminal device and a semantic understanding method to solve the problem of low accuracy in intent recognition in multi-business domain scenarios.

[0007] In a first aspect, this application provides a terminal device, including: A communication device configured to communicate with at least one external device; The controller is configured as follows: Obtain the natural language text input by the user; The natural language text is input into an intent classification model to perform intent classification on the natural language text, thereby obtaining the target domain category and target device identifier corresponding to the natural language text output by the intent classification model; the target device identifier is used to characterize the target device; the target device is the terminal device or the external device; Among a plurality of preset domain adapters, a target domain adapter corresponding to the target domain category is determined, and the target domain adapter is attached to the base model; wherein, the plurality of domain adapters share the base model, and the domain adapter is a fine-tuned branch obtained by fine-tuning the parameters of the base model; the base model is a pre-trained multilingual large model; The natural language text is input into the base model that is equipped with the target domain adapter. The natural language text is then subjected to intent recognition and slot extraction to obtain the original slot information corresponding to the natural language text output by the base model. The original slot information is parameter information that characterizes the user's intent. Map the original slot information to the device instructions corresponding to the target device; If the target device represented by the target device identifier is the terminal device, then the device instruction is executed; If the target device identified by the target device identifier is the external device, the communication device is controlled to send the device instruction to the external device.

[0008] The above technical solution has the following beneficial effects or advantages: This embodiment uses an intent classification model to perform preliminary intent classification on the natural language text input by the user, outputting the target domain category and target device. Then, it matches and attaches the corresponding target domain adapter from multiple domain adapters. The base model with the target domain adapter attached performs intent recognition and slot extraction to obtain the original slot information, which is then mapped to the device command corresponding to the target device. Depending on the target device, the device command is directly executed or sent. This embodiment achieves coarse-grained classification at the domain level through the intent classification model, and fine-grained intent recognition and slot extraction within the domain through domain adapters, forming a two-level processing architecture. This solves the problem of intent confusion in multi-business domain scenarios and improves the accuracy of intent recognition. Simultaneously, multiple domain adapters share the same base model, avoiding the resource waste caused by deploying large models independently for each business domain, and reducing memory usage and computational overhead. Furthermore, by distinguishing the target device as a terminal device or an external device, it enables correct instruction routing and parallel delivery across device scenarios, meeting the multi-device linkage needs of smart homes and the Internet of Things.

[0009] In some embodiments of this application, the controller maps the original slot information to device instructions corresponding to the target device, specifically configured as follows: Determine the slot name and slot value in the original slot information; wherein, the slot name is used to represent the parameter category of the user intent, and the slot value is used to represent the specific parameter content corresponding to the parameter category, and the parameter category includes operation object or operation attribute; The slot type is determined based on the slot name and the slot value; The slot value is normalized according to the normalization path corresponding to the slot type to obtain the standard slot value; Map the slot name to the target terminal instruction field corresponding to the target device; Based on the target terminal instruction field and the standard slot value, the device instruction corresponding to the target device is generated.

[0010] The above technical solution has the following beneficial effects or advantages: The original slot information includes the slot name and slot value. The slot type can be determined based on the slot name and slot value. The slot value is then normalized according to the normalization path corresponding to the slot type to obtain a standard slot value. The slot name is then mapped to the target terminal instruction field corresponding to the target device. Finally, device instructions are generated based on the target terminal instruction field and the standard slot value. This embodiment classifies the original slot values ​​and processes them according to the corresponding normalization path, converting the inconsistent formats, mixed units, and diverse expressions in user natural language into standardized slot values. This solves the problem that the original slot values ​​cannot directly adapt to the instruction format requirements of devices from different brands and protocols, allowing the semantic understanding results to be directly adapted to specific terminal instruction formats, thus improving the system's versatility and compatibility.

[0011] In some embodiments of this application, the controller determines the slot type based on the slot name and the slot value, specifically configured as follows: If the slot name belongs to a time-related field, or the slot value matches a preset time regular expression, then the slot type is determined to be time-related. If the slot name belongs to a numeric field, or the slot value matches a preset number plus unit regular expression, then the slot type is determined to be numeric. If the slot name does not belong to either a numeric or time-related field, then the slot type is determined to be a name type.

[0012] The above technical solution has the following beneficial effects or advantages: By combining the field type of the slot name with the regular expression matching of the slot value, the slot type is divided into three categories: time type, numeric type and name type, thus realizing the automatic identification and classification of slot type.

[0013] In some embodiments of this application, the controller normalizes the slot value according to the normalization path corresponding to the slot type to obtain a standard slot value, specifically configured as follows: If the slot type is time-based, then the slot value is converted to a standard time format to obtain the standard slot value; If the slot type is numerical, then the slot value is converted into a standard numerical value and a standard unit to obtain the standard slot value; If the slot type is a name type, then the slot value is mapped to a standard entity value to obtain the standard slot value.

[0014] The above technical solution has the following beneficial effects or advantages: For three different slot types—time, numerical, and name—three differentiated normalization paths are used to process the slot values: time format conversion, numerical unit conversion, and standard entity mapping. This achieves standardized processing of different types of slot values, enabling all types of slot values ​​to undergo targeted normalization. The output standard slot values ​​have a unified data format and clear semantic meaning.

[0015] In some embodiments of this application, the controller maps the slot value to a standard entity value, specifically configured as follows: Obtain a standard entity knowledge base table; the standard entity knowledge base table includes a mapping relationship between standard entity values ​​and aliases; the aliases include at least one of multilingual expressions, abbreviations, colloquialisms, and misspellings; Search for the first alias that is the same as the slot value in the standard entity knowledge base table, and map the slot value to the standard entity value corresponding to the first alias.

[0016] The above technical solution has the following beneficial effects or advantages: By constructing a standard entity knowledge base table, various expressions that may appear in users' natural language (including multilingual expressions, abbreviations, colloquialisms, spelling errors, etc.) are uniformly mapped to standard entity values. By searching for aliases with the same slot value in the standard entity knowledge base table and mapping the slot value to the corresponding standard entity value, we can effectively handle name variations in users' spoken expressions, improve the fault tolerance and standardization of entity name recognition, and reduce entity matching failures caused by non-standard user expressions.

[0017] In some embodiments of this application, the controller is further configured to: If the first alias does not exist in the standard entity knowledge base table, then the edit distance or string similarity between the slot value and each alias in the standard entity knowledge base table is calculated, and a second alias whose edit distance is less than a preset distance or whose string similarity is greater than a preset first similarity threshold is determined. The slot value is then mapped to the standard entity value corresponding to the second alias. If the second alias does not exist in the standard entity knowledge base table, then each alias in the standard entity knowledge base table is encoded as an alias vector, the slot value is encoded as a slot vector, the cosine similarity between the slot vector and each alias vector is calculated, a third alias with a cosine similarity greater than a preset second similarity threshold is determined, and the slot value is mapped to the standard entity value corresponding to the third alias.

[0018] The above technical solution has the following beneficial effects or advantages: A three-tiered progressive entity matching mechanism is constructed, consisting of exact matching, edit distance or string similarity matching, and vector cosine similarity matching. When the first-level exact matching fails, edit distance or string similarity matching is used for fallback. If this also fails, vector retrieval is employed for a second fallback. This three-tiered matching mechanism ensures the accuracy of high-confidence matching results while effectively improving the recall rate of entity matching. It can handle complex variations such as high spelling errors and semantically similar but literally different characters, further enhancing the robustness of name slot normalization.

[0019] In some embodiments of this application, the controller is further configured to: Obtain the slot name terminal instruction mapping table; the slot name terminal instruction mapping table is indexed in three dimensions by business domain category, device type, and slot name, and stores the corresponding terminal instruction fields, field types, and value passing methods; the terminal instruction fields are used to represent the identifiable instruction parameter names corresponding to the device type; the field types are used to represent the numerical type of the terminal instruction fields; the value passing methods are used to represent the processing method of assigning the standard slot value to the terminal instruction fields; Based on the target domain category, the device type of the target device, and the slot name, a three-dimensional index retrieval is performed in the slot name terminal instruction mapping table to determine the corresponding target terminal instruction field, field type, and value transmission method. Based on the target terminal instruction field, the field type, and the value transmission method, the standard slot value is filled into the target terminal instruction field to generate the device instruction corresponding to the target device.

[0020] The above technical solution has the following beneficial effects or advantages: A slot name terminal command mapping table is constructed using a three-dimensional index of business domain category, device type, and slot name. This table stores the corresponding terminal command fields, field types, and value passing methods. During command mapping, the target terminal command fields, field types, and value passing methods are determined based on the three-dimensional index. Then, standard slot values ​​are filled into the target terminal command fields to generate device commands. This allows for flexible mapping from slot names to command fields on different devices, adapting to scenarios involving multiple business domains, device types, and slot names. Furthermore, the slot name terminal command mapping table supports hot updates; adding a new device type or business domain only requires expanding the mapping table configuration, without modifying the core parsing logic, thus improving system scalability and maintenance convenience.

[0021] In some embodiments of this application, the controller is further configured to: Obtain context information; the context information includes at least one of historical dialogues, device information, language information, and device status; A structured input is constructed based on the natural language text, the context information, and the preset prompt word template; The structured input is fed into the intent classification model to classify the natural language text into intents.

[0022] The above technical solution has the following beneficial effects or advantages: By combining historical dialogues, device information, language information, device status, and other contextual information with natural language text, and constructing structured input according to a preset prompt word template, the input is fed into the intent classification model. This multi-dimensional contextual fusion mechanism enables the intent classification model to make comprehensive judgments based on dialogue history, device attributes, language environment, and operating status, improving its ability to handle complex situations such as referential resolution, multi-turn dialogues, multilingual scenarios, and device status constraints. This makes the intent classification results more closely match the user's actual interaction scenario and reduces classification bias caused by missing contextual information.

[0023] In some embodiments of this application, the domain adapter includes a low-rank decomposition matrix, and the controller attaches the target domain adapter to the base model, specifically configured as follows: The low-rank decomposition matrix of the target domain adapter is injected into the attention layer of the base model, and the parameters of the base model are kept unchanged during the mounting process.

[0024] The above technical solution has the following beneficial effects or advantages: Domain adapters are implemented by injecting low-rank factorization matrices into the attention layer. This requires adjusting only a few parameters to achieve domain-specific intent recognition and slot extraction capabilities. The number of parameters in a single domain adapter is far less than that of the base model, reducing training costs and storage overhead. Furthermore, the base model parameters remain unchanged during the attachment process, allowing multiple domain adapters to share the same base model. Adding a new business domain only requires training and attaching the new domain adapter, without modifying the base model, enabling rapid iteration and plug-and-play functionality.

[0025] Secondly, this application provides a semantic understanding method applied to a terminal device, the terminal device being communicatively connected to at least one external device; the method includes: Obtain the natural language text input by the user; The natural language text is input into an intent classification model to perform intent classification on the natural language text, thereby obtaining the target domain category and target device identifier corresponding to the natural language text output by the intent classification model; the target device identifier is used to characterize the target device; the target device is the terminal device or the external device; Among a plurality of preset domain adapters, a target domain adapter corresponding to the target domain category is determined, and the target domain adapter is attached to the base model; wherein, the plurality of domain adapters share the base model, and the domain adapter is a fine-tuned branch obtained by fine-tuning the parameters of the base model; the base model is a pre-trained multilingual large model; The natural language text is input into the base model that is equipped with the target domain adapter. The natural language text is then subjected to intent recognition and slot extraction to obtain the original slot information corresponding to the natural language text output by the base model. The original slot information is parameter information that characterizes the user's intent. Map the original slot information to the device instructions corresponding to the target device; If the target device represented by the target device identifier is the terminal device, then the device instruction is executed; If the target device identified by the target device identifier is the external device, the communication device is controlled to send the device instruction to the external device.

[0026] The above technical solution has the following beneficial effects or advantages: This embodiment uses an intent classification model to perform preliminary intent classification on the natural language text input by the user, outputting the target domain category and target device. Then, it matches and attaches the corresponding target domain adapter from multiple domain adapters. The base model with the target domain adapter attached performs intent recognition and slot extraction to obtain the original slot information, which is then mapped to the device command corresponding to the target device. Depending on the target device, the device command is directly executed or sent. This embodiment achieves coarse-grained classification at the domain level through the intent classification model, and fine-grained intent recognition and slot extraction within the domain through domain adapters, forming a two-level processing architecture. This solves the problem of intent confusion in multi-business domain scenarios and improves the accuracy of intent recognition. Simultaneously, multiple domain adapters share the same base model, avoiding the resource waste caused by deploying large models independently for each business domain, and reducing memory usage and computational overhead. Furthermore, by distinguishing the target device as a terminal device or an external device, it enables correct instruction routing and parallel delivery across device scenarios, meeting the multi-device linkage needs of smart homes and the Internet of Things. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram illustrating the operational scenarios between a terminal device and a control device provided in some embodiments of this application; Figure 2 This is a schematic diagram of the hardware configuration of a terminal device provided in some embodiments of this application; Figure 3 This is a schematic diagram of the software configuration of a terminal device provided in some embodiments of this application; Figure 4 A flowchart illustrating the semantic understanding method provided in some embodiments of this application; Figure 5 Schematic diagram of the input and output of an intent classification model provided in some embodiments of this application; Figure 6 A schematic diagram of a domain adapter provided for some embodiments of this application; Figure 7 A schematic diagram illustrating the slot normalization and instruction mapping process provided in some embodiments of this application; Figure 8 This is a flowchart illustrating the name-type slot normalization process provided in some embodiments of this application; Figure 9A schematic diagram of the semantic understanding architecture provided for some embodiments of this application; Figure 10 Timing diagrams for semantic understanding provided in some embodiments of this application; Figure 11 This is a schematic diagram of the architecture of a terminal instruction mapping module provided in some embodiments of this application. Detailed Implementation

[0029] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0030] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0031] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0032] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0033] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0034] In this embodiment, the terminal device 200 generally refers to a device with screen display and data processing capabilities. For example, the terminal device 200 includes, but is not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0035] Figure 1 This is a schematic diagram illustrating an operational scenario between a terminal device and a control device provided in some embodiments of this application. For example... Figure 1As shown, a user can operate a terminal device 200 via touch operation, a mobile terminal 300, and a control device 100. The control device 100 receives user input commands and converts them into control commands that the terminal device 200 can recognize and respond to. For example, the control device 100 can be a remote control, a stylus, a gamepad, etc.

[0036] The mobile terminal 300 can function as a control device for human-computer interaction between the user and the terminal device 200. The mobile terminal 300 can also function as a communication device for establishing a communication connection with the terminal device 200 and exchanging data. In some embodiments, the mobile terminal 300 can install software applications with the terminal device 200 to establish a connection and communication via network communication protocols, achieving one-to-one control operations and data communication. It can also transmit audio and video content displayed on the mobile terminal 300 to the terminal device 200 for synchronized display.

[0037] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running the application of the control terminal device 200.

[0038] like Figure 1 The document also shows that the terminal device 200 communicates with the server 400 via various communication methods. The terminal device 200 can communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0039] Terminal device 200 can provide broadcast television reception function, and can also be equipped with intelligent network television function with computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0040] Figure 2 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of terminal device 200.

[0041] In some embodiments, the terminal device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface 280.

[0042] In some embodiments, detector 230 is used to acquire signals from the external environment or to interact with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0043] In some embodiments, the display 260 includes display function components for presenting images and driving components for driving image display. The display 260 is used to receive and display image signals output from the controller 250. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces, etc.

[0044] In some embodiments, the communication device 220 is a component used to communicate with external devices or the server 400 according to various communication protocol types. The terminal device 200 may have multiple communication devices 220 depending on the supported communication methods. For example, when the terminal device 200 supports wireless network communication, it may have a communication device 220 with WiFi functionality. When the terminal device 200 supports Bluetooth connection communication, it needs to have a communication device 220 with Bluetooth functionality.

[0045] The communication device 220 enables the terminal device 200 to communicate with external devices or the server 400 via wireless or wired connections. Wired connections utilize data cables, interfaces, or other components to connect the terminal device 200 to external devices. Wireless connections utilize wireless signals or wireless networks. The terminal device 200 can establish a direct connection with external devices or indirectly through gateways, routers, or other connection devices.

[0046] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first to an nth interface for input / output. The controller 250 controls the operation of the terminal device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the terminal device 200.

[0047] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0048] In some embodiments, a user can input user commands through a graphical user interface (GUI) displayed on a display 260, and the user input interface 280 receives the user input commands through the graphical user interface (GUI).

[0049] In some embodiments, the audio output device 270 can be the built-in speaker of the terminal device 200 or an external audio output device connected to the terminal device 200. For the external audio output device connected to the terminal device 200, the terminal device 200 may also be provided with an external audio output terminal, through which the audio output device can be connected to the terminal device 200 to output sound from the terminal device 200.

[0050] In some embodiments, the user input interface 280 can be used to receive instructions from user input.

[0051] In order to perform user interaction, in some embodiments, the terminal device 200 may run an operating system. The operating system is a computer program used to manage and control the hardware and software resources in the terminal device 200. The operating system can control the terminal device to provide a user interface; for example, the operating system can directly control the terminal device to provide a user interface, or it can provide a user interface by running applications. The operating system also allows users to interact with the terminal device 200.

[0052] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system that is deeply customized based on a specific operating platform, or an independent operating system specifically developed for terminal devices.

[0053] An operating system can be divided into different modules or levels based on the functions it implements, for example... Figure 3 As shown, in some embodiments, the system is divided into three layers, from top to bottom: the Applications layer (referred to as the "application layer"), the middleware layer, and the hardware layer.

[0054] In some embodiments, the application layer provides services and interfaces for applications, enabling the terminal device 200 to run applications and interact with the user based on the applications. The application layer may contain at least one application, which may be a built-in Windows program, system settings program, or clock program of the operating system; or it may be an application developed by a third-party developer. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0055] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0056] like Figure 3 As shown, Figure 3 Provided for some embodiments of this application Figure 1 The diagram shows the software configuration of the terminal device. In some embodiments, the system of the terminal device 200 can be divided into three layers, from top to bottom: the application layer, the middleware layer, and the hardware layer.

[0057] The application layer mainly includes commonly used applications on TVs, as well as the application framework. The commonly used applications are mainly browser-based applications, such as HTML5 apps, and native apps. An application framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, and the user interface for these functions (toolbar, status bar, menu, dialog box).

[0058] Native apps can support online or offline access, push notifications, or access to local resources.

[0059] The middleware layer includes various television protocols, multimedia protocols, and system components. Middleware can use the basic services (functions) provided by system software to connect different parts of application systems or different applications on the network, achieving resource sharing and function sharing.

[0060] The hardware layer mainly includes the Hardware Abstraction Layer (HAL) interface, hardware, and drivers. The HAL interface is a unified interface for all TV chips, with the specific logic implemented by each individual chip. Drivers mainly include: audio drivers, display drivers, Bluetooth drivers, camera drivers, Wi-Fi drivers, USB drivers, HDMI drivers, sensor drivers (such as fingerprint sensors, temperature sensors, pressure sensors, etc.), and power drivers.

[0061] It should be noted that the above examples are merely a simple division of operating system functions and do not limit the specific form of the operating system of the terminal device 200 in this application embodiment. Depending on the functions of the terminal device, the type of operating system, and other factors, the number of layers and the specific type of the operating system may take other forms.

[0062] Terminal device 200 supports voice interaction, allowing users to control it via natural language commands to perform various operations such as channel switching, volume adjustment, appliance control, and media asset search. Terminal device 200 needs to accurately understand the user's natural language commands and convert them into executable device commands.

[0063] Existing semantic understanding solutions typically employ a single model to process natural language instructions across all business domains. For example, a single BERT, RoBERTa, or small Seq2Seq model can be used to simultaneously perform intent recognition and slot extraction, directly outputting intent labels and slot key-value pairs.

[0064] However, in practical applications, users' natural language commands often involve multiple different business domains, such as TV control, smart home device control, and media content search. Semantic overlap and intent confusion exist between these different business domains. For example, the natural language command "turn up" might mean increasing the volume in TV control, but in air conditioner control it might mean increasing the air conditioner temperature. A single model struggles to effectively distinguish these cross-domain semantic differences, leading to a decrease in intent recognition accuracy and affecting the correct execution of user commands by the terminal device. Furthermore, when adding new business domains, training data for the new domains must be collected, and the entire model must be retrained, resulting in high iteration costs and hindering rapid business expansion.

[0065] Furthermore, when a user's natural language command involves multiple devices, such as simultaneously controlling a washing machine and a television, it is necessary to accurately identify the multiple devices involved and correctly route the generated device commands to the appropriate devices for execution. A single model, when handling such cross-device scenarios, is prone to causing command routing confusion due to low intent recognition accuracy, thus degrading the user experience.

[0066] In some embodiments, a semantic understanding model can be trained and deployed independently for each business domain. This approach can improve the accuracy of intent recognition in each domain through domain isolation. However, as the number of business domains increases, the number of models that need to be deployed in the system increases accordingly, and the memory usage and inference computing power consumption increase linearly, increasing system resource consumption. At the same time, each domain model is trained and maintained independently, and cannot share common semantic knowledge. Each business domain needs to independently label large-scale training data, resulting in low data utilization. In addition, the operational and maintenance work such as model version management, canary release, and testing in this approach increases exponentially with the number of models, resulting in high operational and maintenance complexity.

[0067] In some embodiments, a large model prompt approach can also be adopted, which uses a single general-purpose large model, such as GPT-4 or the Claude model, and implements multi-domain intent recognition and slot extraction through the Prompt project. This approach has a fast development speed and does not require separate model training for each domain. However, the inference latency of the large model is relatively high, typically between 2000-5000 milliseconds, which cannot meet the strict requirements for real-time response in scenarios such as smart homes and in-vehicle interactions. Secondly, the large model is deployed on a cloud server and relies on cloud API calls, making private deployment impossible. User data needs to be uploaded to the cloud for processing, posing a risk of data privacy leakage. Furthermore, it is greatly affected by network stability, and its availability decreases in scenarios with poor network conditions.

[0068] To address the aforementioned issues, the terminal device 200 provided in this embodiment can execute a semantic understanding method. This method performs coarse-grained domain-level classification of natural language text using an intent classification model to determine the target domain category and target device identifier. Then, it matches and mounts the target domain adapter from multiple domain adapters to a base model. The base model with the mounted target domain adapter performs fine-grained intent recognition and slot extraction within the domain to obtain original slot information. This original slot information is then mapped to device instructions corresponding to the target device. Depending on the target device, the device instructions are directly executed or sent, thereby resolving the intent confusion problem in multi-service domain scenarios and improving the accuracy of intent recognition.

[0069] The intent classification model is a classification model based on a generative large language model. It performs preliminary intent classification on user-input natural language text, outputting the target domain category and target device identifier corresponding to the natural language text. The target domain category represents the business domain involved in the natural language text, including but not limited to television control, air conditioner control, washing machine control, media asset search, and weather query. The target device identifier represents the target device, which is the device to be controlled by the natural language text. This can be the terminal device 200 itself or an external device communicatively connected to the terminal device 200.

[0070] Domain adapters are fine-tuned branches obtained by training the base model with fine-tuned parameters. They are used to enhance the intent recognition and slot extraction capabilities for specific business domains based on the base model. Multiple domain adapters share the same base model, and only the target domain adapter corresponding to the current target domain category is attached during inference, achieving domain-level parameter isolation and capability enhancement. The base model is a pre-trained multilingual large model with cross-domain general semantic understanding capabilities.

[0071] It should be noted that, in this application, the original slot information refers to the parameter information representing the user's intent output by the base model, including the slot name and slot value. The slot name is used to represent the parameter category of the user's intent, such as the operation object (channel, volume, temperature, etc.) or operation attribute (direction, amplitude, etc.). The slot value is used to represent the specific parameter content corresponding to the parameter category. The standard slot value refers to the standardized parameter content obtained after normalizing the original slot value. Device instructions refer to control instructions that can be recognized and executed by the terminal device or external device.

[0072] The semantic understanding method provided in the embodiments of this application will be described below.

[0073] The terminal device 200 capable of applying the semantic understanding method includes at least a communication device 220 and a controller 250. The communication device 220 is configured to communicate with at least one external device. The controller 250 is configured to execute specific steps of the semantic understanding method. Figure 4 The diagram shown is a flowchart of the semantic understanding method provided in this application embodiment, which specifically includes the following steps: S401, Obtain the natural language text input by the user.

[0074] During the operation of terminal device 200, users can input natural language text into terminal device 200 through various methods such as voice input and text input. For example, users can trigger the voice input function by pressing the voice button on the remote control and speak natural language commands. Terminal device 200 collects the user's voice signal through a sound acquisition device and converts the voice signal into natural language text through Automatic Speech Recognition (ASR) technology. Alternatively, users can input natural language commands in text form through an application installed on mobile terminal 300, and terminal device 200 receives the natural language text through communication device 220.

[0075] In some embodiments, the natural language text can be text in multiple languages ​​such as Chinese, English, and Japanese. The terminal device 200 supports multilingual input and can process natural language commands in different languages.

[0076] S402, input the natural language text into the intent classification model, perform intent classification on the natural language text, and obtain the target domain category and target device identifier corresponding to the natural language text output by the intent classification model.

[0077] After the terminal device 200 obtains the natural language text input by the user, it inputs the natural language text into the intent classification model. The intent classification model performs a preliminary intent classification on the natural language text to determine the target domain category and target device identifier corresponding to the natural language text.

[0078] The intent classification model is a generative large language model-based classification model. Its input is natural language text or structured input constructed from natural language text, and its output is the target domain category and target device identifier corresponding to the natural language text. The intent classification model can achieve domain classification and device identification through the Prompt project, without the need to train a separate classification model for each business domain.

[0079] The target domain category is used to represent the business domain involved in the natural language text. For example, when the natural language text is a TV volume adjustment command, the corresponding target domain category is TV control; when the natural language text is an air conditioner temperature adjustment command, the corresponding target domain category is air conditioner control; and when the natural language text is a movie playback command, the corresponding target domain category is media asset search.

[0080] The target device identifier is used to identify the target device, which is the device to be controlled by the natural language text. It can be the terminal device 200 itself or an external device that is communicatively connected to the terminal device 200. For example, if the natural language text is a TV volume adjustment command, then the target device is the terminal device 200 (smart TV); if the natural language text is an air conditioner temperature adjustment command, then the target device is an external device (air conditioner) that is communicatively connected to the terminal device 200.

[0081] In some embodiments, the intent classification model can employ the Qwen series of generative large language models, such as lightweight generative models like Qwen2.5-0.5B, Qwen2.5-1.5B, and Qwen2.5-3B. Besides Qwen, open-source generative models such as the Llama series, Gemma series, ChatGLM series, Phi series, and InternLM series can also be used. In resource-constrained scenarios, discriminative classification models such as RoBERTa, ALBERT, Electra, TextCNN, and FastText can also be employed. The model selection can be flexibly adjusted based on deployment resources, latency requirements, and accuracy requirements; this application does not limit the specific model.

[0082] In some embodiments, the intent classification model uses Prompt engineering constraints to output structured results in JSON format, containing two fields: domain category and target device identifier (target_device), facilitating parsing by downstream modules. For inference optimization, the intent classification model can employ INT8 or INT4 quantization (such as GPTQ or AWQ) and use vLLM or TensorRT-LLM inference acceleration frameworks, supporting optimization techniques such as PagedAttention, Continuous Batching, and PrefixCaching to reduce inference latency and meet real-time interaction requirements.

[0083] In some embodiments, such as Figure 5 As shown, the domain categories output by the intent classification model include media asset search, TV control, home control, general question answering, lifestyle services, semantically ambiguous categories, and other categories. While outputting the target domain category, the intent classification model also jointly infers and outputs the target device identifier based on device entities in natural language text and device context in historical word pairs. For example, if a user says "turn up the temperature," and "refrigerator" is mentioned in historical word pairs, the intent classification model outputs the target device as a refrigerator; if the current terminal device is an air conditioner, the output target device is an air conditioner. Compared to the traditional named entity recognition plus rule-based approach, generative models can better handle complex contextual scenarios such as referential resolution and omission recovery.

[0084] In some embodiments, the terminal device 200 can also acquire contextual information and combine natural language text and contextual information to construct structured input, and input the structured input into an intent classification model to improve the accuracy of intent classification. Specifically, the terminal device 200 can construct structured input based on natural language text, contextual information, and preset prompt word templates, and input the structured input into an intent classification model to classify the intent of the natural language text.

[0085] The contextual information includes, but is not limited to, historical dialogues, device information, language information, and device status. Historical dialogues are records of multiple rounds of dialogue preceding the current one, used to assist the intent classification model in resolving referential issues and understanding multi-turn dialogues. For example, if a user entered an "air conditioner turn on" command in the previous round and entered a "temperature adjustment" command in the current round, the intent classification model can determine, based on historical dialogues, that the target domain category of the current command is the air conditioner control domain, and the target device is the air conditioner.

[0086] The device information includes the device type, name, and model of the terminal device 200 and its connected external devices, which is used to assist the intent classification model in identifying the target device. For example, if the terminal device 200 is connected to multiple air conditioners, the intent classification model can determine which air conditioner the user command specifically targets based on the device information.

[0087] Language information includes the user's current language, which helps the intent classification model adapt to multilingual scenarios. For example, if the user inputs a natural language command in English, the intent classification model can adjust the command output to English format based on the language information.

[0088] Device status includes the current operating status of terminal device 200 and its connected external devices, which is used to assist the intent classification model in classifying intents under device status constraints. For example, if the TV is currently off and the user inputs a volume adjustment command, the intent classification model can determine based on the device status that the command may be directed to other devices rather than the TV.

[0089] In addition, contextual information may include other information such as sensor data and user characteristics, which can be used to assist the intent classification model in optimizing classification results by combining multi-dimensional information.

[0090] In some embodiments, the preset prompt word template includes fields such as system prompts, user input, and context information. The terminal device 200 fills the user input field with natural language text, fills the context information field with context information, assembles it into structured input according to the format of the prompt word template, and inputs the structured input into the intent classification model.

[0091] S403: Determine the target domain adapter corresponding to the target domain category from among the preset multiple domain adapters, and mount the target domain adapter to the base model.

[0092] After determining the target domain category through the intent classification model, the terminal device 200 selects the target domain adapter corresponding to the target domain category from a number of preset domain adapters and attaches the target domain adapter to the base model.

[0093] Multiple domain adapters share the same base model, with each adapter corresponding to a specific business domain. These adapters enhance the intent recognition and slot extraction capabilities of the corresponding business domain, building upon the base model. Each domain adapter is a fine-tuned branch obtained by fine-tuning the parameters of the base model, resulting in a significantly smaller number of parameters compared to the base model itself. The base model is a pre-trained, multilingual, large-scale model with cross-domain general semantic understanding capabilities.

[0094] In some embodiments, the base model can be any large open-source model such as Qwen, Llama, Gemma, ChatGLM, or Baichuan. The selection of the base model can be flexibly adjusted according to deployment resources, latency requirements, and accuracy requirements; this application does not limit the specific model.

[0095] In some embodiments, the domain adapter can be a Low-Rank Adaptation (LoRA) branch, which includes a low-rank factorization matrix. LoRA is a lightweight parameter fine-tuning technique that injects the low-rank factorization matrix only into the attention layer of the base model. During training, all parameters of the base model are frozen, and only the parameters of the low-rank matrix are updated, achieving domain-specific semantic capability adaptation with a minimal number of parameters. Specifically, domain adaptation is achieved by injecting the low-rank factorization matrix into the attention layer of the base model. The low-rank factorization matrix of the target domain adapter is injected into the attention layer of the base model, and the parameters of the base model remain unchanged during the mounting process.

[0096] The low-rank decomposition matrix consists of two low-rank matrices (a decreasing-rank matrix and an increasing-rank matrix). These two low-rank matrices are injected into the query, key, value, and output matrices of the attention layer of the base model, enabling low-rank approximate updates of the attention layer parameters. During the mounting process, all parameters of the base model remain frozen, and domain adaptation is achieved only through the parameters of the low-rank decomposition matrix, allowing multiple domain adapters to share the same base model.

[0097] Using the LoRA approach to mount domain adapters requires only a few parameter adjustments to achieve domain-specific intent recognition and slot extraction capabilities. The number of parameters in a single domain adapter is far less than that of the base model, reducing training costs and storage overhead. Furthermore, keeping the base model parameters unchanged during the mounting process allows for the addition of new business domains simply by training and mounting a new domain adapter, without modifying the base model, enabling rapid iteration and plug-and-play functionality.

[0098] In some embodiments, the rank of the low-rank factorization matrix is ​​64, the scaling factor (alpha) is 128, and the dropout rate is 0.05. The number of parameters for each domain adapter is approximately 0.1% to 1% of the base model. During training, the base model parameters are frozen, and only the parameters of the low-rank factorization matrix are trained. The learning rate can be set to 1e-5, the batch size can be set to 16, the number of training epochs can be set to 5, and the loss function is the token-level cross-entropy of slot extraction.

[0099] In some embodiments, such as Figure 6As shown, the adapters for various domains include TV control adapters, Internet of Things (IoT) control adapters, media asset content retrieval adapters, etc.

[0100] The TV control adapter is used to process TV-related intentions, such as volume adjustment, channel switching, playback control, and input source switching. For example, if the natural language text is "fast forward 5 minutes", the TV control adapter recognizes the intention as controlling forward playback (control.play.forward) and outputs the original slot information as: slot name is duration, slot value is 5 minutes, i.e., slot = {duration: "5 minutes"}.

[0101] The IoT control adapter is used to process control intentions for smart home devices such as refrigerators, air conditioners, lights, robot vacuums, and curtains. For example, if the natural language text is "set the refrigerator temperature to 4℃", the IoT control adapter recognizes the intention as setting control for the IoT device (iot.device.set) and outputs the following raw slot information: the slot name is the device name (device_name), and the slot value is "refrigerator"; the slot name is the working state (work_state), and the slot value is the temperature; the slot name is the target value (final_value), and the slot value is 4℃, i.e., slot = {device_name: "refrigerator", work_state: "temperature", final_value: "4℃"}.

[0102] The media asset content retrieval adapter is used to process media asset search and playback intents, such as movies, music, news, variety shows, etc. For example, if the natural language text is "play movie A", the media asset content retrieval adapter recognizes the intent as video playback (video.play) and outputs the original slot information as: the slot name is the movie title (video_name), and the slot value is movie A, that is, slot = {video_name: "movie A"}.

[0103] It should be noted that each domain adapter outputs domain-specific raw slot information (slot name + slot value). The slot value is raw natural language text, which needs to be standardized by the slot normalization module.

[0104] In some embodiments, the terminal device 200 also supports multi-LoRA dynamic switching (LoRA Hot-Swap), meaning that the same base model can serve multiple business domains simultaneously. When the intent classification model outputs multiple target domain categories, the terminal device 200 can simultaneously attach multiple target domain adapters to the base model. Each domain adapter runs in parallel, performing intent recognition and slot extraction on the corresponding domain portion of the natural language text, thereby achieving parallel processing of multiple intents across devices.

[0105] S404: Input natural language text into a base model equipped with a target domain adapter, perform intent recognition and slot extraction on the natural language text, and obtain the original slot information corresponding to the natural language text output by the base model.

[0106] After the terminal device 200 attaches the target domain adapter to the base model, it inputs natural language text into the base model with the target domain adapter attached. With the assistance of the target domain adapter, the base model performs fine-grained intent recognition and slot extraction within the domain, and obtains the original slot information corresponding to the natural language text output by the base model.

[0107] The original slot information consists of parameter information representing user intent, including at least one slot. Each slot includes a slot name and a slot value. The slot name represents the parameter category of the user intent, such as the operation object (channel, volume, temperature, etc.) or operation attribute (direction, amplitude, etc.). The slot value represents the specific parameter content corresponding to the parameter category.

[0108] For example, when the natural language text is a TV volume adjustment command, the original slot information obtained after intent recognition and slot extraction may include: slot name is the operation object, slot value is volume; slot name is the operation direction, slot value is large; slot name is the operation amplitude, slot value is one point.

[0109] For example, when the natural language text is an air conditioner temperature adjustment command, after intent recognition and slot extraction, the original slot information obtained may include: the slot name is the operation object, and the slot value is temperature; the slot name is the operation direction, and the slot value is high; the slot name is the operation range, and the slot value is 2 degrees.

[0110] S405 maps the original slot information to the device command corresponding to the target device.

[0111] After obtaining the original slot information, the terminal device 200 maps the original slot information to the device command corresponding to the target device. Since different devices have different command formats, parameter names, parameter types, etc., the terminal device 200 needs to normalize the original slot information and map the command fields to generate device commands that the target device can recognize.

[0112] like Figure 7 The diagram shown is a flowchart illustrating the slot normalization and instruction mapping process provided in this embodiment of the application, specifically including the following steps: S701, determine the slot name and slot value in the original slot information.

[0113] The terminal device 200 determines the slot name and slot value for each slot from the raw slot information output by the base model. The slot name represents the parameter category of the user's intent, including the operation object or operation attribute, etc. The slot value represents the specific parameter content corresponding to the parameter category.

[0114] S702, determine the slot type based on the slot name and slot value.

[0115] Terminal device 200 determines the slot type based on the field type of the slot name and the characteristics of the slot value. Slot types include three categories: time-based, numeric, and name-based.

[0116] Specifically, if the slot name belongs to a time-related field, or the slot value matches a preset time regular expression, then the slot type is determined to be time-related. Time-related fields include, but are not limited to, time, date, and duration. Time regular expressions are used to match various time expression formats, such as 3 PM, tomorrow, 2 hours later, etc.

[0117] If the slot name is a numeric field, or the slot value matches a preset regular expression for a number plus a unit, then the slot type is determined to be numeric. Numeric fields include, but are not limited to, temperature, volume, and brightness. The regular expression for a number plus a unit is used to match combinations of numbers and units, such as 2 degrees, 30 degrees, 50%, etc.

[0118] If the slot name is neither a numeric nor a time-related field, then the slot type is determined to be name-based. Name-based slots typically include entity names such as device name, channel name, application name, and media asset name.

[0119] By combining the field type of the slot name with the regular expression matching of the slot value, the slot type is divided into three categories: time type, numeric type and name type, thus realizing the automatic identification and classification of slot type.

[0120] S703, normalizes the slot value according to the normalization path corresponding to the slot type to obtain the standard slot value.

[0121] After determining the slot type, the terminal device 200 normalizes the slot value according to the normalization path corresponding to the slot type to obtain the standard slot value. Different slot types correspond to different normalization paths.

[0122] Specifically, if the slot type is time-based, the slot value is converted to a standard time format to obtain a standard slot value. For example, a slot value of 3 PM is converted to the standard time format 15:00; a slot value of tomorrow is converted to a standard date format; and a slot value of 2 hours later is calculated based on the current time to obtain a standard timestamp.

[0123] In some embodiments, the normalization of time-related slot values ​​can be implemented through a time transformation module. This module employs a rule engine and context resolution, independent of the large model, ensuring determinism and high accuracy in time transformation. The time transformation module includes a time lexical analyzer, a contextual time base component, and a time format standardization component. The time lexical analyzer uses a combination of regular expressions and a time dictionary to identify time entities from natural language text and classifies them into five categories: absolute time (e.g., January 15, 2024, 15:30, 3 PM, March 8), relative time (e.g., tomorrow, next week, next month, three days later, two hours later), fuzzy time (e.g., morning, noon, afternoon, evening, midnight), duration (e.g., 5 minutes, 2 hours, one and a half hours, half an hour), and period (e.g., daily, Monday, weekday, weekend).

[0124] The context time reference component uses the current system time (including time zone) as the base, and combines the device's time zone information and time context information from historical conversations to resolve relative and omitted times. For example, if the current system time is 10:30 AM on January 14, 2024, and the user enters a relative time expression for 3 PM tomorrow, the date offset will be calculated based on the current system time, and after resolution using the context time reference, the corresponding standard time result will be: 3 PM on January 15, 2024.

[0125] The time format standardization component is the output stage of the time-related slot normalization process, used to convert the identified and resolved time entities into a unified standard output format. For example, for absolute time representations, the ISO 8601 format can be used, and for duration representations, it can be converted into integer values ​​in seconds.

[0126] In some embodiments, the time conversion module may be replaced by a rules engine, a large model Few-shot inference, or a third-party time parsing library (such as duckling).

[0127] If the slot type is numerical, the slot value is converted to a standard value and a standard unit to obtain a standard slot value. For example, a slot value of 2 degrees is converted to the standard value 2 and the standard unit Celsius; a slot value of 30 degrees is converted to the standard value 30 and the standard unit Celsius; a slot value that is slightly larger is converted to the standard value 1 and the corresponding standard unit according to a preset amplitude mapping table.

[0128] In some embodiments, the normalization of numerical slot values ​​can be achieved using a LoRA fine-tuning model. Specifically, the terminal device 200 can train a dedicated LoRA fine-tuning model based on a lightweight generative large model (such as Qwen2.5-0.5B). The LoRA fine-tuning model integrates the four steps of numerical extraction, unit identification, unit conversion, and fuzzy quantifier resolution, directly outputting structured standard slot values. The training data consists of manually labeled pairs of original and standard slot values, covering various physical quantity types such as time, temperature, brightness, volume, length, weight, percentage, and fuzzy quantifiers. The input to the LoRA fine-tuning model is the original slot value, slot name, and device type (used to assist in the context resolution of fuzzy quantifiers). The output is a structured standard slot value in JSON format, containing two fields: value and standard unit.

[0129] For example, given an initial slot value of 5min and a slot name of duration, the LoRA fine-tuning model outputs the standard slot value as: {"value": 5, "unit": "min"}. Using the LoRA fine-tuning model to uniformly process numerical slot values ​​can merge multiple steps into a single step, simplifying the processing flow and improving the ability to resolve and generalize fuzzy quantifiers.

[0130] In some embodiments, the LoRA fine-tuning model can be replaced by a rule engine (regular expression + unit conversion table) or large model Few-shot inference.

[0131] If the slot type is name-based, the slot value is mapped to a standard entity value to obtain the standard slot value. The normalization of name-based slot values ​​is implemented through a standard entity knowledge base table, as described below.

[0132] For three different slot types—time, numerical, and name—three differentiated normalization paths are used to process the slot values: time format conversion, numerical unit conversion, and standard entity mapping. This achieves standardized processing of different types of slot values, enabling all types of slot values ​​to undergo targeted normalization. The output standard slot values ​​have a unified data format and clear semantic meaning.

[0133] like Figure 8 The diagram shown is a flowchart illustrating the name-based slot normalization process provided in this application embodiment, which specifically includes the following steps: S801, retrieve the standard entity knowledge base table.

[0134] Terminal device 200 acquires a preset standard entity knowledge base table. This table includes a mapping between standard entity values ​​and aliases. Aliases include at least one of the following: multilingual expressions, abbreviations, colloquialisms, and misspellings.

[0135] For example, the standard entity value for channel TV1 is TV1, and its aliases can include various forms such as CCTV1, CCTV-1, CCTV-1, tv1, TV1, etc.

[0136] In some embodiments, the construction and maintenance of the standard entity knowledge base table includes four stages: initial construction, alias expansion, multilingual coverage, and hot updating. In the initial construction stage, the terminal device 200 can import standard entity values ​​and their official names from official app stores, media asset libraries, and device registry entries. In the alias expansion stage, the terminal device 200 can continuously expand the alias set through user log mining (statistical analysis of high-frequency miss natural language text), crowdsourced annotation, and large model-generated aliases. In the multilingual coverage stage, the terminal device 200 maintains aliases in multiple languages ​​such as Chinese, English, Japanese, and Korean for each standard entity value. In the hot updating stage, the standard entity knowledge base table supports hot updating; adding aliases does not require restarting the service, and the terminal device 200 can update the standard entity knowledge base table in real time based on user feedback and business changes, maintaining the timeliness and accuracy of the standard entity knowledge base table.

[0137] S802, search for the first alias that is the same as the slot value in the standard entity knowledge base table, and map the slot value to the standard entity value corresponding to the first alias.

[0138] Terminal device 200 searches for an alias (i.e., the first alias) that is exactly the same as the slot value in the standard entity knowledge base table. If the first alias is found, the slot value is mapped to the standard entity value corresponding to the first alias.

[0139] For example, if the slot value is CCTV1, and the standard entity value corresponding to the alias CCTV1 is found in the standard entity knowledge base table, then the slot value CCTV1 will be mapped to the standard entity value TV1.

[0140] S803, if the first alias does not exist in the standard entity knowledge base table, calculate the edit distance or string similarity between the slot value and each alias in the standard entity knowledge base table, determine the second alias whose edit distance is less than the preset distance or whose string similarity is greater than the preset first similarity threshold, and map the slot value to the standard entity value corresponding to the second alias.

[0141] If no alias exactly matching the slot value is found in the standard entity knowledge base table (i.e., the first alias does not exist), the terminal device 200 performs a rollback process using edit distance or string similarity matching. Specifically, it calculates the edit distance or string similarity between the slot value and each alias in the standard entity knowledge base table, determines the alias whose edit distance is less than a preset distance or whose string similarity is greater than a preset first similarity threshold (i.e., the second alias), and maps the slot value to the standard entity value corresponding to the second alias.

[0142] The Levenshtein Distance refers to the minimum number of edit operations (insertion, deletion, substitution) required to convert two strings into the same string. String similarity can be calculated in various ways, such as Jaccard similarity, Jaro-Winkler similarity, etc.

[0143] For example, the slot value is Yangshi 1 (with a spelling error), and no exactly identical alias is found in the standard entity knowledge base table. The Levenshtein Distance between Yangshi 1 and each alias is calculated, and it is found that the Levenshtein Distance between Yangshi 1 and CCTV 1 is 1 (substituting "shi" (city) with "shi" (vision)), which is less than the preset distance 2. Then the second alias is determined as CCTV 1, and the slot value Yangshi 1 is mapped to the standard entity value TV1.

[0144] S804: If the second alias does not exist in the standard entity knowledge base table, each alias in the standard entity knowledge base table is encoded as an alias vector, the slot value is encoded as a slot vector, the cosine similarity between the slot vector and each alias vector is calculated, a third alias whose cosine similarity is greater than a preset second similarity threshold is determined, and the slot value is mapped to the standard entity value corresponding to the third alias.

[0145] If no match is obtained through Levenshtein Distance or string similarity matching (that is, the second alias does not exist), the terminal device 200 performs secondary fallback processing by using vector retrieval. Specifically, each alias in the standard entity knowledge base table is encoded as an alias vector, the slot value is encoded as a slot vector, the cosine similarity between the slot vector and each alias vector is calculated, an alias whose cosine similarity is greater than the preset second similarity threshold (that is, the third alias) is determined, and the slot value is mapped to the standard entity value corresponding to the third alias.

[0146] The alias vector and the slot vector can be obtained by encoding through a pre-trained language vector model (such as bge-m3, multilingual-e5, text2vec-multilingual, etc.). Cosine similarity is used to measure the directional similarity between two vectors, with a value range from -1 to 1. The closer the value is to 1, the more similar the two vectors are.

[0147] For example, the slot value is Zhongshi Yitao (with a relatively special expression), and no match is obtained through Levenshtein Distance matching. Zhongshi Yitao is encoded as a slot vector, each alias is encoded as an alias vector, and the cosine similarity is calculated. It is found that the cosine similarity between Zhongshi Yitao and CCTV-1 is 0.92, which is greater than the preset second similarity threshold 0.85. Then the third alias is determined as CCTV-1, and the slot value Zhongshi Yitao is mapped to the standard entity value TV1.

[0148] It should be noted that the standard entity knowledge base table can be used alone or combined with methods such as vector retrieval, edit distance, pinyin / transliteration matching, large model Few-shot reasoning, and knowledge graph reasoning to form a multi-level fallback strategy. The combination order and thresholds of each method can be adjusted based on actual needs.

[0149] S704 maps the slot name to the target terminal instruction field corresponding to the target device.

[0150] After normalizing the slot values ​​to obtain standard slot values, the terminal device 200 maps the slot names to the target terminal instruction fields corresponding to the target device. Different devices may use different instruction field names for the same parameter category, so mapping is required based on the device type of the target device.

[0151] In some embodiments, the terminal device 200 has a pre-defined slot name terminal instruction mapping table. The mapping table uses a three-dimensional index of business domain category, device type and slot name to store the corresponding terminal instruction fields, field types and value passing methods.

[0152] The terminal instruction field is used to characterize the identifiable instruction parameter name corresponding to the device type. For example, for the slot name "volume", the terminal instruction field for a smart TV can be "volume", while the terminal instruction field for an air conditioner can be "fan_speed" (where "volume" represents the fan speed in the air conditioning field).

[0153] Field types are used to characterize the numeric type of terminal command fields, such as integer (int), floating-point (float), string (string), boolean (boolean), etc.

[0154] The value passing method is used to characterize the processing method of assigning standard slot values ​​to terminal command fields, including direct assignment, enumeration mapping, and range truncation. For example, for volume control, the standard slot value is the integer 55, and the value passing method is direct assignment; for air conditioning mode, the standard slot value is cooling, and the value passing method is enumeration mapping, which maps cooling to an enumeration value that the device can recognize.

[0155] For example, as shown in Table 1 below, this is a slot name terminal instruction mapping table provided in an embodiment of this application: Table 1

[0156] Terminal device 200 performs a three-dimensional index search in the slot name terminal instruction mapping table based on the target domain category, target device type and slot name to determine the corresponding target terminal instruction field, field type and value transmission method.

[0157] S705 generates device instructions corresponding to the target device based on the target terminal instruction field and standard slot value.

[0158] After determining the target terminal instruction field, field type, and value transmission method, the terminal device 200 fills the standard slot value into the target terminal instruction field based on this information to generate the device instruction corresponding to the target device.

[0159] Specifically, the terminal device 200 converts the standard slot value into the data format corresponding to the field type according to the value transmission method, and fills the converted standard slot value into the target terminal instruction field to generate a device instruction containing the target terminal instruction field and the standard slot value.

[0160] For example, for a TV volume adjustment command, the target domain category is determined to be TV control, and the target device is terminal device 200 (smart TV) after intent classification. The original slot information is obtained through slot extraction: the slot name is the operation object and the slot value is volume; the slot name is the operation direction and the slot value is large; the slot name is the operation amplitude and the slot value is one point. After slot normalization, volume is determined to be a name class, mapped to the standard entity value `volume`; large and one point are determined to be value classes, mapped to the standard value 1 and the standard unit `step`. In the slot name terminal command mapping table, using TV control, smart TV, and operation object as a three-dimensional index search, the target terminal command field is determined to be `volume`, the field type is `int`, and the value is passed incrementally. A device command is generated: the `volume` field value is incremented by 1.

[0161] In some embodiments, during the process of mapping the original slot information to the device instructions corresponding to the target device, the terminal device 200 may detect whether there are empty slots or fuzzy quantifier slots in the original slot information. If empty slots or fuzzy quantifier slots are detected in the original slot information, corresponding default values ​​are assigned to the empty slots or fuzzy quantifier slots according to the preset default value rules, and the default values ​​are used as standard slot values ​​in subsequent instruction mapping.

[0162] Among them, empty slots refer to slots with empty or missing values, while fuzzy quantifier slots refer to slots with values ​​that are vaguely described as "a little larger," "a little smaller," or "slightly larger." Default value rules include the correspondence between slot names and default values. For example, for empty slots in volume control, the default value can be set to 20; for empty slots in temperature control, the default value can be set to 1. By using default values ​​to handle empty and fuzzy quantifiers, it is ensured that executable device commands can still be generated even when the user does not explicitly specify parameters. This avoids command generation failures due to missing or fuzzy slot values, improving the system's fault tolerance and user experience.

[0163] S406, if the target device identified by the target device identifier is a terminal device, then execute the device instruction.

[0164] After generating a device instruction, terminal device 200 executes the corresponding operation based on the target device. If the target device is terminal device 200 itself, terminal device 200 directly executes the device instruction. For example, if the target device is a smart TV and the device instruction is a volume increase instruction, then terminal device 200 will increase the volume by the corresponding amount.

[0165] S407, if the target device identified by the target device identifier is an external device, the control communication device sends a device command to the external device.

[0166] If the target device is an external device that is communicatively connected to the terminal device 200, the terminal device 200 controls the communication device 220 to send a device command to the external device. For example, if the target device is an air conditioner and the device command is a temperature setting command, the terminal device 200 sends the device command to the air conditioner through the communication device 220 (such as a WiFi module, Bluetooth module, etc.), and the air conditioner performs the corresponding temperature adjustment operation after receiving the device command.

[0167] To more clearly illustrate the semantic understanding method provided in the embodiments of this application, the following description is provided in conjunction with specific scenarios.

[0168] Scenario 1: Single device control.

[0169] The user inputs the natural language text "Set the refrigerator temperature to 4℃". First, the terminal device 200 classifies the natural language text using an intent classification model, outputting the target domain category as home control and the target device as refrigerator. Then, the terminal device 200 identifies the IoT control adapter as the target domain adapter among multiple domain adapters and attaches the IoT control adapter to the base model. The base model with the attached IoT control adapter performs intent recognition and slot extraction on the natural language text, outputting the intent result as IoT device setting (iot.device.set). The original slot information is: slot name: device name (device_name), slot value: refrigerator; slot name: work_state (work_state), slot value: temperature; slot name: final_value (final_value), slot value: 4℃. That is, the original slot information is: {device_name: "refrigerator", work_state: "temperature", final_value: "4℃"}.

[0170] Then, the terminal device 200 normalizes the original slot information. Specifically, "refrigerator" is a name class, mapped to the standard entity value "refrigerator" through a lookup table in the standard entity knowledge base; "temperature" is a name class, mapped to the standard entity value "temperature" through a lookup table; and "4℃" is a value class, converted to the standard value 4 and the standard unit Celsius through a LoRA fine-tuning model. That is, the normalized slot information is: {device_name: "refrigerator", work_state: "temperature", final_value: 4}.

[0171] Terminal device 200 retrieves the terminal instruction field `temp_func` from the three-dimensional index of the slot name terminal instruction mapping table, with the business domain being home control, the target device being a refrigerator, and the slot name `work_state` being `temperature`. It then assigns the value `temp_set` to `temp_func`, i.e., `temp_func="temp_set"`. Similarly, it retrieves the terminal instruction field `temp_value` from the three-dimensional index of the slot name terminal instruction mapping table, with the business domain being home control, the target device being a refrigerator, and the slot name `final_value` being 4. The final generated device instruction is: `{"device_id":"fridge", "temp_func":"temp_set", "temp_value":4}`.

[0172] Since the target device is an external device (refrigerator), the terminal device 200 sends a device command to the refrigerator through the communication device 220, and the refrigerator performs a temperature adjustment operation after receiving the command.

[0173] Scenario 2: Multiple intents across devices.

[0174] The user inputs the natural language text "Set the washing machine to dry cleaning mode, and play movie A on the TV". First, the terminal device 200 classifies the natural language text using an intent classification model, outputting two target domain categories and a target device identifier: the first target domain category is appliance control, and the target device is the washing machine; the second target domain category is media asset search, and the target device is the TV. The terminal device 200 can simultaneously attach both an appliance control adapter and a media asset retrieval adapter to the base model, with both adapters running in parallel. The appliance control adapter outputs the intent result as IoT device settings (iot.device.on), with the original slot information as: slot name "Working State" and slot value "Dry Cleaning Mode", slot name "Action" and slot value "On", i.e., {work_state: "dry-cleaning", "action":"on"}; the media asset retrieval adapter outputs the intent result as video playback (video.play), with the original slot information as: slot name "Movie Name" and slot value "Movie A", slot name "Action" and slot value "Video Playback", i.e., {video_name: "Movie A", "action":"video.play"}.

[0175] Then, the terminal device 200 normalizes the two sets of original slot information respectively. The normalized slot information is as follows: {work_state: "dry_cleaning_mode", "action":"on"}; {video_name: "movie A", "action":"video.play"}.

[0176] Terminal device 200 performs instruction mapping on the two sets of slot information respectively. For the first set of slot information, the terminal instruction field wash_mode is mapped and assigned the value mode_07, and the terminal instruction field apparatus_op is mapped and assigned the value on. The device instruction instructs the washing machine to switch to dry cleaning mode, i.e. {"device_id":"washer", "appliance_op":"on", "wash_mode":"mode_07"}. For the second set of slot information, the terminal instruction field media_name is mapped and assigned the value movie_001, and the terminal instruction field media_op is mapped and assigned the value play, so as to generate a device instruction instructing the TV to play movie A, i.e. {"device_id":"tv_living_room_01", "media_op":"play", "media_name":"movie_001"}.

[0177] Terminal device 200 sends a dry cleaning mode command to the washing machine through communication device 220, and at the same time directly executes the TV playback command, realizing cross-device parallel control.

[0178] Scenario 3: Fuzzy semantic understanding.

[0179] The user inputs the natural language text "The volume is too loud". The intent classification model outputs the target domain category as home control, and the target device as the TV. After attaching the home control adapter, the base model outputs the intent result as volume down (control.volume.down). The original slot information is: slot name is operation direction and slot value is small, slot name is adjustment range and slot value is empty. Since the user did not explicitly specify a specific adjustment value, the slot value is empty. During the normalization process, the terminal device 200 detects that the slot value for adjustment range is empty or an ambiguous quantifier, and assigns it a default value of 20 according to the preset default value rule. After normalization, the terminal instruction field volume_func is mapped and assigned the value volume_down, and the terminal instruction field value is mapped and assigned the value 20, to generate a device instruction to reduce the TV volume by 20, i.e., {"device_id":"tv", "volume_func":"volume_down", "value":20}. Since the target device is the terminal device 200 itself, the terminal device 200 directly executes the device instruction to reduce the TV volume by 20.

[0180] like Figure 9 , Figure 10 As shown, Figure 9 This is a schematic diagram of the semantic understanding architecture provided in the embodiments of this application. Figure 10 This is a timing diagram of semantic understanding provided in an embodiment of this application. The architecture, from top to bottom, includes an intent classification module, a domain intent slot extraction module, and a terminal command mapping module.

[0181] The architecture's input information includes multi-dimensional information such as device information, user query text, historical dialogue records, language information, and device status or sensor features. The above input information is first sent to the intent classification module, which includes a domain classification unit and a target device identification unit, which are used to output the target domain category and target device identifier, respectively, to complete the global routing classification.

[0182] The domain intent slot extraction module is located downstream of the intent classification module. The domain intent slot extraction module includes a base model and multiple domain adapters. Each domain adapter is an independent LoRA fine-tuning branch, including TV control adapter, IoT control adapter, media asset content retrieval adapter, etc. All domain adapters share the same base model. The corresponding target domain adapter is dynamically loaded according to the target domain category output by the intent classification module. The base model and the target domain adapter work together to complete intent recognition and slot extraction.

[0183] The terminal instruction mapping module is located downstream of the domain intent slot extraction module. It includes a slot value normalization unit and a terminal instruction mapping unit. The slot value normalization unit is used to standardize the extracted raw slot values. The terminal instruction mapping unit is used to determine the target device based on the target device identifier output by the intent classification module, convert the normalized slot information into instruction fields that the target device can recognize, and finally output the device instruction that the terminal can execute and send it to the corresponding device for execution.

[0184] like Figure 11 The diagram shown is an architectural schematic of the terminal command mapping module provided in an embodiment of this application. The terminal command mapping module receives raw slot information output from the domain intent slot extraction module. The raw slot information is input in the form of slot name and slot value key-value pairs.

[0185] The terminal command mapping module includes a slot value normalization unit and a terminal command mapping unit. Raw slot information is first sent to the slot value normalization unit, which is divided into time-based processing units, numerical processing units, and name-based processing units according to slot type. The time-based processing unit is equipped with a time conversion module to resolve and standardize the formats of various time representations. The numerical processing unit outputs structured and standardized numerical values ​​based on various domain adapters. The name-based processing unit combines a multi-level matching mechanism of precise matching from the knowledge base table and semantic similarity retrieval to achieve name entity normalization.

[0186] After the standard slot information has been normalized, it is sent to the downstream terminal instruction mapping unit. The terminal instruction mapping unit configures the slot name terminal instruction mapping table, and converts the standardized slot information into instruction fields that can be recognized by the target device through the mapping relationship. Finally, it outputs structured parameters consisting of terminal instruction fields and standardized values, so that the terminal can generate executable control instructions.

[0187] Based on the aforementioned terminal device 200, this application embodiment also provides a semantic understanding method, which may include the following steps: S401, Obtain the natural language text input by the user.

[0188] S402, input the natural language text into the intent classification model, perform intent classification on the natural language text, and obtain the target domain category and target device identifier corresponding to the natural language text output by the intent classification model; the target device identifier is used to represent the target device, which is the terminal device 200 or an external device.

[0189] S403, determine the target domain adapter corresponding to the target domain category from multiple preset domain adapters, and attach the target domain adapter to the base model; wherein, multiple domain adapters share the base model, the domain adapter is a fine-tuned branch obtained by fine-tuning the parameters of the base model; the base model is a pre-trained multilingual large model.

[0190] S404, input the natural language text into the base model with the target domain adapter, perform intent recognition and slot extraction on the natural language text, and obtain the original slot information corresponding to the natural language text output by the base model; the original slot information is parameter information representing the user's intent.

[0191] S405 maps the original slot information to the device command corresponding to the target device.

[0192] S406, if the target device identified by the target device identifier is a terminal device, then execute the device instruction.

[0193] S407, if the target device identified by the target device identifier is an external device, the control communication device sends a device command to the external device.

[0194] The same or similar parts among the various embodiments in this specification can be referred to mutually, and will not be repeated here.

[0195] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0196] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

Claims

1. A terminal device, characterized in that, include: A communication device configured to communicate with at least one external device; The controller is configured as follows: Obtain the natural language text input by the user; The natural language text is input into the intent classification model, and the intent classification is performed on the natural language text to obtain the target domain category and target device identifier corresponding to the natural language text output by the intent classification model. The target device identifier is used to identify the target device; The target device is either the terminal device or the external device; Among a plurality of preset domain adapters, a target domain adapter corresponding to the target domain category is determined, and the target domain adapter is attached to the base model; wherein, the plurality of domain adapters share the base model, and the domain adapter is a fine-tuned branch obtained by fine-tuning the parameters of the base model; the base model is a pre-trained multilingual large model; The natural language text is input into the base model that is equipped with the target domain adapter. The natural language text is then subjected to intent recognition and slot extraction to obtain the original slot information corresponding to the natural language text output by the base model. The original slot information is parameter information that characterizes the user's intent. Map the original slot information to the device instructions corresponding to the target device; If the target device represented by the target device identifier is the terminal device, then the device instruction is executed; If the target device identified by the target device identifier is the external device, the communication device is controlled to send the device instruction to the external device.

2. The terminal device according to claim 1, characterized in that, The controller maps the original slot information to the device command corresponding to the target device, specifically configured as follows: Determine the slot name and slot value in the original slot information; wherein, the slot name is used to represent the parameter category of the user intent, and the slot value is used to represent the specific parameter content corresponding to the parameter category, and the parameter category includes operation object or operation attribute; The slot type is determined based on the slot name and the slot value; The slot value is normalized according to the normalization path corresponding to the slot type to obtain the standard slot value; Map the slot name to the target terminal instruction field corresponding to the target device; Based on the target terminal instruction field and the standard slot value, the device instruction corresponding to the target device is generated.

3. The terminal device according to claim 2, characterized in that, The controller determines the slot type based on the slot name and the slot value, specifically configured as follows: If the slot name belongs to a time-related field, or the slot value matches a preset time regular expression, then the slot type is determined to be time-related. If the slot name belongs to a numeric field, or the slot value matches a preset number plus unit regular expression, then the slot type is determined to be numeric. If the slot name does not belong to either a numeric or time-related field, then the slot type is determined to be a name type.

4. The terminal device according to claim 2, characterized in that, The controller normalizes the slot value according to the normalization path corresponding to the slot type to obtain a standard slot value, specifically configured as follows: If the slot type is time-based, then the slot value is converted to a standard time format to obtain the standard slot value; If the slot type is numerical, then the slot value is converted into a standard numerical value and a standard unit to obtain the standard slot value; If the slot type is a name type, then the slot value is mapped to a standard entity value to obtain the standard slot value.

5. The terminal device according to claim 4, characterized in that, The controller maps the slot value to a standard entity value, specifically configured as follows: Obtain a standard entity knowledge base table; the standard entity knowledge base table includes a mapping relationship between standard entity values ​​and aliases; the aliases include at least one of multilingual expressions, abbreviations, colloquialisms, and misspellings; Search for the first alias that is the same as the slot value in the standard entity knowledge base table, and map the slot value to the standard entity value corresponding to the first alias.

6. The terminal device according to claim 5, characterized in that, The controller is also configured to: If the first alias does not exist in the standard entity knowledge base table, then the edit distance or string similarity between the slot value and each alias in the standard entity knowledge base table is calculated, and a second alias whose edit distance is less than a preset distance or whose string similarity is greater than a preset first similarity threshold is determined. The slot value is then mapped to the standard entity value corresponding to the second alias. If the second alias does not exist in the standard entity knowledge base table, then each alias in the standard entity knowledge base table is encoded as an alias vector, the slot value is encoded as a slot vector, the cosine similarity between the slot vector and each alias vector is calculated, a third alias with a cosine similarity greater than a preset second similarity threshold is determined, and the slot value is mapped to the standard entity value corresponding to the third alias.

7. The terminal device according to claim 2, characterized in that, The controller is also configured to: Obtain the slot name terminal instruction mapping table; the slot name terminal instruction mapping table is indexed in three dimensions by business domain category, device type, and slot name, and stores the corresponding terminal instruction fields, field types, and value passing methods; the terminal instruction fields are used to represent the identifiable instruction parameter names corresponding to the device type; the field types are used to represent the numerical type of the terminal instruction fields; the value passing methods are used to represent the processing method of assigning the standard slot value to the terminal instruction fields; Based on the target domain category, the device type of the target device, and the slot name, a three-dimensional index retrieval is performed in the slot name terminal instruction mapping table to determine the corresponding target terminal instruction field, field type, and value transmission method. Based on the target terminal instruction field, the field type, and the value transmission method, the standard slot value is filled into the target terminal instruction field to generate the device instruction corresponding to the target device.

8. The terminal device according to claim 1, characterized in that, The controller is also configured to: Obtain context information; the context information includes at least one of historical dialogues, device information, language information, and device status; A structured input is constructed based on the natural language text, the context information, and the preset prompt word template; The structured input is fed into the intent classification model to classify the natural language text into intents.

9. The terminal device according to claim 1, characterized in that, The domain adapter includes a low-rank decomposition matrix, and the controller attaches the target domain adapter to the base model, specifically configured as follows: The low-rank decomposition matrix of the target domain adapter is injected into the attention layer of the base model, and the parameters of the base model are kept unchanged during the mounting process.

10. A semantic understanding method, characterized in that, The method is applied to a terminal device, which is communicatively connected to at least one external device; the method includes: Obtain the natural language text input by the user; The natural language text is input into an intent classification model to perform intent classification on the natural language text, thereby obtaining the target domain category and target device identifier corresponding to the natural language text output by the intent classification model; the target device identifier is used to characterize the target device; the target device is the terminal device or the external device; Among a plurality of preset domain adapters, a target domain adapter corresponding to the target domain category is determined, and the target domain adapter is attached to the base model; wherein, the plurality of domain adapters share the base model, and the domain adapter is a fine-tuned branch obtained by fine-tuning the parameters of the base model; the base model is a pre-trained multilingual large model; The natural language text is input into the base model that is equipped with the target domain adapter. The natural language text is then subjected to intent recognition and slot extraction to obtain the original slot information corresponding to the natural language text output by the base model. The original slot information is parameter information that characterizes the user's intent. Map the original slot information to the device instructions corresponding to the target device; If the target device represented by the target device identifier is the terminal device, then the device instruction is executed; If the target device identified by the target device identifier is the external device, the communication device is controlled to send the device instruction to the external device.

Images