Natural language command control system and method based on artificial intelligence and intelligent glasses

Abstract

The invention discloses a natural language command control system and method based on artificial intelligence and intelligent glasses, the system comprises an intelligent glasses system and a model server, and the model server is configured with a generative artificial intelligence large language model; the intelligent glasses system is used for acquiring first user voice, performing semantic analysis on the first user voice, generating at least one piece of first prompt information according to analyzed semantics, and sending the at least one piece of first prompt information to the model server; the model server is used for obtaining at least one task execution command through the generative artificial intelligence large language model according to the at least one piece of first prompt information and sending the task execution command to the intelligent glasses system; the intelligent glasses system is also used for executing at least one operation corresponding to the at least one task execution command. According to the invention, the convenience of equipment control based on the intelligent glasses system can be improved, and the intelligence and interactivity of the intelligent glasses can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of smart glasses technology, and in particular to an artificial intelligence-based natural language command control system, method, and smart glasses. Background Art

[0002] With the development of computer technology, smart glasses are becoming more and more popular. However, existing smart glasses are expensive, and in addition to their own functions as smart glasses, they usually only have the functions of listening to music and making or receiving calls. The functions are relatively simple and the degree of intelligence is relatively low. Summary of the invention

[0003] The embodiments of the present application provide a natural language command control system, method and smart glasses based on artificial intelligence, which are used to improve the convenience of device control based on the smart glasses system, and improve the intelligence and interactivity of the smart glasses.

[0004] On the one hand, an embodiment of the present application provides a natural language command control system based on a generative artificial intelligence large language model, the system comprising a smart glasses system and a model server, wherein the generative artificial intelligence large language model is configured on the model server;

[0005] The smart glasses system is used to obtain a first user voice, perform semantic analysis on the first user voice, generate at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the model server;

[0006] The model server is used to obtain at least one task execution command and send it to the smart glasses system according to the at least one first prompt information sent by the smart glasses system through the generative artificial intelligence large language model;

[0007] The smart glasses system is further configured to execute at least one operation corresponding to the at least one task execution command.

[0008] In one aspect, an embodiment of the present application further provides smart glasses for natural language commands based on a generative artificial intelligence large language model, the smart glasses comprising: a frame, temples, a microphone, a processor, and a memory;

[0009] The temple is connected to the frame, and the processor is electrically connected to the microphone and the memory;

[0010] The memory stores one or more programs executable by the processor, wherein the one or more programs include multiple instructions, and the multiple instructions are used to:

[0011] Acquiring a first user's voice through the microphone;

[0012] Performing semantic analysis on the first user's speech;

[0013] Obtain at least one task execution command based on the parsed semantics through a generative artificial intelligence large language model; and

[0014] At least one operation corresponding to the at least one task execution command is executed.

[0015] On the one hand, the embodiment of the present application also provides a natural language command control method based on a generative artificial intelligence large language model, which is applied to a smart wearable device system, and the method includes:

[0016] Acquire a first user voice, perform semantic analysis on the first user voice and obtain a analysis result;

[0017] Using the generative artificial intelligence large language model, according to the parsing result, obtain at least one task execution command; and

[0018] At least one operation corresponding to the at least one task execution command is executed.

[0019] In the embodiments of the present application, a smart glasses system utilizes a generative artificial intelligence big language model to implement device control based on natural language voice commands, thereby improving the convenience of device control based on the smart glasses system. In addition, due to the scalability and self-creativity of the generative artificial intelligence big language model, the intelligence and interactivity of the smart glasses system can be further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] Figure 1 A schematic diagram of the structure of a natural language command control system based on a generative artificial intelligence large language model provided in one embodiment of the present application;

[0022] Figure 2 for Figure 1 A schematic diagram of an application scenario of the control system shown;

[0023] Figure 3 A schematic diagram of the structure of a natural language command control system based on a generative artificial intelligence large language model provided by another embodiment of the present application;

[0024] Figure 4 Based on Figure 1 and Figure 3 A schematic diagram of a first system framework of the control system shown;

[0025] Figure 5 Based on Figure 1 and Figure 3 A schematic diagram of a second system framework of the control system shown;

[0026] Figure 6 Based on Figure 1 and Figure 3 A schematic diagram of a third system framework of the control system shown;

[0027] Figure 7 Based on Figure 1 and Figure 3 A schematic diagram of a fourth system framework of the control system shown;

[0028] Figure 8 Based on Figure 1 and Figure 3 A schematic diagram of a fifth system framework of the control system shown;

[0029] Fig. 9 Based on Figure 1 and Figure 3 A schematic diagram of a sixth system framework of the control system shown;

[0030] Fig.10 Based on Figure 1 and Figure 3 A schematic diagram of a seventh system framework of the control system shown;

[0031] Fig.11 Based on Figure 1 and Figure 3 A schematic diagram of an eighth system framework of the control system shown;

[0032] Fig.12 A schematic diagram of the internal structure of the smart glasses provided in one embodiment of the present application;

[0033] Fig.13 A schematic diagram of the external structure of the smart glasses provided in one embodiment of the present application;

[0034] Fig.14 A flowchart for implementing a natural language command control method based on a generative artificial intelligence large language model provided in one embodiment of the present application;

[0035] Fig.15 and Fig.16 for Fig.14 Schematic diagram of the working mechanism of the control method shown. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0037] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present invention, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or adding the possibility of one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0038] Furthermore, the terms “first”, “second”, “third”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0039] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present invention belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present invention.

[0040] See also Figure 1 , Figure 1 This is a schematic diagram of the structure of a natural language command control system based on a generative artificial intelligence large language model provided in one embodiment of the present application. Figure 1 As shown, the control system 100 includes: a smart glasses system 110 and a model server 120.

[0041] Among them, the model server 120 can be a single server or a distributed server cluster composed of multiple servers, on which a generative artificial intelligence large language model (GAILLM) is configured. For ease of understanding, it is collectively referred to as a GAILLM server below.

[0042] Specifically, the smart glasses system 110 is used to obtain a first user voice, perform semantic analysis on the first user voice, generate at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the GAILLM server 120 .

[0043] The GAILLM server 120 is used to obtain at least one task execution command and send it to the smart glasses system 110 through the generative artificial intelligence large language model according to the at least one first prompt information sent by the smart glasses system 110.

[0044] The smart glasses system 110 is further configured to execute at least one operation corresponding to the at least one task execution command.

[0045] The first user voice includes at least one user voice command. The smart glasses system 110 performs semantic analysis on the first user voice through natural language processing (NLP), and generates at least one first prompt message according to the analyzed semantics. The first prompt message includes the analyzed semantics. Specifically, the smart glasses system 110 can use each semantic as a prompt message, or use multiple associated semantics as a prompt message according to the association relationship between the semantics.

[0046] The generative artificial intelligence large language model can be, for example but not limited to: Open AI's ChatGPT, Google's Bard, and other models with similar functions. The generative artificial intelligence large language model is trained using a large number of semantics and corresponding task execution commands as samples. Optionally, the same task execution command may correspond to multiple similar semantics, and the same semantics may correspond to different task execution commands. The task execution command is used to instruct the target object to perform at least one target task. One of the target tasks is associated with at least one corresponding operation. The task execution command includes description information of the target task. The description information of the target task is used to indicate the content of the target task, such as which operations need to be performed.

[0047] For example, assuming that the first user's voice contains the semantics of "I want to call Simon" or "I want to talk to Simon immediately", the generative artificial intelligence large language model obtains the corresponding task execution command based on the semantics to instruct the smart glasses system 110 to execute the operation of calling Simon.

[0048] Furthermore, the task execution command may also include description information of the executor of each target task, such as the name, type, and functions of the executor.

[0049] like Figure 2As shown, in an actual application, the smart glasses system 110 includes smart glasses 111, and the user can wear the smart glasses 111 and speak a first user voice. The smart glasses 111 obtain the first user voice through a built-in microphone, perform semantic analysis on the first user voice, and use a built-in prompt generator to generate at least one first prompt information according to the analyzed semantics and send it to the GAILLM server 120 (for example, the analyzed semantics can be sent to the GAILLM server 120 as the first prompt information), so that the GAILLM server 120 uses GAILLM to obtain at least one task execution command according to the first prompt information. Each task execution command corresponds to at least one operation, which can include but is not limited to: making a call, sending a message, sending an email, searching the Internet, calling network services, positioning and navigation, calling other third-party software development tools (Software Development Kit, SDK) to execute the tasks provided by the SDK, and controlling related devices in the Internet of Things, etc.

[0050] The prompt generator can be a software module or a microcontroller configured with the software module, which is used to generate corresponding prompt information based on the semantics in the parsed user voice, and the semantics is obtained by parsing each user voice, such as: the first prompt information and the second prompt information below.

[0051] Optionally, the prompt generator can also be configured in the GAILLM server 120, and the smart glasses 111 send the parsed semantics to the GAILLM server 120. The GAILLM server 120 uses the prompt generator to generate at least one first prompt message and inputs the first prompt message into the generative artificial intelligence large language model.

[0052] Figure 3 This is a schematic diagram of the structure of a natural language command control system based on a generative artificial intelligence large language model provided by another embodiment of the present application. Optionally, in other implementations of the present application, such as Figure 3 As shown, the smart glasses system 110 includes: smart glasses 111 and a smart mobile terminal 112. Furthermore, a corresponding application (APP) is installed in the smart glasses 111 or the smart mobile terminal 112, and the smart glasses 111 and the smart mobile terminal 112 establish a data connection via Bluetooth and use the APP for data interaction.

[0053] The smart glasses 111 may be open smart glasses, and the specific structure thereof may be referred to as follows: Fig.12 and Fig.13 Related description of the illustrated embodiment.

[0054] The intelligent mobile terminal 112 may include, but is not limited to: a cellular phone, a smart phone, other wireless communication devices, a personal digital assistant, an audio player, other media players, a music recorder, a video recorder, a camera, other media recorders, a smart radio, a laptop computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a moving picture expert group (MPEG-1 or MPEG-2) audio layer 3 (MP3) player, a digital camera, and an intelligent wearable device (such as a smart watch, a smart bracelet, etc.). The intelligent mobile terminal 112 is also installed with an Android, IOS or other operating system.

[0055] Specifically, the smart glasses 111 are used to acquire the first user's voice through a built-in microphone and send it to the smart mobile terminal 112 via Bluetooth.

[0056] The smart mobile terminal 112 is used to convert the first user's voice into a first text through a voice-to-text engine and perform semantic analysis on the first text, generate the at least one first prompt message according to the analyzed semantics, and send the at least one first prompt message to the GAILLM server 120. The voice-to-text engine can be configured in the smart mobile terminal 112, or can also be configured in a server in the cloud, such as a voice-to-text server, a prompt server or a GAILLM server 120. The smart mobile terminal 112 can convert the first user's voice into a corresponding first text by sending the first user's voice to a server configured with a voice-to-text engine. The smart mobile terminal 112 can be configured with a prompt generator to generate the at least one first prompt message according to the analyzed semantics.

[0057] The GAILLM server 120 is also used to obtain the at least one task execution command and send it to the smart mobile terminal 112 through the generative artificial intelligence large language model according to the at least one first prompt information sent by the smart mobile terminal 112.

[0058] The intelligent mobile terminal 112 is further configured to execute at least one operation corresponding to the at least one task execution command.

[0059] Optionally, in other embodiments of the present application, the smart mobile terminal 112 is also used to generate multiple first prompt messages according to the parsed semantics, and send the multiple first prompt messages and the order in which the semantics corresponding to each of the first prompt messages appear in the first text to the GAILLM server 120.

[0060] The GAILLM server 120 is also used to obtain multiple task execution commands through the generative artificial intelligence large language model based on the multiple first prompt messages and the sequence information sent by the smart mobile terminal 112, and send the multiple task execution commands and the execution sequence information of the multiple task execution commands to the smart mobile terminal 112, and the execution order corresponds to the sequence.

[0061] The intelligent mobile terminal 112 is further configured to execute operations corresponding to each of the task execution commands according to the execution sequence information.

[0062] Optionally, in other embodiments of the present application, the smart mobile terminal 112 is also used to: generate multiple first prompt messages according to the parsed semantics, and send the multiple first prompt messages to the GAILLM server 120 one by one in the order in which the semantics corresponding to each first prompt message appear in the first text.

[0063] Specifically, after sending a first prompt message, if the smart mobile terminal 112 receives at least one corresponding task execution command sent by the GAILLM server 120, it will send the next first prompt message to the GAILLM server 120; or, after sending a first prompt message, the smart mobile terminal 112 receives at least one corresponding task execution command sent by the GAILLM server 120, and after executing the at least one corresponding task execution command, it will send the next first prompt message to the GAILLM server 120.

[0064] Optionally, in other implementations of the present application, based on the above embodiments, the GAILLM server 120 is further configured to:

[0065] Determine, by means of the generative artificial intelligence large language model, whether there is information that needs to be supplemented or confirmed according to the at least one first prompt information;

[0066] When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command through the generative artificial intelligence large language model according to the at least one first prompt information;

[0067] When there is information that needs to be supplemented or confirmed, a text containing prompt information of the information that needs to be supplemented or confirmed is generated through the generative artificial intelligence large language model, and the text is sent to the smart mobile terminal 112;

[0068] Receive the second prompt information sent by the smart mobile terminal 112, and determine whether there is information that needs to be supplemented or confirmed based on all the received prompt information through the generative artificial intelligence large language model;

[0069] When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command through the generative artificial intelligence large language model according to all the received prompt information; and

[0070] When there is information that needs to be supplemented or confirmed, return to execute the step of generating a text of prompt information containing the information that needs to be supplemented or confirmed through the generative artificial intelligence large language model, and sending the text to the smart mobile terminal 112.

[0071] The smart mobile terminal 112 is also used to convert the text into speech using a text-to-speech engine, and send the speech to the smart glasses 111. The text-to-speech engine can be configured locally in the smart mobile terminal 112, or can also be configured in a cloud server, such as a text-to-speech server, a prompt server, or a GAILLM server 120. The smart mobile terminal 112 can convert the text into corresponding speech by sending the text to a server configured with a text-to-speech engine.

[0072] The smart glasses 111 are also used to receive the voice via the Bluetooth, play the voice via the built-in speaker, obtain the second user's voice via the microphone, and send the second user's voice to the smart mobile terminal 112 via the Bluetooth.

[0073] The smart mobile terminal 112 is also used to use the speech-to-text engine to convert the second user speech into a second text, perform semantic analysis on the second text, generate the second prompt information according to the semantics in the analyzed second text, and send the second prompt information to the GAILLM server 120.

[0074] Alternatively, when there is information that needs to be supplemented or confirmed, the reply generated by the generative artificial intelligence large language model may also include: at least one task execution command and text of prompt information containing the information that needs to be supplemented or confirmed, so as to confirm or consult the user for more information while executing the corresponding operation, so as to obtain further task execution commands based on the user's reply, thereby further improving the flexibility and intelligence of task execution.

[0075] For example, assuming that the first user voice is to reserve the first conference room at 3 pm, the generative artificial intelligence language model can be used to obtain the first task execution command and the text containing the prompt information "Who are the participants? Do I need to notify them?" The smart mobile terminal 112 converts the text into voice and sends it to the smart glasses 111 for playback. At the same time, according to the first task execution command, the operation of reserving the first conference room at 3 pm with the cloud conference management server is executed.

[0076] After playing the voice, the smart glasses 111 obtain the second user voice and send it to the smart mobile terminal 112, such as the user voice of "Call Peter and Simon to inform them that the meeting will be held in the first conference room at 3 pm". The smart mobile terminal 112 converts the second user voice into a second text, performs semantic analysis on the second text, generates corresponding prompt information based on the analyzed semantics, and sends it to the generative artificial intelligence large language model. The generative artificial intelligence large language model generates a second task execution command based on the corresponding prompt information, so that the smart mobile terminal 112 executes the operation of obtaining the phone numbers of Peter and Simon according to the second task execution command, generating a notification voice such as "Meeting will be held in the first conference room at 3 pm, please attend on time", and calling Peter and Simon using the phone number and playing the notification voice.

[0077] Optionally, in other embodiments of the present application, Figure 3 As shown, the smart glasses system 110 includes smart glasses 111 and a prompt server 113 , and the control system 100 also includes a speech-to-text server 130 and a text-to-speech server 140 .

[0078] The smart glasses 111 are used to acquire the first user's voice through a built-in microphone and send the first user's voice to the prompt server 113 .

[0079] The prompt server 113 is configured to send the first user voice to the voice-to-text server 130 .

[0080] The speech-to-text server 130 is configured to convert the first user speech into a first text and send the first text to the prompt server 113 .

[0081] The prompt server 113 is further configured to perform semantic analysis on the first text, generate the at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the GAILLM server 120. The prompt server 113 may be configured with a prompt generator to generate the at least one first prompt information according to the analyzed semantics.

[0082] The GAILLM server 120 is also used to determine whether there is information that needs to be supplemented or confirmed based on the at least one first prompt information through the generative artificial intelligence large language model; when the information that needs to be supplemented or confirmed does not exist, according to the at least one first prompt information, the at least one task execution command is obtained through the generative artificial intelligence large language model, and the at least one task execution command is sent to the prompt server 113; when the information that needs to be supplemented or confirmed exists, according to the at least one first prompt information, the at least one task execution command is obtained through the generative artificial intelligence large language model, and the at least one task execution command and the prompt information of the information that needs to be supplemented or confirmed are sent to the prompt server 113.

[0083] The prompt server 113 is also used to send a control instruction to at least one device in the Internet of Things according to the at least one task execution command to control the at least one device to perform the operation specified by the at least one task execution command, and to generate corresponding text according to the prompt information and send the text to the text-to-speech server 140.

[0084] The text-to-speech server 140 is used to convert the text into speech and send the speech to the prompt server 113 .

[0085] The prompt server 113 is also used to send the voice to the smart glasses 111.

[0086] The smart glasses 111 are also used to play the voice through the built-in speakers.

[0087] Optional, such as Figure 3 As shown, in other embodiments of the present application, the smart glasses system 110 includes smart glasses 111, a smart mobile terminal 112 and a prompt server 113, and the control system 100 also includes a speech-to-text server 130 and a text-to-speech server 140.

[0088] The smart glasses 111 are used to acquire the first user's voice through a built-in microphone and send the first user's voice to the smart mobile terminal 112.

[0089] The smart mobile terminal 112 is used to send the first user's voice to the voice-to-text server 130 .

[0090] The speech-to-text server 130 is configured to convert the first user speech into a first text and send the first text to the smart mobile terminal 112 .

[0091] The smart mobile terminal 112 is further configured to send the first text to the prompt server 113 .

[0092] The prompt server 113 is configured to perform semantic analysis on the first text, generate the at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the GAILLM server 120 .

[0093] The GAILLM server 120 is also used to determine whether there is information that needs to be supplemented or confirmed based on the at least one first prompt information through the generative artificial intelligence large language model; when the information that needs to be supplemented or confirmed does not exist, according to the at least one first prompt information, the at least one task execution command is obtained through the generative artificial intelligence large language model, and the at least one task execution command is sent to the prompt server 113; when the information that needs to be supplemented or confirmed exists, according to the at least one first prompt information, the at least one task execution command is obtained through the generative artificial intelligence large language model, and the prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command are sent to the prompt server 113.

[0094] The prompt server 113 is also used to send a control instruction to at least one device in the Internet of Things according to the at least one task execution command to control the at least one device to perform the operation specified by the at least one task execution command, and to generate corresponding text according to the prompt information and send the text to the smart mobile terminal 112.

[0095] The smart mobile terminal 112 is further configured to send the text to the text-to-speech server 140 .

[0096] The text-to-speech server 140 is used to convert the text into speech and send the speech to the smart mobile terminal 112 .

[0097] The smart mobile terminal 112 is also used to send the voice to the smart glasses 111.

[0098] The smart glasses 111 are also used to play the voice through the built-in speakers.

[0099] Optionally, in other embodiments of the present application, Figure 3 As shown, the smart glasses system 110 includes smart glasses 111 , a smart mobile terminal 112 and a prompt server 113 .

[0100] The smart glasses 111 are used to acquire the first user's voice through a built-in microphone and send the first user's voice to the smart mobile terminal 112 .

[0101] The intelligent mobile terminal 112 is used to convert the first user voice into a first text and send the first text to the prompt server 113.

[0102] The prompt server 113 is further configured to perform semantic analysis on the first text, generate the at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the GAILLM server 120 .

[0103] The GAILLM server 120 is also used to determine whether there is information that needs to be supplemented or confirmed based on the at least one first prompt information through the generative artificial intelligence large language model; when the information that needs to be supplemented or confirmed does not exist, according to the at least one first prompt information, the at least one task execution command is obtained through the generative artificial intelligence large language model, and the at least one task execution command is sent to the prompt server 113; when the information that needs to be supplemented or confirmed exists, according to the at least one first prompt information, the at least one task execution command is obtained through the generative artificial intelligence large language model, and the prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command are sent to the prompt server 113.

[0104] The prompt server 113 is also used to send a control instruction to at least one device in the Internet of Things according to the at least one task execution command to control the at least one device to perform the operation specified by the at least one task execution command, and to generate corresponding text according to the prompt information and send the text to the smart mobile terminal 112.

[0105] The smart mobile terminal 112 is also used to convert the text into speech and send the speech to the smart glasses 111.

[0106] The smart glasses 111 are also used to play the voice through the built-in speakers.

[0107] Optionally, in other embodiments of the present application, the smart glasses system 110 includes smart glasses 111 and a smart mobile terminal 112, and the control system 100 further includes a prompt server. The prompt server is configured with a speech-to-text engine, and the smart mobile terminal 112 is configured with a text-to-speech engine.

[0108] The smart glasses 111 are used to obtain the first user's voice through a built-in microphone and send the first user's voice to the prompt server through a wireless network.

[0109] The prompt server is used to convert the first user voice into a first text through the voice-to-text engine and perform semantic analysis on the first text, generate the at least one first prompt message according to the analyzed semantics, and send the at least one first prompt message to the smart mobile terminal 112. Wherein, when sending the first user voice, the smart glasses 111 also send the identification information of the smart mobile terminal 112 to the prompt server, so that the prompt server sends the at least one first prompt message to the smart mobile terminal 112 according to the identification information. Alternatively, the prompt server pre-stores the association information between the smart glasses 111 and the smart mobile terminal 112, and when sending the first user voice, the smart glasses 111 also send their own identity information to the prompt server, so that the prompt server sends the at least one first prompt message to the smart mobile terminal 112 according to the identity information and the association information. Alternatively, the prompt server can also send the at least one first prompt message to the smart glasses 111, so that the at least one first prompt message is forwarded to the smart mobile terminal 112 through the smart glasses 111.

[0110] The smart mobile terminal 112 is configured to send the at least one first prompt information to the GAILLM server 120 .

[0111] The GAILLM server 120 is also used to determine whether there is information that needs to be supplemented or confirmed based on the at least one first prompt information through the generative artificial intelligence large language model; when there is no information that needs to be supplemented or confirmed, the at least one task execution command is obtained through the generative artificial intelligence large language model based on the at least one first prompt information, and the at least one task execution command is sent to the smart mobile terminal 112; when there is information that needs to be supplemented or confirmed, the at least one task execution command is obtained through the generative artificial intelligence large language model based on the at least one first prompt information, and the prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command are sent to the smart mobile terminal 112. The prompt information of the information that needs to be supplemented or confirmed is generated by the GAILLM server 120 based on the determined information that needs to be supplemented or confirmed, and is used to indicate or prompt what the information that needs to be supplemented or confirmed is.

[0112] The intelligent mobile terminal 112 is further configured to execute at least one operation corresponding to the at least one task execution command.

[0113] The smart mobile terminal 112 is further configured to generate corresponding text according to the prompt information, convert the text into speech through the text-to-speech engine, and send the speech to the smart glasses 111. For example, the prompt information may be "Confirm which Peter in the contact list you are calling", and the corresponding text may be "Which Peter do you need to call? Peter (1), phone number 12345, Peter (2), phone number 23456, Peter (3), phone number 34567".

[0114] The smart glasses 111 are also used to play the voice through the built-in speakers.

[0115] Optionally, in other embodiments of the present application, the smart glasses system 110 includes: smart glasses 111 and a smart mobile terminal 112, and the generative artificial intelligence large language model is configured on the smart mobile terminal 112.

[0116] The smart glasses 111 are used to acquire the first user's voice through a built-in microphone and send it to the smart mobile terminal 112 via Bluetooth.

[0117] The smart mobile terminal 112 is used to convert the first user's voice into a first text through a voice-to-text engine and perform semantic analysis on the first text, obtain at least one task execution command according to the parsed semantics through the generative artificial intelligence large language model configured on the smart mobile terminal 112, and perform at least one operation corresponding to the at least one task execution command. The voice-to-text engine can be configured in the smart mobile terminal 112 or in a voice-to-text server. The smart mobile terminal 112 can send the first user's voice to the voice-to-text server 130, and use the voice-to-text server 130 to convert the first user's voice into the first text, or use the voice-to-text server 130 to convert the first user's voice into the first text and perform semantic analysis on the first text.

[0118] Furthermore, the smart mobile terminal 112 is also used to determine whether there is information that needs to be supplemented or confirmed based on the parsed semantics through the generative artificial intelligence big language model; when the information that needs to be supplemented or confirmed does not exist, obtain the at least one task execution command through the generative artificial intelligence big language model based on the parsed semantics; when the information that needs to be supplemented or confirmed exists, generate a text of prompt information containing the information that needs to be supplemented or confirmed through the generative artificial intelligence big language model, convert the text into speech using a text-to-speech engine, and send the speech to the smart glasses 111.

[0119] The smart glasses 111 are also used to receive the voice via the Bluetooth, play the voice via the built-in speaker, obtain the second user's voice via the microphone, and send the second user's voice to the smart mobile terminal 112 via the Bluetooth.

[0120] The intelligent mobile terminal 112 is also used to use the speech-to-text engine to convert the second user's speech into a second text, perform semantic analysis on the second text, and determine whether there is information that needs to be supplemented or confirmed based on the semantics in the parsed second text (or all the parsed semantics, such as the semantics in the first text and the semantics in the second text) through the generative artificial intelligence large language model; when the information that needs to be supplemented or confirmed does not exist, obtain the at least one task execution command through the generative artificial intelligence large language model based on all the parsed semantics; when the information that needs to be supplemented or confirmed exists, return to execute the step of generating a text of prompt information containing the information that needs to be supplemented or confirmed through the generative artificial intelligence large language model.

[0121] Furthermore, the smart mobile terminal 112 is also used to determine whether the smart mobile terminal 112 is locally configured with the generative artificial intelligence large language model based on the installation log or preset mark when receiving the first user voice. If the smart mobile terminal 112 is locally configured with the generative artificial intelligence large language model, the parsed semantics are input into the local generative artificial intelligence large language model to obtain at least one task execution command through the local generative artificial intelligence large language model; otherwise, at least one first prompt message is generated according to the parsed semantics, and the generated at least one first prompt message is sent to the GAILLM server 120.

[0122] Optionally, in other embodiments of the present application, the generative artificial intelligence large language model may also be configured in the smart glasses 111. The smart glasses 111 are also used to: obtain the first user voice, perform semantic analysis on the first user voice, input the parsed semantics into the generative artificial intelligence large language model to execute at least one task command through the generative artificial intelligence large language model, and execute at least one operation corresponding to the at least one task execution command. Specifically, the smart glasses 111 may use a local speech-to-text engine, or through a speech-to-text server 130, to convert the first user voice into a corresponding first text, and perform semantic analysis on the first text. Among them, a prompt generator may be configured in the smart glasses 111 to generate at least one first prompt message based on the parsed semantics, and the smart glasses 111 input the at least one first prompt message into the generative artificial intelligence large language model to execute at least one task command through the generative artificial intelligence large language model.

[0123] Furthermore, the smart glasses 111 are also used to: determine whether the generative artificial intelligence large language model is installed locally based on the installation log; when the generative artificial intelligence large language model is installed locally on the smart glasses 111, input the parsed semantics into the generative artificial intelligence large language model to execute at least one task command through the generative artificial intelligence large language model; when the generative artificial intelligence large language model is not installed locally on the smart glasses 111, generate at least one first prompt message based on the parsed semantics, and send the first prompt message to the GAILLM server 120 or the smart mobile terminal 112 to obtain the at least one task execution command through the generative artificial intelligence large language model on the GAILLM server 120 or the smart mobile terminal 112.

[0124] Alternatively, the smart glasses 111 are also used to: determine the installation location of the generative artificial intelligence large language model according to pre-stored model configuration information; when the generative artificial intelligence large language model is installed locally on the smart glasses 111, the parsed semantics are input into the generative artificial intelligence large language model to execute at least one task command through the generative artificial intelligence large language model; when the generative artificial intelligence large language model is installed on the GAILLM server 120, at least one first prompt message is generated according to the parsed semantics, and the first prompt message is sent to the GAILLM server 120, or the first prompt message is forwarded to the GAILLM server 120 through the smart mobile terminal 112 to obtain the at least one task execution command through the generative artificial intelligence large language model on the GAILLM server 120; when the generative artificial intelligence large language model is installed on the smart mobile terminal 112, at least one first prompt message is generated according to the parsed semantics, and the first prompt message is sent to the smart mobile terminal 112 to obtain the at least one task execution command through the generative artificial intelligence large language model on the smart mobile terminal 112.

[0125] Furthermore, in other embodiments of the present application, the smart glasses 111, the smart mobile terminal 112 or the prompt server 113 are also used to: after receiving the at least one task execution command, determine the executor of the target task according to the description information of the target task in the task execution command or the description information of the executor of the target task, and distribute each task execution command to the corresponding executor, so that each executor performs at least one operation associated with the corresponding task execution command.

[0126] See also Figure 4 , Figure 4 Based on Figure 1 and Figure 3 The schematic diagram of the first system framework of the control system shown in FIG. Figure 4As shown, the control system includes: smart glasses 111, a smart mobile terminal 112 (such as a smart phone or a smart watch), and a GAILLM server 120. GAILLM is configured on the GAILLM server 120. An application (App) for wireless communication with the smart glasses 111 via, for example, Bluetooth is installed on the smart mobile terminal 112. Optionally, the smart glasses 111 and the GAILLM server 120 communicate wirelessly via, for example, a 4G or 5G cellular network or WIFI. The smart mobile terminal 112 and the GAILLM server 120 communicate wirelessly via, for example, a 4G or 5G cellular network or WIFI.

[0127] The smart glasses 111 obtain the first user voice and send the first user voice to the smart mobile terminal 112. The smart mobile terminal 112 converts the first user voice into a first text through a voice-to-text engine and performs semantic analysis on the first text, generates the at least one first prompt information according to the analyzed semantics, and sends the at least one first prompt information to the GAILLM server 120. The GAILLM server 120 obtains the at least one task execution command according to the at least one first prompt information through GAILLM and sends it to the smart mobile terminal 112. The smart mobile terminal 112 executes at least one operation corresponding to the at least one task execution command.

[0128] See also Figure 5 , Figure 5 Based on Figure 1 and Figure 3 Schematic diagram of the second system framework of the control system shown in FIG. Figure 5 As shown, the control system includes: smart glasses 111 and a smart phone 112. GAILLM is embedded in the smart phone 112. The smart phone 112 is also installed with a mobile phone App for wireless communication with the smart glasses 111 via Bluetooth.

[0129] User speech in natural language (i.e., Figure 5A voice command in an audio format such as "Call Peter" is sent from the smart glasses 11 to the App in the smartphone 112 via Bluetooth. The App converts the user's voice into text through a speech-to-text engine and performs semantic analysis on the text, and generates a first prompt message based on the analyzed semantics, which is used to inquire the generative artificial intelligence large language model about the intention of the user's voice command. Then, the first prompt message is input into GAILLM to obtain a reply output by GAILLM. The reply indicates "make a call" and includes a first task execution command. According to the first task execution command, the App executes an operation to obtain Peter's phone number in the smartphone contact list and sends the operation result to GAILLM, where the operation result may include, for example, all phone numbers associated with Peter recorded in the contact list.

[0130] According to the operation result, if there is only one Peter in the contact list, GAILLM outputs a second task execution command to enable the APP to make a call according to Peter's phone number in the contact list.

[0131] If Peter is not in the contact list, GAILLM outputs a prompt message asking the user for Peter's phone number and a reply to the third task execution command, so that the APP generates a text containing the prompt message, such as "Please enter Peter's phone number", and converts the text into a synthesized speech in audio format through a text-to-speech engine and sends it to the smart glasses 111 for playback. Then the APP parses the user voice obtained by the smart glasses 111 to obtain the phone number answered by the user, and executes the call operation according to the phone number answered by the user according to the third task execution command.

[0132] If there are multiple Peters on the contact phone, such as Peter Zhang, Peter Wang, and Peter Li, GAILLM outputs a reply containing a prompt message to the user to confirm which Peter needs to be called, so that the APP generates a text containing the prompt message, converts the text into a synthesized voice in audio format through a text-to-speech engine, and sends it to the smart glasses 111 for playback. After playing the synthesized voice, the smart glasses 111 obtain the second user voice (such as: Peter Li) and send it to the APP. The APP converts the user voice into text through a speech-to-text engine and performs semantic analysis on the text, generates a second prompt message based on the analyzed semantics, and inputs it into GAILLM. GAILLM determines the phone number of Peter Li to be called based on the second prompt message and outputs a fourth task instruction command, so that the APP performs a call operation according to the phone number according to the fourth task execution command.

[0133] See also Figure 6 , Figure 6 Based on Figure 1 and Figure 3 The schematic diagram of the third system framework of the control system shown in FIG. Figure 6 As shown, the control system includes: smart glasses 111, a smart phone 112 and a GAILLM server 120. GAILLM is configured in the GAILLM server 120. The smart phone 112 is also installed with a mobile phone App for wireless communication with the smart glasses 111 via Bluetooth. The mobile phone App also wirelessly communicates with the GAILLM server 120 via WIFI or a cellular network.

[0134] User speech in natural language (i.e., Figure 6 A voice command in an audio format such as "call Peter" is sent from the smart glasses 11 to the App in the smartphone 112 via Bluetooth. The App converts the user's voice into text through a speech-to-text engine and performs semantic analysis on the text, generates prompt information based on the analyzed semantics, and then sends the prompt information to the GAILLM server 120. The GAILLM server 120 inputs the prompt information into GAILLM to obtain a reply output by GAILLM and sends it to the App in the smartphone 112. The reply includes prompt information of information that needs to be supplemented or confirmed and a task execution command. The App generates a text containing prompt information of information that needs to be supplemented or confirmed for the reply, and converts the text into a synthesized speech in an audio format through a text-to-speech engine and sends it to the smart glasses 111 for playback, so as to ask the user for more information or confirm the reply. At the same time, the App executes at least one operation specified by the task execution command in the reply.

[0135] See also Figure 7 , Figure 7 Based on Figure 1 and Figure 3 Schematic diagram of the fourth system framework of the control system shown in FIG. Figure 7 As shown, the control system includes: smart glasses 111, a smart phone 112, and a prompt server 113 and a GAILLM server 120 in the cloud. GAILLM is configured in the GAILLM server 120. The smart phone 112 is also installed with a mobile phone App for wireless communication with the smart glasses 111 via Bluetooth. The mobile phone App also wirelessly communicates with the prompt server 113 via WIFI or a cellular network.

[0136] User speech in natural language (i.e., Figure 7 A voice command in an audio format such as "call Peter" is sent from the smart glasses 11 to the mobile phone App in the smart phone 112 via Bluetooth. The mobile phone App converts the user's voice into a first text through a speech-to-text engine and converts the first text (i.e., Figure 7The prompt server 113 performs semantic analysis on the first text, generates prompt information based on the analyzed semantics, and then sends the prompt information to the GAILLM server 120. The GAILLM server 120 inputs the prompt information into GAILLM to obtain a reply output by GAILLM and sends it to the prompt server 113. The reply includes prompt information of information that needs to be supplemented or confirmed and a task execution command. The prompt server 113 generates a second text (i.e., prompt information of information that needs to be supplemented or confirmed and a task execution command) for the reply. Figure 7 The mobile phone App converts the prompt information in the second text into a synthesized speech in audio format through a text-to-speech engine and sends it to the smart glasses 111 for playback, so as to ask the user for more information or confirm the reply. At the same time, the mobile phone App executes at least one operation specified by the task execution command in the reply.

[0137] See also Figure 8 , Figure 8 Based on Figure 1 and Figure 3 Schematic diagram of the fifth system framework of the control system shown in FIG. Figure 8 As shown, the control system includes: smart glasses 111, a smart phone 112, and a prompt server 113, a GAILLM server 120, a speech-to-text server 130, and a text-to-speech server 140 in the cloud. GAILLM is configured in the GAILLM server 120. The smart phone 112 is installed with a mobile phone App for wireless communication with the smart glasses 111 via Bluetooth. The mobile phone App also wirelessly communicates with the prompt server 113, the speech-to-text server 130, and the text-to-speech server 140 via WIFI or a cellular network.

[0138] User speech in natural language (i.e., Figure 8 A voice command in an audio format such as "call Peter" is sent from the smart glasses 11 to the mobile phone App in the smart phone 112 via Bluetooth. The mobile phone App converts the user's voice into a first text through the voice-to-text server 130 and writes the first text (i.e., Figure 8The prompt server 113 performs semantic analysis on the first text, generates prompt information based on the analyzed semantics, and then sends the prompt information to the GAILLM server 120. The GAILLM server 120 inputs the prompt information into GAILLM to obtain a reply output by GAILLM and sends it to the prompt server 113. The reply includes prompt information of information that needs to be supplemented or confirmed and a task execution command. The prompt server 113 generates a second text (i.e., prompt information of information that needs to be supplemented or confirmed and a task execution command) for the reply. Figure 8 The mobile phone App receives a reply in a text format in the second text format and sends the second text to the mobile phone App. The mobile phone App converts the prompt information in the second text into a synthesized voice in an audio format through the text-to-speech server 140 and sends it to the smart glasses 111 for playback, so as to ask the user for more information or confirm the reply. At the same time, the mobile phone App executes at least one operation specified by the task execution command in the reply.

[0139] See also Fig. 9 , Fig. 9 Based on Figure 1 and Figure 3 Schematic diagram of the sixth system framework of the control system shown in FIG. Fig. 9 As shown, the control system includes: smart glasses 111 and a GAILLM server 120 in the cloud. GAILLM is configured in the GAILLM server 120. The smart phone 112 is installed with a speech-to-text engine, a text-to-speech engine, and a mobile phone App for wireless communication with the smart glasses 111 via Bluetooth. Fig. 9 The system framework shown is Figure 8 The difference in the system framework shown is that in Fig. 9 In the system framework shown, the operations of voice-to-text conversion, generating prompt information, and text-to-speech conversion are all completed locally on the smartphone 112 by the mobile app in the smartphone 112 .

[0140] See also Fig.10 , Fig.10 Based on Figure 1 and Figure 3 The schematic diagram of the seventh system framework of the control system shown in FIG. Fig.10 As shown, the control system includes: smart glasses 111 and a prompt server 113, a GAILLM server 120, a speech-to-text server 130 and a text-to-speech server 140 in the cloud. GAILLM is configured in the GAILLM server 120. The smart glasses 111 communicate wirelessly with the prompt server 113 via WIFI or a cellular network.

[0141] A user wearing smart glasses 111 issues a voice command in natural language, for example: turn on the bedroom light in 10 minutes. Smart glasses 111 obtain the user command in the form of audio data and send the user command to the prompt server 113. Prompt server 113 converts the user command in the form of audio data into a first text through speech-to-text server 130, performs semantic analysis on the first text, generates prompt information based on the analyzed semantics, and then sends the prompt information to GAILLM server 120. GAILLM server 120 inputs the prompt information into GAILLM to obtain a reply output by GAILLM and sends it to the prompt server 113. The reply includes prompt information of information that needs to be supplemented or confirmed or a task execution command.

[0142] For example, when there is only one lamp in the bedroom, the reply includes a task execution command, so that the prompt server 113 executes an operation of sending a command to the bedroom lamp, which is an IoT device, via WIFI or a cellular network after 10 minutes according to the task execution command to control the bedroom lamp to turn on.

[0143] When there are multiple lights in the bedroom, the reply includes prompt information to confirm or inquire which light to turn on to the user. The prompt server 113 generates a text containing the prompt information for the reply, converts the text into a voice in audio format through the text-to-speech server 140, and sends the converted voice to the smart glasses 111 for playback so as to confirm or inquire which light in the bedroom needs to be turned on to the user in the form of audio data. After playing the voice, the smart glasses 111 obtain the user voice in the form of audio data (for example: turn on the bedside lamp in the bedroom) and send the voice to the prompt server 113. The prompt server 113 converts the user voice in the form of audio data into a second text through the voice-to-text server 130, performs semantic analysis on the second text, generates prompt information based on the analyzed semantics, and then sends the prompt information to the GAILLM server 120. The GAILLM server 120 inputs the prompt information into GAILLM to obtain the reply output by GAILLM and sends it to the prompt server 113. The reply includes a task execution command, so that the prompt server 113 executes an operation of sending a command to the bedside lamp as an IoT device via WIFI or a cellular network after 10 minutes according to the task execution command to control the bedside lamp to turn on.

[0144] See also Fig.11 , Fig.11 Based on Figure 1 and Figure 3 The schematic diagram of the eighth system framework of the control system shown in FIG. Fig.11As shown, the control system includes: smart glasses 111 and a prompt server 113 and a GAILLM server 120 in the cloud. GAILLM is configured in the GAILLM server 120. A mobile phone App for wireless communication with the smart glasses 111 via Bluetooth is installed in the smart phone 112. The mobile phone App also wirelessly communicates with the prompt server 113 via WIFI or a cellular network.

[0145] The user wearing the smart glasses 111 issues a voice command in natural language, for example: turn on the bedroom light in 10 minutes. The smart glasses 111 obtain the user command in the form of audio data and send the user command to the smart phone 112. The smart phone 112 converts the user command into a text-formatted command and sends the text-formatted command to the prompt server 113. The prompt server 113 performs semantic analysis on the text-formatted command, generates prompt information based on the parsed semantics, and then sends the prompt information to the GAILLM server 120. The GAILLM server 120 inputs the prompt information into GAILLM to obtain a reply output by GAILLM and sends it to the prompt server 113. The reply includes prompt information of information that needs to be supplemented or confirmed and a task execution command. The prompt server 113 executes an operation of sending a command to the bedroom light, which is an IoT device, in 10 minutes according to the task execution command to control the bedroom light to turn on. At the same time, the prompt server 113 generates a text-formatted reply containing prompt information of information that needs to be supplemented or confirmed to be sent to the user for the reply, and sends the text-formatted reply to the smart phone 112. The smartphone 112 converts the reply in text form to be sent to the user into voice and sends the converted voice to the smart glasses 111, and the smart glasses 111 plays the voice to confirm or consult more information to the user in the form of audio data.

[0146] In this embodiment, the smart glasses system utilizes the generative artificial intelligence big language model to implement device control based on natural language voice commands, thereby improving the convenience of device control based on the smart glasses system. In addition, due to the scalability and self-creativity of the generative artificial intelligence big language model, the intelligence and interactivity of the smart glasses system can be further improved.

[0147] See also Fig.12 , Fig.12 A schematic diagram of the internal structure of the smart glasses provided in one embodiment of the present application. Fig.13 This is a schematic diagram of the external structure of the smart glasses provided in one embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown in the figure. Fig.12 and Fig.13As shown, the smart glasses 200 include: a frame 201, at least one temple 202, at least one microphone 203, at least one speaker 204, at least one processor 205 and at least one memory 206. Fig.12 and Fig.13 This is only a best example. In actual applications, the smart glasses 200 may also have Fig.12 and Fig.13 Fewer or more components may be shown.

[0148] The frame 201 may be, for example, a front frame with lenses (eg, sunglass lenses, transparent lenses, or corrective lenses). The at least one temple 202 may include, for example, a left temple 202A and a right temple 202B.

[0149] The temple 202 is connected to the frame 201, and the processor 205 is electrically connected to the microphone 203, the speaker 204, and the memory 205. The microphone 203, the speaker 204, the processor 205, and the memory 206 are arranged on at least one of the temples 202 and / or the frame 201. Preferably, the temple 202 is detachably connected to the frame 201.

[0150] The processor 205 includes: a central processing unit (CPU) and a DSP (Digital Signal Processing). The DSP is used to process the voice data acquired by the microphone 203. The CPU is preferably a MCU (Microcontroller Unit).

[0151] The memory 206 is a non-temporary memory, which may specifically include: RAM (Random Access Memory) and a flash memory component, in which one or more programs executable by the processor 205 are stored, and the one or more programs include multiple instructions. The multiple instructions are used to obtain the first user's voice through the microphone 203, perform semantic analysis on the first user's voice, obtain at least one task execution command according to the parsed semantics through the generative artificial intelligence large language model, and execute at least one operation corresponding to the at least one task execution command. Among them, the generative artificial intelligence large language model can be configured locally in the smart glasses 200, or in other devices, such as: smart mobile terminals or other cloud servers. The multiple instructions can obtain the at least one task execution command by inputting the parsed semantics into the local generative artificial intelligence large language model, or by sending the parsed semantics to other devices configured with the generative artificial intelligence large language model.

[0152] Optionally, in other embodiments of the present application, the generative artificial intelligence large language model is configured on a model server (such as Figure 1 The GAILLM server 120 in the control system shown in the figure), the one or more programs also include a speech-to-text engine, the smart glasses 200 also include a wireless communication component 207 electrically connected to the processor 205, and the multiple instructions are also used to: convert the first user voice into a first text through the speech-to-text engine; perform semantic analysis on the first text, and generate at least one first prompt message according to the analyzed semantics; send the at least one first prompt message to the model server through the wireless communication component 207, so as to obtain at least one task execution command according to the at least one first prompt message through the generative artificial intelligence large language model in the model server; and receive the at least one task execution command sent by the model server through the wireless communication component 207.

[0153] The wireless communication component 207 includes a wireless signal transceiver and its surrounding circuits, which can be specifically arranged in the inner cavity of the frame 201 and / or at least one of the temples 202. The wireless signal transceiver can use, but is not limited to, at least one of the WiFi (Wireless Fidelity) protocol, NFC (Near Field Communication) protocol, ZigBee (ZigBee), UWB (UltraWideband, carrier-free communication), RFID (Radio Frequency Identification) protocol, and cellular mobile communication (Cellular Mobile Communication) protocol (such as 3G / 4G / 5G, etc.) for data transmission.

[0154] Optionally, in other embodiments of the present application, the generative artificial intelligence large language model is configured on a model server, and the smart glasses 200 also include a wireless communication component 207 electrically connected to the processor 205, and the multiple instructions are also used to: send the first user voice to the model server through the wireless communication component 207, so that the first user voice is converted into a first text through the speech-to-text engine in the model server and the first text is semantically parsed, and the generative artificial intelligence large language model in the model server obtains the at least one task execution command according to the parsed semantics; and receive the at least one task execution command sent by the model server through the wireless communication component 207.

[0155] Optionally, the speech-to-text engine may be configured in the model server or in the speech-to-text server. The smart glasses 200 or the model server may also convert the first user's speech into the first text by interacting with data in the speech-to-text server.

[0156] Optionally, in other embodiments of the present application, the multiple instructions are also used to: receive the multiple task execution commands and the execution order information of each task execution command sent by the model server through the wireless communication component 207; and execute operations corresponding to each task execution command according to the execution order information.

[0157] Optionally, in other embodiments of the present application, the model server sends the obtained multiple task execution commands to the smart glasses 200 one by one, and the multiple instructions are also used to: after executing the operation corresponding to a task execution command, send a notification message to the model server, so that the model server sends the next task execution command to the smart glasses 200 according to the notification message.

[0158] Optionally, in other embodiments of the present application, the multiple instructions are also used to: receive, through the wireless communication component 207, a voice prompt containing information that needs to be supplemented or confirmed, sent by the model server, and play the voice through the speaker; obtain a second user voice through the microphone 203, and send the second user voice to the model server through the wireless communication component 207, so that the model server obtains the at least one task execution command based on the first user voice and the second user voice through the generative artificial intelligence large language model.

[0159] Optionally, in other embodiments of the present application, the smart glasses 200 also include a wireless communication component 207 electrically connected to the processor 205, and the multiple instructions are also used to send control instructions to at least one device in the Internet of Things through the wireless communication component 207 according to the at least one task execution command, so as to control the at least one device to perform the operation specified by the at least one task execution command.

[0160] Optionally, in other embodiments of the present application, the smart glasses 200 also include a wireless communication component 207 electrically connected to the processor 205, and the generative artificial intelligence large language model is configured on the model server, and the multiple instructions are also used to: send the first user voice to the prompt server through the wireless communication component 207, so that the prompt server converts the first user voice into a first text through a speech-to-text engine and performs semantic analysis on the first text, generates at least one first prompt message according to the parsed semantics, and sends the at least one first prompt message to the model server, wherein the model server obtains the at least one task execution command according to the at least one first prompt message through the generative artificial intelligence large language model and sends it to the prompt server; and receives the at least one task execution command sent by the prompt server through the wireless communication component 207.

[0161] Optionally, in other embodiments of the present application, the smart glasses 200 also include a wireless communication component 207 electrically connected to the processor 205, and the generative artificial intelligence large language model is configured on the model server, and the multiple instructions are also used to: send the first user voice to the prompt server through the wireless communication component 207, so that the prompt server converts the first user voice into a first text through a speech-to-text engine and performs semantic analysis on the first text, and generates at least one first prompt message based on the analyzed semantics; receive the at least one first prompt message sent by the prompt server through the wireless communication component 207, and send the at least one first prompt message to the model server, so that the model server obtains the at least one task execution command based on the at least one first prompt message through the generative artificial intelligence large language model; and receive the at least one task execution command sent by the model server through the wireless communication component 207.

[0162] Optionally, in other embodiments of the present application, the generative artificial intelligence large language model can also be configured in the smart glasses 200, or configured in a smart mobile terminal. The smart glasses 200 also include a Bluetooth component 208 electrically connected to the processor 205. The Bluetooth component 208 includes a Bluetooth signal transceiver and surrounding circuits, which can be specifically arranged in the inner cavity of the frame 201 and / or at least one temple 202. The Bluetooth component 208 can be linked to a smart mobile terminal such as a smart phone or a smart watch for phone calls, music and data communications.

[0163] The multiple instructions are also used to obtain at least one task execution command based on the parsed semantics through the generative artificial intelligence large language model configured in the smart glasses 200.

[0164] The multiple instructions are also used to interact with the smart mobile terminal through the Bluetooth component 208, so as to forward the at least one first prompt information to the model server through the smart mobile terminal and receive the at least one task execution command sent by the model server, and obtain the at least one task execution command through the smart mobile terminal.

[0165] Optionally, in other embodiments of the present application, the smart glasses 200 further include an input device 209 electrically connected to the processor 205. The input device 209 includes a touch sensor and a button. The button may be a power button on the smart glasses 200.

[0166] Optionally, in other embodiments of the present application, the smart glasses 200 further include an indicator light and / or a buzzer electrically connected to the processor 205, and the multiple instructions are further used to output prompt information through the indicator light and / or the buzzer, and the prompt information is used to indicate the state of the smart glasses 200, which includes a working state and an idle state, and the working state includes: starting to pick up voice state, voice picking state, ending to pick up voice state, and voice processing state. Among them, the indicator light can be an LED (Light Emitting Diode) lamp.

[0167] Optionally, in other embodiments of the present application, the smart glasses 200 further include a data sensing component electrically connected to the processor 205, the data sensing component including at least one of a positioning component, an inertial measurement unit (IMU) sensor, a temperature sensor, a proximity sensor, a humidity sensor, an electronic compass, a timer, a camera, and a pedometer. The positioning component may be, but is not limited to, a positioning component based on GPS (Global Positioning System) or Beidou satellite.

[0168] The multiple instructions are also used to obtain the sensing data of the at least one device, and obtain the at least one task execution command according to the acquired sensing data and the parsed semantics through the generative artificial intelligence large language model. Or,

[0169] The multiple instructions are also used to obtain the sensing data of the at least one device, and send the sensing data of the at least one device and the at least one prompt information to the model server or the smart mobile terminal through the wireless communication component 207, so that the generative artificial intelligence large language model on the model server or the smart mobile terminal obtains at least one task execution command based on the sensing data and the at least one prompt information. For example: if the first user voice contains the semantics of "remind me when I walk near the cinema", the generative artificial intelligence large language model obtains a task execution command based on the real-time positioning data sent by the smart glasses 200 and the first prompt information containing the semantics when the user of the smart glasses 200 is about to arrive at the cinema, and sends the task execution command to the smart glasses 200 to instruct the smart glasses 200 to play the prompt voice of "you have arrived near the cinema".

[0170] Optionally, in other embodiments of the present application, the smart glasses 200 further include a voice biometrics module, which is used to identify the user of the smart glasses 200 by using the acquired voiceprint of the user to enable the above-mentioned voice control function based on the smart glasses 200.

[0171] Furthermore, the smart glasses 200 also include a battery 210 for providing power support for the electronic components of the smart glasses 200, such as the microphone 203, the speaker 204, the processor 205, and the memory 206.

[0172] The electronic components of the above-mentioned smart glasses can be connected through a bus.

[0173] It should be noted that the relationship between the components of the above-mentioned smart glasses can be a replacement relationship or a superposition relationship. That is, all the components in the above-mentioned embodiment can be installed on one smart glass, or a part of the above-mentioned components can be selectively installed according to needs. When it is a replacement relationship, the smart glasses are also provided with a connection interface for peripherals, and the connection interface can be at least one of a PS / 2 interface, a serial interface, a parallel interface, an IEEE1394 interface, a USB (Universal Serial Bus) interface, etc. The function of the replaced component can be realized by a peripheral connected to the connection interface, such as: an external speaker, an external sensor, etc.

[0174] For details not covered in this embodiment, please refer to the above Figures 1 to 3 and the following Figures 14 to 16 The relevant description in the illustrated embodiment will not be repeated here.

[0175] In this embodiment, the generative artificial intelligence big language model is used in smart glasses to realize device control based on natural language voice commands, thereby improving the convenience of device control based on smart glasses. In addition, due to the scalability and self-creativity of the generative artificial intelligence big language model, the intelligence and interactivity of the smart glasses can be further improved.

[0176] See also Fig.14 , Fig.14 This is a flowchart of the implementation of a natural language command control method based on a generative artificial intelligence large language model provided in an embodiment of the present application. The method can be applied to a smart wearable device system, such as Figure 1 and Figure 3 The smart glasses system 110 in the illustrated embodiment. The smart wearable device may include, but is not limited to: a smart helmet, a smart headset, a smart earring, a smart watch, and the like. Fig.12 and Fig.13 As shown in the smart glasses, etc. Fig.14 As shown, the method comprises the following steps:

[0177] S301, obtaining a first user voice, performing semantic analysis on the first user voice and obtaining an analysis result;

[0178] S302, using a generative artificial intelligence large language model to obtain at least one task execution command according to the parsing result; and

[0179] S303: Execute at least one operation corresponding to the at least one task execution command.

[0180] The generative artificial intelligence language model can be configured in any device in the smart wearable device system, such as Figure 3 In the smart glasses 111, the smart mobile terminal 112 or the prompt server 113 in the smart glasses system 110 shown, or other cloud servers may also be configured, such as Figure 1 and Figure 3 In the GAILLM server 120 shown.

[0181] Optionally, in other embodiments of the present application, the smart wearable device system includes a smart wearable device and a smart mobile terminal, the generative artificial intelligence large language model is configured on a model server, the first user voice is obtained, the first user voice is semantically parsed and a parsing result is obtained, the generative artificial intelligence large language model is used to obtain at least one task execution command according to the parsing result, and the step of executing at least one operation corresponding to the at least one task execution command includes:

[0182] Acquire the first user's voice through the smart wearable device, convert the first user's voice into a first text using a speech-to-text engine, perform semantic analysis on the first text, generate at least one first prompt message according to the analyzed semantics, and send the first prompt message to the smart mobile terminal through Bluetooth;

[0183] Sending the at least one first prompt information to the model server through the smart mobile terminal, so as to obtain at least one task execution command according to the at least one first prompt information through the generative artificial intelligence large language model in the model server; and

[0184] The at least one task execution command sent by the model server is received through the intelligent mobile terminal, and at least one operation corresponding to the at least one task execution command is executed.

[0185] Optionally, in other embodiments of the present application, the smart wearable device system includes a smart wearable device and a smart mobile terminal, the generative artificial intelligence large language model is configured on a model server, the first user voice is obtained, the first user voice is semantically parsed and a parsing result is obtained, the generative artificial intelligence large language model is used to obtain at least one task execution command according to the parsing result, and the step of executing at least one operation corresponding to the at least one task execution command includes:

[0186] Acquire the first user's voice through the smart wearable device and send it to the smart mobile terminal via Bluetooth;

[0187] By means of the smart mobile terminal, the first user speech is converted into a first text by means of a speech-to-text engine and the first text is semantically parsed, at least one first prompt message is generated according to the parsed semantics, and the at least one first prompt message is sent to the model server, so as to obtain at least one task execution command according to the at least one first prompt message through the generative artificial intelligence large language model in the model server; and

[0188] The at least one task execution command sent by the model server is received through the intelligent mobile terminal, and at least one operation corresponding to the at least one task execution command is executed.

[0189] Optionally, in other embodiments of the present application, the smart wearable device system includes a smart wearable device and a smart mobile terminal, the generative artificial intelligence large language model is configured in the smart mobile terminal, the first user voice is obtained, the first user voice is semantically parsed and the parsing result is obtained, the generative artificial intelligence large language model is used to obtain at least one task execution command according to the parsing result, and the step of executing at least one operation corresponding to the at least one task execution command includes:

[0190] Acquire the first user's voice through the smart wearable device, and send it to the smart mobile terminal through Bluetooth; and

[0191] Through the intelligent mobile terminal, a speech-to-text engine is used to convert the first user's speech into a first text, and the first text is semantically parsed and a parsing result is obtained. The generative artificial intelligence large language model is used to obtain at least one task execution command based on the parsing result, and at least one operation corresponding to the at least one task execution command is executed.

[0192] Optionally, in other embodiments of the present application, the smart mobile terminal generates a plurality of first prompt messages according to the parsed semantics, and sends the plurality of first prompt messages and the information of the order in which the semantics corresponding to each of the first prompt messages appear in the first text to the model server; the model server obtains a plurality of task execution commands through the generative artificial intelligence large language model according to the plurality of first prompt messages and the order information, and sends the plurality of task execution commands and the execution order information of the plurality of task execution commands to the smart mobile terminal, and the execution order corresponds to the order;

[0193] The step of receiving, by the smart mobile terminal, the at least one task execution command sent by the model server, and executing at least one operation corresponding to the at least one task execution command, further includes:

[0194] The multiple task execution commands and the execution sequence information of the multiple task execution commands sent by the model server are received through the intelligent mobile terminal, and the operations corresponding to each of the task execution commands are performed according to the execution sequence information.

[0195] Optionally, in other embodiments of the present application, the smart mobile terminal generates multiple first prompt messages according to the parsed semantics, and sends the multiple first prompt messages to the model server one by one.

[0196] Optionally, in other embodiments of the present application, the model server determines whether there is information that needs to be supplemented or confirmed according to the at least one first prompt information through the generative artificial intelligence big language model; when the information that needs to be supplemented or confirmed does not exist, the at least one task execution command is obtained through the generative artificial intelligence big language model according to the at least one first prompt information; when the information that needs to be supplemented or confirmed exists, the generative artificial intelligence big language model is used to generate a text of the prompt information containing the information that needs to be supplemented or confirmed, and the text is sent to the smart mobile terminal;

[0197] The method further includes:

[0198] The text sent by the model server is received through the smart mobile terminal, the text is converted into speech using a text-to-speech engine, and the speech is sent to the smart glasses;

[0199] Play the voice through the smart glasses, obtain the second user's voice, and send the second user's voice to the smart mobile terminal through the Bluetooth;

[0200] Through the smart mobile terminal, using a speech-to-text engine, the second user's voice is converted into a second text, the second text is semantically parsed, and a second prompt information is generated according to the semantics in the parsed second text, and the second prompt information is sent to the model server, so that the model server uses the generative artificial intelligence big language model to determine whether there is information that needs to be supplemented or confirmed based on all the received prompt information. When the information that needs to be supplemented or confirmed does not exist, the model server obtains the at least one task execution command through the generative artificial intelligence big language model based on all the received prompt information. When the information that needs to be supplemented or confirmed exists, the model server returns to execute the step of generating a text of the prompt information containing the information that needs to be supplemented or confirmed through the generative artificial intelligence big language model, and sending the text to the smart mobile terminal.

[0201] Optionally, in other embodiments of the present application, the steps of performing semantic parsing on the first text and obtaining a parsing result, using the generative artificial intelligence large language model, obtaining at least one task execution command according to the parsing result, and executing at least one operation corresponding to the at least one task execution command include:

[0202] By means of the smart mobile terminal, semantic analysis is performed on the first text to obtain a first analysis result, and by means of the generative artificial intelligence large language model, it is determined whether there is information that needs to be supplemented or confirmed according to the first analysis result;

[0203] When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command through the intelligent mobile terminal according to the first analysis result and using the generative artificial intelligence large language model;

[0204] When there is information that needs to be supplemented or confirmed, the smart mobile terminal uses the generative artificial intelligence large language model to generate a text of prompt information containing the information that needs to be supplemented or confirmed, uses a text-to-speech engine to convert the text into speech, and sends the speech to the smart wearable device via the Bluetooth;

[0205] Play the voice through the smart wearable device, obtain the second user's voice, and send the second user's voice to the smart mobile terminal through the Bluetooth;

[0206] By using the smart mobile terminal, the second user's speech is converted into a second text by using a speech-to-text engine, the second text is semantically parsed and a second parsing result is obtained, and by using the generative artificial intelligence large language model, it is determined whether there is the information that needs to be supplemented or confirmed according to the first parsing result and the second parsing result;

[0207] When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command by the intelligent mobile terminal according to the first analysis result and the second analysis result and using the generative artificial intelligence large language model; and

[0208] When there is information that needs to be supplemented or confirmed, return to execute the step of using the generative artificial intelligence large language model to generate a text of prompt information containing the information that needs to be supplemented or confirmed.

[0209] Optionally, in other embodiments of the present application, the smart wearable system includes a smart wearable device and a prompt server, the first user voice is obtained, the first user voice is semantically parsed and the parsing result is obtained; at least one task execution command is obtained according to the parsing result by using the generative artificial intelligence large language model; and the step of executing at least one operation corresponding to the at least one task execution command includes:

[0210] Acquire the first user's voice by using the built-in microphone of the smart wearable device, and send the first user's voice to the prompt server;

[0211] Sending the first user's voice to a voice-to-text server through the prompt server, so as to convert the first user's voice into a first text by using the voice-to-text server;

[0212] Through the prompt server, semantic analysis is performed on the first text returned by the speech-to-text server, and the at least one first prompt message is generated according to the analyzed semantics, and the at least one first prompt message is sent to the model server, so that the model server uses the generative artificial intelligence large language model to determine whether there is information that needs to be supplemented or confirmed according to the at least one first prompt message, wherein when the information that needs to be supplemented or confirmed does not exist, the model server obtains the at least one task execution command according to the at least one first prompt message through the generative artificial intelligence large language model, and sends the at least one task execution command to the prompt server; when the information that needs to be supplemented or confirmed exists, the model server obtains the at least one task execution command according to the at least one first prompt message through the generative artificial intelligence large language model, and sends the prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command to the prompt server;

[0213] Sending, through the prompt server, a control instruction to at least one device in the Internet of Things according to the at least one task execution command, so as to control the at least one device to perform the operation specified by the at least one task execution command, and generating corresponding text according to the prompt information, and sending the text to a text-to-speech server, so as to convert the text into speech by using the text-to-speech server;

[0214] Sending the voice returned by the text-to-speech server to the smart wearable device through the prompt server; and

[0215] The voice is played through the smart wearable device.

[0216] Optionally, in other embodiments of the present application, the smart wearable device system includes a smart wearable device, a smart mobile terminal, and a prompt server, the generative artificial intelligence large language model is configured in the smart mobile terminal, the first user voice is obtained, the first user voice is semantically parsed and the parsing result is obtained, the generative artificial intelligence large language model is used, at least one task execution command is obtained according to the parsing result, and the step of executing at least one operation corresponding to the at least one task execution command specifically includes:

[0217] Acquire the first user's voice through the smart wearable device, and send the first user's voice to the prompt server through a wireless network;

[0218] By means of the prompt server, the first user speech is converted into a first text by means of a speech-to-text engine, the first text is semantically parsed, at least one first prompt message is generated according to the parsed semantics, and the at least one first prompt message is sent to the smart wearable device through the wireless network;

[0219] Sending the at least one first prompt information to the smart mobile terminal via Bluetooth through the smart wearable device; and

[0220] Through the intelligent mobile terminal, using the generative artificial intelligence large language model, at least one task execution command is obtained according to the at least one first prompt information, and at least one operation corresponding to the at least one task execution command is executed.

[0221] Optionally, in other embodiments of the present application, the smart wearable device system includes a smart wearable device, a smart mobile terminal, and a prompt server, the generative artificial intelligence large language model is configured on the model server, the first user voice is obtained, the first user voice is semantically parsed and the parsing result is obtained, the generative artificial intelligence large language model is used, at least one task execution command is obtained according to the parsing result, and the steps of executing at least one operation corresponding to the at least one task execution command specifically include:

[0222] Acquire the first user's voice through the smart wearable device, and send the first user's voice to the prompt server through a wireless network;

[0223] By means of the prompt server, the first user speech is converted into a first text by means of a speech-to-text engine, the first text is semantically parsed, at least one first prompt message is generated according to the parsed semantics, and the at least one first prompt message is sent to the smart wearable device through the wireless network;

[0224] Sending the at least one first prompt information to the smart mobile terminal via Bluetooth through the smart wearable device;

[0225] Sending the at least one first prompt information to the model server through the smart mobile terminal, so that the model server uses the generative artificial intelligence large language model to obtain at least one task execution command according to the at least one first prompt information; and

[0226] The at least one task execution command sent by the model server is received through the intelligent mobile terminal, and at least one operation corresponding to the at least one task execution command is executed.

[0227] Optionally, in other embodiments of the present application, the smart wearable device system includes a smart wearable device, a smart mobile terminal, and a prompt server, the generative artificial intelligence large language model is configured on the model server, the first user voice is obtained, the first user voice is semantically parsed and the parsing result is obtained, the generative artificial intelligence large language model is used, at least one task execution command is obtained according to the parsing result, and the steps of executing at least one operation corresponding to the at least one task execution command specifically include:

[0228] Acquire the first user's voice through the smart wearable device, and send the first user's voice to the prompt server through a wireless network;

[0229] By means of the prompt server, the first user speech is converted into a first text by means of a speech-to-text engine, the first text is semantically parsed, at least one first prompt message is generated according to the parsed semantics, and the at least one first prompt message is sent to the smart wearable device through the wireless network;

[0230] The at least one first prompt information is sent to the model server through the wireless network by the smart wearable device, so that the model server uses the generative artificial intelligence large language model to obtain at least one task execution command according to the at least one first prompt information;

[0231] Receiving, by the smart wearable device, the at least one task execution command sent by the model server using a wireless network, and sending the at least one task execution command to the smart mobile terminal using Bluetooth; and

[0232] At least one operation corresponding to the at least one task execution command is executed through the intelligent mobile terminal.

[0233] The following will be combined Fig.15 and 16 The working mechanism of the above control method is explained. Fig.15 as well as Fig.16 As shown in boxes 401 to 413 in FIG, first, the user's voice will be converted into text by the "speech to text" engine. The user's voice in text format will be used to generate prompt information to guide the "generative artificial intelligence large language model" on how to analyze and reply to the user's voice. The prompt information is generated based on the semantics of the parsed text. For example, the prompt information will indicate the intention of the "generative artificial intelligence large language model" to reply to the user's voice. The "generative artificial intelligence large language model" generates a reply based on the prompt information.

[0234] like Fig.15 As shown, the reply may be to instruct the smart glasses system to ask the user for more information or to confirm the information, or it may be to instruct the corresponding device to perform an operation corresponding to the prompt information, such as: making a call, accessing a contact list, accessing a calendar, sending an email, sending a text message, searching the Internet, calling a network service, positioning and navigating, calling a mobile application SDK (such as iOS SDK, Android SDK), calling a third-party application SDK (such as OpenRice, Cathay), launching other applications, etc., or it may be to instruct the corresponding device to perform the corresponding operation while instructing the smart glasses system to ask the user for more information or to confirm the information.

[0235] like Fig.16 As shown in boxes 404 and 413 in , if the reply is to instruct to perform a corresponding operation, then the corresponding operation is performed. If the reply is to ask the user for information or to confirm with the user, the text of the prompt information containing the information that needs to be asked or confirmed is converted into voice and the voice is played. If the reply not only instructs to perform a corresponding operation but also instructs to ask the user for information or to confirm with the user, then the corresponding operation is performed first, and then it is confirmed whether the reply also instructs to ask the user for information or to confirm with the user, that is, whether the task is completed? If the reply also instructs to ask the user for information or to confirm with the user, it means that the task is not completed, then return to box 404: generate the corresponding prompt voice and play the voice.

[0236] Furthermore, if the reply indicates asking the user for more information, the user voice command is obtained again after playing the voice, and the process continues in this order until the end.

[0237] For example, if the user voice command is "I want to call Simon", the smart glasses system generates a first prompt message to inquire the generative artificial intelligence large language model about the intention of the user voice command. The generative artificial intelligence large language model indicates in the output reply that the intention is "make a call" and contains a first task execution command for obtaining Simon's phone number. According to the first task execution command, the smart glasses system performs an operation of obtaining Simon's phone number in the smartphone contact list and sends the operation result to the generative artificial intelligence large language model. If there is only one Simon in the contact list, the generative artificial intelligence large language model outputs a second task execution command so that the smart glasses system makes a call according to Simon's phone number in the contact list. If there is no Simon in the contact list, the generative artificial intelligence large language model outputs a reply containing a prompt message asking the user for Simon's phone number and a third task execution command, so that the smart glasses system asks the user Simon's phone number in the form of voice according to the reply, and performs a call operation according to the phone number answered by the user according to the third task execution command. If there are multiple Simons on the contact's phone, the generative artificial intelligence large language model output includes a prompt message to the user to confirm which Simon needs to be called and a reply to the fourth task execution command, so that the smart glasses system confirms which Simon needs to be called to the user in the form of voice based on the reply, and executes the call operation according to the phone number confirmed by the user based on the fourth task execution command.

[0238] For details not covered in this embodiment, please refer to the above Figures 1 to 13 The relevant description in the illustrated embodiment will not be repeated here.

[0239] In this embodiment, the smart wearable device system utilizes the generative artificial intelligence big language model to implement device control based on natural language voice commands, thereby improving the convenience of device control based on the smart wearable device system. In addition, due to the scalability and self-creativity of the generative artificial intelligence big language model, the intelligence and interactivity of the smart wearable device system can be further improved.

[0240] The present application also provides a non-transitory computer-readable storage medium, which may be disposed in the smart glasses or smart wearable devices in the above embodiments. The non-transitory computer-readable storage medium may be the above Fig.12The memory 206 in the illustrated embodiment. The computer-readable storage medium stores a computer program, which, when executed by the processor, implements the natural language command control method based on the generative artificial intelligence large language model described in the aforementioned embodiments. Furthermore, the computer-storable medium may also be a U disk, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk, or an optical disk, etc., which may store program codes.

[0241] In the several embodiments provided in this application, it should be understood that the disclosed smart glasses, control systems and control methods can be implemented in other ways. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the connections or direct connections or communication connections shown or discussed can be indirect connections or communication connections through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0242] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0243] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0244] The above is a description of the smart glasses, control system and control method provided in the present application. For technicians in this field, according to the ideas of the embodiments of the present application, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A natural language command control system based on a generative artificial intelligence large language model, characterized in that: The system includes a smart glasses system and a model server, wherein a generative artificial intelligence large language model is configured on the model server; The smart glasses system is used to obtain a first user voice, perform semantic analysis on the first user voice, generate at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the model server; The model server is used to obtain at least one task execution command and send it to the smart glasses system according to the at least one first prompt information sent by the smart glasses system through the generative artificial intelligence large language model; The smart glasses system is further configured to execute at least one operation corresponding to the at least one task execution command.

2. The control system according to claim 1, characterized in that: The smart glasses system includes smart glasses and a smart mobile terminal; The smart glasses are used to acquire the first user's voice through a built-in microphone and send it to the smart mobile terminal through Bluetooth; The smart mobile terminal is used to convert the first user speech into a first text through a speech-to-text engine and perform semantic analysis on the first text, generate the at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the model server; The model server is further configured to obtain, according to the at least one first prompt message sent by the smart mobile terminal, the at least one task execution command through the generative artificial intelligence large language model and send the command to the smart mobile terminal; The intelligent mobile terminal is further used to execute at least one operation corresponding to the at least one task execution command.

3. The control system according to claim 2, characterized in that: The intelligent mobile terminal is further used to generate a plurality of first prompt information according to the parsed semantics, and send the plurality of first prompt information and the order in which the semantics corresponding to each of the first prompt information appear in the first text to the model server; The model server is further used to obtain multiple task execution commands through the generative artificial intelligence large language model according to the multiple first prompt information and the sequence information sent by the smart mobile terminal, and send the multiple task execution commands and the execution sequence information of the multiple task execution commands to the smart mobile terminal, and the execution sequence corresponds to the sequence; The intelligent mobile terminal is further used to execute operations corresponding to each of the task execution commands according to the execution sequence information.

4. The control system according to claim 2, characterized in that: The intelligent mobile terminal is further used to generate multiple first prompt information according to the parsed semantics, and send the multiple first prompt information to the model server one by one according to the order in which the semantics corresponding to each first prompt information appear in the first text.

5. The control system according to claim 2, characterized in that: The model server is also used for: Determining, by means of the generative artificial intelligence large language model, whether there is information that needs to be supplemented or confirmed according to the at least one first prompt information; When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command through the generative artificial intelligence large language model according to the at least one first prompt information; When the information that needs to be supplemented or confirmed exists, a text containing prompt information of the information that needs to be supplemented or confirmed is generated through the generative artificial intelligence large language model, and the text is sent to the smart mobile terminal; Receiving the second prompt information sent by the smart mobile terminal, and determining whether there is the information that needs to be supplemented or confirmed based on all the received prompt information through the generative artificial intelligence large language model; When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command through the generative artificial intelligence large language model according to all the received prompt information; as well as When the information that needs to be supplemented or confirmed exists, return to the step of generating a text of prompt information containing the information that needs to be supplemented or confirmed by the generative artificial intelligence large language model, and sending the text to the smart mobile terminal; The smart mobile terminal is further used to convert the text into speech using a text-to-speech engine, and send the speech to the smart glasses; The smart glasses are further configured to receive the voice via the Bluetooth, play the voice via a built-in speaker, acquire the second user's voice via the microphone, and send the second user's voice to the smart mobile terminal via the Bluetooth; The smart mobile terminal is also used to use the speech-to-text engine to convert the second user's speech into a second text, perform semantic analysis on the second text, generate the second prompt information according to the semantics in the parsed second text, and send the second prompt information to the model server.

6. The control system according to claim 1, characterized in that: The smart glasses system includes smart glasses and a prompt server, and the control system also includes a speech-to-text server and a text-to-speech server; The smart glasses are used to obtain the first user's voice through a built-in microphone and send the first user's voice to the prompt server; The prompt server is used to send the first user voice to the voice-to-text server; The speech-to-text server is configured to convert the first user speech into a first text and send the first text to the prompt server; The prompt server is further used to perform semantic analysis on the first text, generate the at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the model server; The model server is further used to determine whether there is information that needs to be supplemented or confirmed based on the at least one first prompt information through the generative artificial intelligence large language model; When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command through the generative artificial intelligence large language model according to the at least one first prompt information, and sending the at least one task execution command to the prompt server; When the information that needs to be supplemented or confirmed exists, obtaining the at least one task execution command according to the at least one first prompt information through the generative artificial intelligence large language model, and sending the prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command to the prompt server; The prompt server is further configured to send a control instruction to at least one device in the Internet of Things according to the at least one task execution command to control the at least one device to perform the operation specified by the at least one task execution command, and to generate a corresponding text according to the prompt information, and send the text to the text-to-speech server; The text-to-speech server is used to convert the text into speech and send the speech to the prompt server; The prompt server is further used to send the voice to the smart glasses; The smart glasses are also used to play the voice through built-in speakers.

7. The control system according to claim 1, characterized in that: The smart glasses system includes smart glasses, a smart mobile terminal and a prompt server; The smart glasses are used to acquire the first user's voice through a built-in microphone and send the first user's voice to the smart mobile terminal; The intelligent mobile terminal is used to convert the first user voice into a first text and send the first text to the prompt server; The prompt server is further used to perform semantic analysis on the first text, generate the at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the model server; The model server is further used to determine whether there is information that needs to be supplemented or confirmed based on the at least one first prompt information through the generative artificial intelligence large language model; When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command through the generative artificial intelligence large language model according to the at least one first prompt information, and sending the at least one task execution command to the prompt server; When the information that needs to be supplemented or confirmed exists, obtaining the at least one task execution command according to the at least one first prompt information through the generative artificial intelligence large language model, and sending the prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command to the prompt server; The prompt server is further configured to send a control instruction to at least one device in the Internet of Things according to the at least one task execution command, so as to control the at least one device to perform the operation specified by the at least one task execution command, and to generate a corresponding text according to the prompt information, and send the text to the smart mobile terminal; The intelligent mobile terminal is further used to convert the text into speech and send the speech to the smart glasses; The smart glasses are also used to play the voice through built-in speakers.

8. The control system according to claim 1, characterized in that: The smart glasses system includes smart glasses and a smart mobile terminal, the control system also includes a prompt server, the prompt server is configured with a speech-to-text engine, and the smart mobile terminal is configured with a text-to-speech engine; The smart glasses are used to obtain the first user's voice through a built-in microphone, and send the first user's voice to the prompt server through a wireless network; The prompt server is configured to convert the first user speech into a first text through the speech-to-text engine and perform semantic analysis on the first text, generate the at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the smart mobile terminal; The smart mobile terminal is used to send the at least one first prompt information to the model server; The model server is further used to determine whether there is information that needs to be supplemented or confirmed based on the at least one first prompt information through the generative artificial intelligence large language model; When the information that needs to be supplemented or confirmed does not exist, the at least one task execution command is obtained through the generative artificial intelligence large language model according to the at least one first prompt information, and the at least one task execution command is sent to the smart mobile terminal; when the information that needs to be supplemented or confirmed exists, the at least one task execution command is obtained through the generative artificial intelligence large language model according to the at least one first prompt information, and the prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command are sent to the smart mobile terminal; The intelligent mobile terminal is further used to execute at least one operation corresponding to the at least one task execution command; The smart mobile terminal is further used to generate corresponding text according to the prompt information, convert the text into speech through the text-to-speech engine, and send the speech to the smart glasses; The smart glasses are also used to play the voice through built-in speakers.

9. The control system according to claim 1, characterized in that: The smart glasses system includes smart glasses and a smart mobile terminal, and the generative artificial intelligence large language model is configured on the smart mobile terminal; The smart glasses are used to acquire the first user's voice through a built-in microphone and send it to the smart mobile terminal through Bluetooth; The smart mobile terminal is used to convert the first user speech into a first text through a speech-to-text engine and perform semantic analysis on the first text, obtain at least one task execution command according to the analyzed semantics through the generative artificial intelligence large language model, and execute at least one operation corresponding to the at least one task execution command; The intelligent mobile terminal is further used to determine whether there is information that needs to be supplemented or confirmed based on the parsed semantics through the generative artificial intelligence large language model; When the information that needs to be supplemented or confirmed does not exist, the at least one task execution command is obtained through the generative artificial intelligence large language model according to the parsed semantics; when the information that needs to be supplemented or confirmed exists, a text containing prompt information of the information that needs to be supplemented or confirmed is generated through the generative artificial intelligence large language model, the text is converted into speech using a text-to-speech engine, and the speech is sent to the smart glasses; The smart glasses are further configured to receive the voice via the Bluetooth, play the voice via a built-in speaker, acquire the second user's voice via the microphone, and send the second user's voice to the smart mobile terminal via the Bluetooth; The smart mobile terminal is further used to use the speech-to-text engine to convert the second user's speech into a second text, perform semantic analysis on the second text, and determine whether the information that needs to be supplemented or confirmed exists according to the semantics in the parsed second text through the generative artificial intelligence large language model; When the information that needs to be supplemented or confirmed does not exist, the at least one task execution command is obtained through the generative artificial intelligence big language model according to all the parsed semantics; when the information that needs to be supplemented or confirmed exists, the step of generating a text of prompt information containing the information that needs to be supplemented or confirmed through the generative artificial intelligence big language model is returned to be executed.

10. A smart glasses for natural language commands based on a generative artificial intelligence large language model, characterized in that: The smart glasses include: a frame, temples, a microphone, a processor and a memory; The temple is connected to the frame, and the processor is electrically connected to the microphone and the memory; The memory stores one or more programs executable by the processor, wherein the one or more programs include multiple instructions, and the multiple instructions are used to: Acquiring a first user's voice through the microphone; Performing semantic analysis on the first user's speech; Obtain at least one task execution command based on the parsed semantics through a generative artificial intelligence large language model; and At least one operation corresponding to the at least one task execution command is executed.

11. The smart glasses according to claim 10, wherein: The generative artificial intelligence large language model is configured on a model server, the one or more programs further include a speech-to-text engine, the smart glasses further include a wireless communication component electrically connected to the processor, and the plurality of instructions are further used to: Converting the first user speech into a first text by the speech-to-text engine; Performing semantic analysis on the first text, and generating at least one first prompt information according to the analyzed semantics; Sending the at least one first prompt information to the model server through the wireless communication component, so as to obtain at least one task execution command according to the at least one first prompt information through the generative artificial intelligence large language model in the model server; as well as The at least one task execution command sent by the model server is received through the wireless communication component.

12. The smart glasses according to claim 10, wherein: The generative artificial intelligence large language model is configured on a model server, the smart glasses further include a wireless communication component electrically connected to the processor, and the plurality of instructions are further used to: The first user voice is sent to the model server through the wireless communication component, so that the first user voice is converted into a first text and the first text is semantically parsed through the speech-to-text engine in the model server, and the at least one task execution command is obtained according to the parsed semantics through the generative artificial intelligence large language model in the model server; as well as The at least one task execution command sent by the model server is received through the wireless communication component.

13. The smart glasses according to claim 12, wherein: The plurality of instructions are also used to: Receiving, via the wireless communication component, a plurality of the task execution commands and execution order information of each of the task execution commands sent by the model server; and According to the execution order information, operations corresponding to each of the task execution commands are executed.

14. The smart glasses according to claim 12, wherein: The model server sends the obtained multiple task execution commands to the smart glasses one by one, and the multiple instructions are also used to: After executing an operation corresponding to a task execution command, a notification message is sent to the model server, so that the model server sends the next task execution command to the smart glasses according to the notification message.

15. The smart glasses according to claim 12, wherein: The smart glasses further include a speaker electrically connected to the processor, and the plurality of instructions are further used to: Receiving, through the wireless communication component, a voice message containing prompt information that needs to be supplemented or confirmed, sent by the model server, and playing the voice message through the speaker; The second user voice is acquired through the microphone, and the second user voice is sent to the model server through the wireless communication component, so that the model server obtains the at least one task execution command according to the first user voice and the second user voice through the generative artificial intelligence large language model.

16. The smart glasses according to claim 10, wherein: The smart glasses also include a wireless communication component electrically connected to the processor, and the multiple instructions are also used to send control instructions to at least one device in the Internet of Things through the wireless communication component according to the at least one task execution command, so as to control the at least one device to perform the operation specified by the at least one task execution command.

17. The smart glasses according to claim 10, wherein: The smart glasses further include a wireless communication component electrically connected to the processor, the generative artificial intelligence large language model is configured on a model server, and the plurality of instructions are further used to: The first user voice is sent to the prompt server through the wireless communication component, so that the prompt server converts the first user voice into a first text through a speech-to-text engine and performs semantic analysis on the first text, generates at least one first prompt message according to the analyzed semantics, and sends the at least one first prompt message to the model server, wherein the model server obtains the at least one task execution command through the generative artificial intelligence large language model according to the at least one first prompt message and sends it to the prompt server; and The at least one task execution command sent by the prompt server is received through the wireless communication component.

18. The smart glasses according to claim 10, wherein: The smart glasses further include a wireless communication component electrically connected to the processor, the generative artificial intelligence large language model is configured on a model server, and the plurality of instructions are further used to: The first user voice is sent to a prompt server through the wireless communication component, so that the prompt server converts the first user voice into a first text through a speech-to-text engine and performs semantic analysis on the first text, and generates at least one first prompt information according to the analyzed semantics; Receiving, through the wireless communication component, the at least one first prompt message sent by the prompt server, and sending the at least one first prompt message to the model server, so that the model server obtains the at least one task execution command through the generative artificial intelligence large language model according to the at least one first prompt message; as well as The at least one task execution command sent by the model server is received through the wireless communication component.

19. A natural language command control method based on a generative artificial intelligence large language model, characterized in that: Applied to a smart wearable device system, the method comprises: Acquire a first user voice, perform semantic analysis on the first user voice and obtain a analysis result; Using the generative artificial intelligence large language model, according to the parsing result, obtain at least one task execution command; and At least one operation corresponding to the at least one task execution command is executed.

20. The control method according to claim 19, characterized in that: The smart wearable device system includes a smart wearable device and a smart mobile terminal, the generative artificial intelligence large language model is configured on a model server, the first user voice is obtained, the first user voice is semantically parsed and a parsing result is obtained, at least one task execution command is obtained according to the parsing result by using the generative artificial intelligence large language model, and at least one operation corresponding to the at least one task execution command is executed, including: Acquire the first user voice through the smart wearable device, convert the first user voice into a first text using a speech-to-text engine, perform semantic analysis on the first text, generate at least one first prompt message according to the analyzed semantics, and send the first prompt message to the smart mobile terminal through Bluetooth; Sending the at least one first prompt information to the model server through the smart mobile terminal, so as to obtain at least one task execution command according to the at least one first prompt information through the generative artificial intelligence large language model in the model server; and The at least one task execution command sent by the model server is received through the intelligent mobile terminal, and at least one operation corresponding to the at least one task execution command is executed.

21. The control method according to claim 19, characterized in that: The smart wearable device system includes a smart wearable device and a smart mobile terminal, the generative artificial intelligence large language model is configured on a model server, the first user voice is obtained, the first user voice is semantically parsed and a parsing result is obtained, at least one task execution command is obtained according to the parsing result by using the generative artificial intelligence large language model, and at least one operation corresponding to the at least one task execution command is executed, including: Acquire the first user's voice through the smart wearable device and send it to the smart mobile terminal via Bluetooth; By means of the smart mobile terminal, the first user speech is converted into a first text by using a speech-to-text engine and the first text is semantically parsed, at least one first prompt message is generated according to the parsed semantics, and the at least one first prompt message is sent to the model server, so as to obtain at least one task execution command according to the at least one first prompt message through the generative artificial intelligence large language model in the model server; and The at least one task execution command sent by the model server is received through the intelligent mobile terminal, and at least one operation corresponding to the at least one task execution command is executed.

22. The control method according to claim 19, characterized in that: The smart wearable device system includes a smart wearable device and a smart mobile terminal, the generative artificial intelligence large language model is configured in the smart mobile terminal, the first user voice is obtained, the first user voice is semantically parsed and a parsing result is obtained, at least one task execution command is obtained according to the parsing result by using the generative artificial intelligence large language model, and at least one operation corresponding to the at least one task execution command is executed, including: Acquire the first user's voice through the smart wearable device, and send it to the smart mobile terminal through Bluetooth; and Through the smart mobile terminal, a speech-to-text engine is used to convert the first user voice into a first text, and the first text is semantically parsed and a parsing result is obtained. The generative artificial intelligence large language model is used to obtain at least one task execution command based on the parsing result, and at least one operation corresponding to the at least one task execution command is executed.

23. The control method according to claim 21, characterized in that: The smart mobile terminal generates a plurality of first prompt messages according to the parsed semantics, and sends the plurality of first prompt messages and the sequence information of the semantics corresponding to each of the first prompt messages in the first text to the model server; the model server obtains a plurality of task execution commands through the generative artificial intelligence large language model according to the plurality of first prompt messages and the sequence information, and sends the plurality of task execution commands and the execution sequence information of the plurality of task execution commands to the smart mobile terminal, wherein the execution sequence corresponds to the sequence; The receiving, by the smart mobile terminal, the at least one task execution command sent by the model server, and executing at least one operation corresponding to the at least one task execution command further includes: The multiple task execution commands and the execution sequence information of the multiple task execution commands sent by the model server are received through the intelligent mobile terminal, and the operations corresponding to the respective task execution commands are performed according to the execution sequence information.

24. The control method according to claim 21, characterized in that: The smart mobile terminal generates multiple pieces of the first prompt information according to the parsed semantics, and sends the multiple pieces of the first prompt information to the model server one by one.

25. The control method according to claim 21, characterized in that: The model server determines whether there is information that needs to be supplemented or confirmed according to the at least one first prompt information through the generative artificial intelligence big language model; when there is no information that needs to be supplemented or confirmed, the model server obtains the at least one task execution command through the generative artificial intelligence big language model according to the at least one first prompt information; when there is information that needs to be supplemented or confirmed, the model server generates a text of the prompt information containing the information that needs to be supplemented or confirmed through the generative artificial intelligence big language model, and sends the text to the smart mobile terminal; The method further comprises: Receiving the text sent by the model server through the smart mobile terminal, converting the text into speech using a text-to-speech engine, and sending the speech to the smart glasses; Playing the voice through the smart glasses, acquiring the second user's voice, and sending the second user's voice to the smart mobile terminal through the Bluetooth; and Through the smart mobile terminal, using a speech-to-text engine, the second user voice is converted into a second text, the second text is semantically parsed, and second prompt information is generated according to the semantics in the parsed second text, and the second prompt information is sent to the model server, so that the model server uses the generative artificial intelligence big language model to determine whether there is the information that needs to be supplemented or confirmed based on all the received prompt information. When the information that needs to be supplemented or confirmed does not exist, the model server obtains the at least one task execution command through the generative artificial intelligence big language model based on all the received prompt information. When the information that needs to be supplemented or confirmed exists, the model server returns to execute the step of generating a text containing the prompt information of the information that needs to be supplemented or confirmed through the generative artificial intelligence big language model, and sending the text to the smart mobile terminal.

26. The control method according to claim 22, characterized in that: The semantically parsing the first text and obtaining a parsing result, using the generative artificial intelligence large language model to obtain at least one task execution command according to the parsing result, and executing at least one operation corresponding to the at least one task execution command includes: By means of the smart mobile terminal, semantic analysis is performed on the first text to obtain a first analysis result, and by means of the generative artificial intelligence large language model, it is determined whether there is information that needs to be supplemented or confirmed according to the first analysis result; When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command through the intelligent mobile terminal according to the first analysis result and using the generative artificial intelligence large language model; When there is the information that needs to be supplemented or confirmed, the smart mobile terminal uses the generative artificial intelligence large language model to generate a text containing the prompt information of the information that needs to be supplemented or confirmed, uses a text-to-speech engine to convert the text into speech, and sends the speech to the smart wearable device via Bluetooth; Play the voice through the smart wearable device, obtain the second user's voice, and send the second user's voice to the smart mobile terminal through the Bluetooth; By means of the smart mobile terminal, the second user's voice is converted into a second text by using a speech-to-text engine, the second text is semantically parsed and a second parsing result is obtained, and by means of the generative artificial intelligence large language model, it is determined whether the information that needs to be supplemented or confirmed exists according to the first parsing result and the second parsing result; When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command by the smart mobile terminal according to the first analysis result and the second analysis result and using the generative artificial intelligence large language model; and When the information that needs to be supplemented or confirmed exists, return to execute the step of using the generative artificial intelligence large language model to generate a text of prompt information containing the information that needs to be supplemented or confirmed.

27. The control method according to claim 19, characterized in that: The smart wearable system includes a smart wearable device and a prompt server, wherein the first user voice is obtained, semantic analysis is performed on the first user voice and an analysis result is obtained; and at least one task execution command is obtained according to the analysis result by using the generative artificial intelligence large language model; and executing at least one operation corresponding to the at least one task execution command, including: Acquiring the first user's voice by using the built-in microphone of the smart wearable device, and sending the first user's voice to the prompt server; Sending the first user voice to a voice-to-text server through the prompt server, so that the first user voice is converted into a first text by the voice-to-text server; and The prompt server performs semantic analysis on the first text returned by the speech-to-text server, generates at least one first prompt message according to the analyzed semantics, and sends the at least one first prompt message to the model server, so that the model server uses the generative artificial intelligence large language model to determine whether there is information that needs to be supplemented or confirmed according to the at least one first prompt message, wherein when there is no information that needs to be supplemented or confirmed, the model server obtains the at least one task execution command according to the at least one first prompt message through the generative artificial intelligence large language model, and sends the at least one task execution command to the prompt server; when there is information that needs to be supplemented or confirmed, the model server obtains the at least one task execution command according to the at least one first prompt message through the generative artificial intelligence large language model, and sends the prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command to the prompt server; Sending, by the prompt server, a control instruction to at least one device in the Internet of Things according to the at least one task execution command, so as to control the at least one device to perform the operation specified by the at least one task execution command, and generating corresponding text according to the prompt information, and sending the text to a text-to-speech server, so as to convert the text into speech by using the text-to-speech server; Sending the voice returned by the text-to-speech server to the smart wearable device through the prompt server; and The voice is played through the smart wearable device.

28. The control method according to claim 19, characterized in that: The smart wearable device system includes a smart wearable device, a smart mobile terminal, and a prompt server, the generative artificial intelligence large language model is configured in the smart mobile terminal, the first user voice is obtained, the first user voice is semantically parsed and the parsing result is obtained, the generative artificial intelligence large language model is used, at least one task execution command is obtained according to the parsing result, and at least one operation corresponding to the at least one task execution command is executed, including: Acquire the first user voice through the smart wearable device, and send the first user voice to the prompt server through a wireless network; By means of the prompt server, converting the first user speech into a first text by using a speech-to-text engine, performing semantic analysis on the first text, generating at least one first prompt message according to the analyzed semantics, and sending the at least one first prompt message to the smart wearable device through the wireless network; Sending the at least one first prompt information to the smart mobile terminal via Bluetooth through the smart wearable device; and Through the smart mobile terminal, using the generative artificial intelligence large language model, at least one task execution command is obtained according to the at least one first prompt information, and at least one operation corresponding to the at least one task execution command is executed.

29. The control method according to claim 19, characterized in that: The smart wearable device system includes a smart wearable device, a smart mobile terminal and a prompt server, the generative artificial intelligence large language model is configured on the model server, the first user voice is obtained, the first user voice is semantically parsed and the parsing result is obtained, the generative artificial intelligence large language model is used, at least one task execution command is obtained according to the parsing result, and at least one operation corresponding to the at least one task execution command is executed, including: Acquire the first user voice through the smart wearable device, and send the first user voice to the prompt server through a wireless network; By means of the prompt server, converting the first user speech into a first text by using a speech-to-text engine, performing semantic analysis on the first text, generating at least one first prompt message according to the analyzed semantics, and sending the at least one first prompt message to the smart wearable device through the wireless network; Sending the at least one first prompt information to the smart mobile terminal via Bluetooth through the smart wearable device; Sending the at least one first prompt information to the model server through the smart mobile terminal, so that the model server uses the generative artificial intelligence large language model to obtain at least one task execution command according to the at least one first prompt information; and The at least one task execution command sent by the model server is received through the intelligent mobile terminal, and at least one operation corresponding to the at least one task execution command is executed.

30. The control method according to claim 19, characterized in that: The smart wearable device system includes a smart wearable device, a smart mobile terminal and a prompt server, the generative artificial intelligence large language model is configured on the model server, the first user voice is obtained, the first user voice is semantically parsed and the parsing result is obtained, the generative artificial intelligence large language model is used, at least one task execution command is obtained according to the parsing result, and at least one operation corresponding to the at least one task execution command is executed, including: Acquire the first user voice through the smart wearable device, and send the first user voice to the prompt server through a wireless network; By means of the prompt server, converting the first user speech into a first text by using a speech-to-text engine, performing semantic analysis on the first text, generating at least one first prompt message according to the analyzed semantics, and sending the at least one first prompt message to the smart wearable device through the wireless network; Sending the at least one first prompt information to the model server through the wireless network by the smart wearable device, so that the model server uses the generative artificial intelligence large language model to obtain at least one task execution command according to the at least one first prompt information; Receiving, by the smart wearable device, the at least one task execution command sent by the model server using a wireless network, and sending the at least one task execution command to the smart mobile terminal using Bluetooth; and At least one operation corresponding to the at least one task execution command is executed through the intelligent mobile terminal.

Images