Artificial intelligence-based natural language command control system and method, and intelligent glasses

Abstract

An artificial intelligence-based natural language command control system and method, and intelligent glasses. The system comprises an intelligent glasses system and a model server; a generative artificial intelligence large language model is provided on the model server; the intelligent glasses system is used for acquiring first user voice, carrying out semantic parsing on the first user voice, generating at least one piece of first prompt information on the basis of a semantic meaning obtained by parsing, and sending the at least one piece of first prompt information to the model server; the model server is used for obtaining at least one task execution command on the basis of the at least one piece of first prompt information by means of the generative artificial intelligence large language model, and sending the task execution command to the intelligent glasses system; the intelligent glasses system is further used for executing at least one operation corresponding to the at least one task execution command. The present application can improve the convenience of device control based on the intelligent glasses system, and improve the intelligence and interactivity of intelligent glasses.

Description

Natural language command control system, method and smart glasses based on artificial intelligence

[0001] This application claims priority to the Chinese patent application filed with the Patent Office of the State Intellectual Property Office of China on July 14, 2023, with application number CN 202310875349.9 and entitled “Natural language command control system, method and smart glasses based on artificial intelligence”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of smart glasses technology, and in particular to an artificial intelligence-based natural language command control system, method, and smart glasses. Background Art

[0003] With the development of computer technology, smart glasses are becoming more and more popular. However, existing smart glasses are expensive and, in addition to their own functions as smart glasses, usually only have the functions of listening to music and making or receiving calls. The functions are relatively simple and the degree of intelligence is relatively low. Technical issues

[0004] The embodiments of the present application provide an artificial intelligence-based natural language command control system, method, and smart glasses, which are used to improve the convenience of device control based on the smart glasses system, and improve the intelligence and interactivity of the smart glasses. Technical Solutions

[0005] On one hand, an embodiment of the present application provides a natural language command control system based on a generative artificial intelligence large language model, the system comprising a smart glasses system and a model server, wherein the generative artificial intelligence large language model is configured on the model server;

[0006] The smart glasses system is configured to obtain a first user voice, perform semantic analysis on the first user voice, generate at least one first prompt message based on the analyzed semantics, and send the at least one first prompt message to the model server;

[0007] The model server is configured to obtain, by using the generative artificial intelligence large language model, at least one task execution command according to the at least one first prompt message sent by the smart glasses system, and send the command to the smart glasses system;

[0008] The smart glasses system is further configured to execute at least one operation corresponding to the at least one task execution command.

[0009] In one aspect, an embodiment of the present application further provides smart glasses for natural language commands based on a generative artificial intelligence large language model, the smart glasses comprising: a frame, temples, a microphone, a processor, and a memory;

[0010] The temple is connected to the frame, and the processor is electrically connected to the microphone and the memory;

[0011] The memory stores one or more programs executable by the processor, wherein the one or more programs include multiple instructions, and the multiple instructions are used to:

[0012] Acquiring a first user's voice through the microphone;

[0013] Performing semantic analysis on the first user's speech;

[0014] Obtain at least one task execution command based on the parsed semantics using a generative artificial intelligence language model; and

[0015] At least one operation corresponding to the at least one task execution command is executed.

[0016] In one aspect, an embodiment of the present application further provides a natural language command control method based on a generative artificial intelligence large language model, which is applied to a smart wearable device system. The method includes:

[0017] Acquire a first user voice, perform semantic analysis on the first user voice and obtain a parsing result;

[0018] Using the generative artificial intelligence large language model, obtain at least one task execution command according to the parsing result; and

[0019] At least one operation corresponding to the at least one task execution command is executed. Beneficial effects

[0020] In each embodiment of the present application, the smart glasses system utilizes a generative artificial intelligence large language model to implement device control based on natural language voice commands, thereby improving the convenience of device control based on the smart glasses system. In addition, due to the scalability and self-creativity of the generative artificial intelligence large language model, the intelligence and interactivity of the smart glasses system can be further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] FIG1 is a schematic diagram of the structure of a natural language command control system based on a generative artificial intelligence large language model provided by an embodiment of the present application;

[0023] FIG2 is a schematic diagram of an application scenario of the control system shown in FIG1 ;

[0024] FIG3 is a schematic diagram of the structure of a natural language command control system based on a generative artificial intelligence large language model provided by another embodiment of the present application;

[0025] FIG4 is a schematic diagram of a first system framework based on the control system shown in FIG1 and FIG3;

[0026] FIG5 is a schematic diagram of a second system framework based on the control system shown in FIG1 and FIG3;

[0027] FIG6 is a schematic diagram of a third system framework based on the control system shown in FIG1 and FIG3;

[0028] FIG7 is a schematic diagram of a fourth system framework based on the control system shown in FIG1 and FIG3;

[0029] FIG8 is a schematic diagram of a fifth system framework based on the control system shown in FIG1 and FIG3;

[0030] FIG9 is a schematic diagram of a sixth system framework based on the control system shown in FIG1 and FIG3;

[0031] FIG10 is a schematic diagram of a seventh system framework based on the control system shown in FIG1 and FIG3;

[0032] FIG11 is a schematic diagram of an eighth system framework based on the control system shown in FIG1 and FIG3 ;

[0033] FIG12 is a schematic diagram of the internal structure of smart glasses provided in one embodiment of the present application;

[0034] FIG13 is a schematic diagram of the external structure of smart glasses provided in one embodiment of the present application;

[0035] FIG14 is a flowchart of an implementation method of a natural language command control method based on a generative artificial intelligence large language model according to an embodiment of the present application;

[0036] 15 and 16 are schematic diagrams of the working mechanism of the control method shown in FIG. 14 . Modes for Carrying Out the Invention

[0037] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0038] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present application, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0039] Furthermore, the terms “first,” “second,” “third,” etc., are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0040] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meaning as in the context of the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning unless clearly defined in the various embodiments of the present application.

[0041] Referring to Figure 1 , which is a schematic diagram of a natural language command control system based on a generative artificial intelligence large language model according to an embodiment of the present application, the control system 100 includes: a smart glasses system 110 and a model server 120 .

[0042] Among them, the model server 120 can be a single server or a distributed server cluster composed of multiple servers, on which a generative artificial intelligence large language model (GAILLM) is configured. For ease of understanding, it is collectively referred to as a GAILLM server below.

[0043] Specifically, the smart glasses system 110 is configured to obtain a first user voice, perform semantic analysis on the first user voice, generate at least one first prompt message according to the analyzed semantics, and send the at least one first prompt message to the GAILLM server 120 .

[0044] The GAILLM server 120 is configured to obtain at least one task execution command and send the command to the smart glasses system 110 based on the at least one first prompt message sent by the smart glasses system 110 through the generative artificial intelligence large language model.

[0045] The smart glasses system 110 is further configured to execute at least one operation corresponding to the at least one task execution command.

[0046] The first user speech includes at least one user voice command. The smart glasses system 110 performs semantic analysis on the first user speech through natural language processing (NLP) and generates at least one first prompt message based on the analyzed semantics. The first prompt message includes the analyzed semantics. Specifically, the smart glasses system 110 may use each semantic as a prompt message, or, based on the association between the semantics, use multiple associated semantics as a single prompt message.

[0047] The generative artificial intelligence large language model can be, for example but not limited to: Open AI's ChatGPT, Google's Bard, and other models with similar functions. The generative artificial intelligence large language model is trained using a large number of semantics and corresponding task execution commands as samples. Optionally, the same task execution command can correspond to multiple similar semantics, and the same semantics can correspond to different task execution commands. The task execution command is used to instruct the target object to perform at least one target task. One of the target tasks is associated with at least one corresponding operation. The task execution command includes description information of the target task. The description information of the target task is used to indicate the content of the target task, such as which operations need to be performed.

[0048] For example, assuming that the first user voice contains the semantics of "I want to call Simon" or "I want to talk to Simon immediately", the generative artificial intelligence language model obtains the corresponding task execution command based on the semantics to instruct the smart glasses system 110 to perform the operation of calling Simon.

[0049] Furthermore, the task execution command may also include description information of the executor of each target task, such as the executor's name, type, functions, etc.

[0050] As shown in FIG2 , in one practical application, a smart glasses system 110 includes smart glasses 111. A user can wear smart glasses 111 and speak a first user voice. Smart glasses 111 capture the first user voice via a built-in microphone, perform semantic analysis on the first user voice, and use a built-in prompt generator to generate at least one first prompt message based on the parsed semantics. The message is then sent to GAILLM server 120 (for example, the parsed semantics can be sent as the first prompt message). GAILLM server 120 then uses GAILLM to obtain at least one task execution command based on the first prompt message. Each task execution command corresponds to at least one operation, which may include, but is not limited to, making a call, sending a message, sending an email, searching the internet, invoking network services, positioning and navigation, invoking a third-party software development kit (SDK) to execute tasks provided by the SDK, and controlling related devices in the Internet of Things.

[0051] The prompt generator can be a software module or a microcontroller configured with the software module, which is used to generate corresponding prompt information based on the semantics in the parsed user voice. The semantics are obtained by parsing each user voice, such as: the first prompt information and the second prompt information below.

[0052] Optionally, the prompt generator can also be configured in the GAILLM server 120. The smart glasses 111 send the parsed semantics to the GAILLM server 120. The GAILLM server 120 uses the prompt generator to generate at least one first prompt message and inputs the first prompt message into the generative artificial intelligence large language model.

[0053] Figure 3 is a schematic diagram of the structure of a natural language command control system based on a generative artificial intelligence large language model, provided in another embodiment of the present application. Optionally, in other embodiments of the present application, as shown in Figure 3 , a smart glasses system 110 includes: smart glasses 111 and a smart mobile terminal 112. Furthermore, a corresponding application (APP) is installed in either the smart glasses 111 or the smart mobile terminal 112. The smart glasses 111 and the smart mobile terminal 112 establish a data connection via Bluetooth and utilize the APP for data exchange.

[0054] Among them, the smart glasses 111 can be open smart glasses, and their specific structure can be found in the relevant descriptions in the embodiments shown in Figures 12 and 13 below.

[0055] Smart mobile terminals 112 may include, but are not limited to, cellular phones, smartphones, other wireless communication devices, personal digital assistants (PDAs), audio players, other media players, music recorders, video recorders, cameras, other media recorders, smart radios, laptop computers, personal digital assistants (PDAs), portable multimedia players (PMPs), Moving Picture Experts Group (MPEG-1 or MPEG-2) Audio Layer 3 (MP3) players, digital cameras, and smart wearable devices (such as smart watches and smart bracelets). Smart mobile terminals 112 may also be installed with Android, iOS, or other operating systems.

[0056] Specifically, the smart glasses 111 are configured to acquire the first user's voice through a built-in microphone and send the voice to the smart mobile terminal 112 via Bluetooth.

[0057] The smart mobile terminal 112 is configured to convert the first user's speech into first text using a speech-to-text engine, perform semantic analysis on the first text, generate at least one first prompt message based on the analyzed semantics, and send the at least one first prompt message to the GAILLM server 120. The speech-to-text engine can be configured in the smart mobile terminal 112 or in a cloud-based server, such as a speech-to-text server, a prompt server, or the GAILLM server 120. The smart mobile terminal 112 can convert the first user's speech into the corresponding first text by sending it to a server equipped with the speech-to-text engine. The smart mobile terminal 112 can also be configured with a prompt generator to generate the at least one first prompt message based on the analyzed semantics.

[0058] The GAILLM server 120 is further configured to obtain, based on the at least one first prompt message sent by the smart mobile terminal 112 , the at least one task execution command through the generative artificial intelligence large language model and send the command to the smart mobile terminal 112 .

[0059] The smart mobile terminal 112 is further configured to execute at least one operation corresponding to the at least one task execution command.

[0060] Optionally, in other embodiments of the present application, the smart mobile terminal 112 is also used to generate multiple first prompt messages based on the parsed semantics, and send the multiple first prompt messages and the order in which the semantics corresponding to each first prompt message appear in the first text to the GAILLM server 120.

[0061] The GAILLM server 120 is also used to obtain multiple task execution commands based on the multiple first prompt messages and the sequence information sent by the smart mobile terminal 112 through the generative artificial intelligence large language model, and send the multiple task execution commands and the execution sequence information of the multiple task execution commands to the smart mobile terminal 112, and the execution order corresponds to the sequence.

[0062] The intelligent mobile terminal 112 is further configured to execute operations corresponding to each task execution command according to the execution sequence information.

[0063] Optionally, in other embodiments of the present application, the smart mobile terminal 112 is also used to: generate multiple first prompt messages based on the parsed semantics, and send the multiple first prompt messages to the GAILLM server 120 one by one in the order in which the semantics corresponding to each first prompt message appear in the first text.

[0064] Specifically, after sending a first prompt message, if the smart mobile terminal 112 receives at least one corresponding task execution command sent by the GAILLM server 120, it will send the next first prompt message to the GAILLM server 120; or, after sending a first prompt message, the smart mobile terminal 112 receives at least one corresponding task execution command sent by the GAILLM server 120, and after executing the corresponding at least one task execution command, it will send the next first prompt message to the GAILLM server 120.

[0065] Optionally, in other implementations of the present application, based on the above embodiment, the GAILLM server 120 is further configured to:

[0066] Determining, by the generative artificial intelligence large language model, whether there is information that needs to be supplemented or confirmed based on the at least one first prompt information;

[0067] When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command through the generative artificial intelligence large language model according to the at least one first prompt information;

[0068] When there is information that needs to be supplemented or confirmed, a text containing prompt information of the information that needs to be supplemented or confirmed is generated by the generative artificial intelligence language model, and the text is sent to the smart mobile terminal 112;

[0069] Receive the second prompt information sent by the smart mobile terminal 112, and determine whether there is information that needs to be supplemented or confirmed based on all the received prompt information through the generative artificial intelligence language model;

[0070] When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command through the generative artificial intelligence large language model based on all received prompt information; and

[0071] When there is information that needs to be supplemented or confirmed, return to the step of executing the generative artificial intelligence language model to generate a text of prompt information containing the information that needs to be supplemented or confirmed, and send the text to the smart mobile terminal 112.

[0072] The smart mobile terminal 112 is further configured to convert the text into speech using a text-to-speech engine and transmit the speech to the smart glasses 111. The text-to-speech engine can be configured locally on the smart mobile terminal 112 or in a cloud server, such as a text-to-speech server, a prompt server, or the GAILLM server 120. The smart mobile terminal 112 can convert the text into speech by transmitting it to the server equipped with the text-to-speech engine.

[0073] The smart glasses 111 are further configured to receive the voice via the Bluetooth, play the voice via the built-in speaker, acquire the second user's voice via the microphone, and send the second user's voice to the smart mobile terminal 112 via the Bluetooth.

[0074] The smart mobile terminal 112 is further configured to utilize the speech-to-text engine to convert the second user speech into a second text, perform semantic analysis on the second text, generate the second prompt information based on the semantics in the analyzed second text, and send the second prompt information to the GAILLM server 120 .

[0075] Alternatively, when there is information that needs to be supplemented or confirmed, the reply generated by the generative artificial intelligence large language model may also include: at least one task execution command and text of prompt information containing information that needs to be supplemented or confirmed, so as to confirm or consult the user for more information while performing the corresponding operation, so as to obtain further task execution commands based on the user's reply, thereby further improving the flexibility and intelligence of task execution.

[0076] For example, if the first user voice message requests a reservation for Conference Room 1 at 3:00 PM, the generative AI language model can be used to obtain a first task execution command and a text message containing the prompt "Who are the meeting participants? Do I need to notify them?" The smart mobile terminal 112 converts this text into speech and sends it to the smart glasses 111 for playback. Simultaneously, based on the first task execution command, the cloud-based conference management server is used to schedule a reservation for Conference Room 1 at 3:00 PM.

[0077] After playing the voice message, smart glasses 111 obtain a second user voice message and send it to smart mobile terminal 112, such as a user voice message that reads, "Call Peter and Simon to inform them of a meeting in Conference Room 1 at 3 p.m." Smart mobile terminal 112 converts the second user voice message into a second text, performs semantic analysis on the second text, generates corresponding prompt information based on the parsed semantics, and sends the second text message to the generative artificial intelligence large language model. The generative artificial intelligence large language model generates a second task execution command based on the prompt information, causing smart mobile terminal 112 to execute the second task execution command, obtain Peter and Simon's phone numbers, generate a notification voice message such as, "Meeting in Conference Room 1 at 3 p.m., please attend on time," call Peter and Simon using the phone numbers, and play the notification voice message.

[0078] Optionally, in other embodiments of the present application, as shown in FIG3 , the smart glasses system 110 includes smart glasses 111 and a prompt server 113 , and the control system 100 further includes a speech-to-text server 130 and a text-to-speech server 140 .

[0079] The smart glasses 111 are configured to acquire the first user's voice through a built-in microphone and send the first user's voice to the prompt server 113 .

[0080] The prompt server 113 is configured to send the first user voice to the speech-to-text server 130 .

[0081] The speech-to-text server 130 is configured to convert the first user speech into a first text and send the first text to the prompt server 113 .

[0082] The prompt server 113 is further configured to perform semantic analysis on the first text, generate the at least one first prompt message based on the analyzed semantics, and send the at least one first prompt message to the GAILLM server 120. The prompt server 113 may be configured with a prompt generator to generate the at least one first prompt message based on the analyzed semantics.

[0083] The GAILLM server 120 is also used to determine whether there is information that needs to be supplemented or confirmed based on the at least one first prompt information through the generative artificial intelligence large language model; when the information that needs to be supplemented or confirmed does not exist, based on the at least one first prompt information, the at least one task execution command is obtained through the generative artificial intelligence large language model, and the at least one task execution command is sent to the prompt server 113; when the information that needs to be supplemented or confirmed exists, the at least one task execution command is obtained based on the at least one first prompt information through the generative artificial intelligence large language model, and the at least one task execution command and the prompt information of the information that needs to be supplemented or confirmed are sent to the prompt server 113.

[0084] The prompt server 113 is also used to send a control instruction to at least one device in the Internet of Things according to the at least one task execution command to control the at least one device to perform the operation specified by the at least one task execution command, and to generate corresponding text according to the prompt information and send the text to the text-to-speech server 140.

[0085] The text-to-speech server 140 is configured to convert the text into speech and send the speech to the prompt server 113 .

[0086] The prompt server 113 is also used to send the voice to the smart glasses 111.

[0087] The smart glasses 111 are further configured to play the voice through built-in speakers.

[0088] Optionally, as shown in Figure 3, in other embodiments of the present application, the smart glasses system 110 includes smart glasses 111, a smart mobile terminal 112 and a prompt server 113, and the control system 100 also includes a speech-to-text server 130 and a text-to-speech server 140.

[0089] The smart glasses 111 are configured to acquire the first user's voice through a built-in microphone and send the first user's voice to the smart mobile terminal 112 .

[0090] The smart mobile terminal 112 is configured to send the first user's voice to the voice-to-text server 130 .

[0091] The speech-to-text server 130 is configured to convert the first user speech into a first text and send the first text to the smart mobile terminal 112 .

[0092] The smart mobile terminal 112 is further configured to send the first text to the prompt server 113 .

[0093] The prompt server 113 is configured to perform semantic analysis on the first text, generate the at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the GAILLM server 120 .

[0094] The GAILLM server 120 is also used to determine whether there is information that needs to be supplemented or confirmed based on the at least one first prompt information through the generative artificial intelligence large language model; when the information that needs to be supplemented or confirmed does not exist, based on the at least one first prompt information, the at least one task execution command is obtained through the generative artificial intelligence large language model, and the at least one task execution command is sent to the prompt server 113; when the information that needs to be supplemented or confirmed exists, the at least one task execution command is obtained based on the at least one first prompt information through the generative artificial intelligence large language model, and the prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command are sent to the prompt server 113.

[0095] The prompt server 113 is also used to send a control instruction to at least one device in the Internet of Things according to the at least one task execution command to control the at least one device to perform the operation specified by the at least one task execution command, and to generate corresponding text according to the prompt information and send the text to the smart mobile terminal 112.

[0096] The smart mobile terminal 112 is further configured to send the text to the text-to-speech server 140 .

[0097] The text-to-speech server 140 is configured to convert the text into speech and send the speech to the smart mobile terminal 112 .

[0098] The smart mobile terminal 112 is also used to send the voice to the smart glasses 111.

[0099] The smart glasses 111 are further configured to play the voice through built-in speakers.

[0100] Optionally, in other embodiments of the present application, as shown in FIG3 , the smart glasses system 110 includes smart glasses 111 , a smart mobile terminal 112 and a prompt server 113 .

[0101] The smart glasses 111 are configured to acquire the first user's voice through a built-in microphone and send the first user's voice to the smart mobile terminal 112 .

[0102] The smart mobile terminal 112 is configured to convert the first user voice into a first text and send the first text to the prompt server 113 .

[0103] The prompt server 113 is further configured to perform semantic analysis on the first text, generate the at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the GAILLM server 120 .

[0104] The GAILLM server 120 is also used to determine whether there is information that needs to be supplemented or confirmed based on the at least one first prompt information through the generative artificial intelligence large language model; when the information that needs to be supplemented or confirmed does not exist, based on the at least one first prompt information, the at least one task execution command is obtained through the generative artificial intelligence large language model, and the at least one task execution command is sent to the prompt server 113; when the information that needs to be supplemented or confirmed exists, the at least one task execution command is obtained based on the at least one first prompt information through the generative artificial intelligence large language model, and the prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command are sent to the prompt server 113.

[0105] The prompt server 113 is also used to send a control instruction to at least one device in the Internet of Things according to the at least one task execution command to control the at least one device to perform the operation specified by the at least one task execution command, and to generate corresponding text according to the prompt information and send the text to the smart mobile terminal 112.

[0106] The smart mobile terminal 112 is further configured to convert the text into speech and send the speech to the smart glasses 111 .

[0107] The smart glasses 111 are further configured to play the voice through built-in speakers.

[0108] Optionally, in other embodiments of the present application, the smart glasses system 110 includes smart glasses 111 and a smart mobile terminal 112, and the control system 100 further includes a prompt server. The prompt server is configured with a speech-to-text engine, and the smart mobile terminal 112 is configured with a text-to-speech engine.

[0109] The smart glasses 111 are configured to acquire the first user's voice through a built-in microphone and send the first user's voice to the prompt server through a wireless network.

[0110] The prompt server is configured to convert the first user's speech into first text using the speech-to-text engine, perform semantic analysis on the first text, generate at least one first prompt message based on the analyzed semantics, and send the at least one first prompt message to the smart mobile terminal 112. In addition, the smart glasses 111 transmit the first user's speech and identification information of the smart mobile terminal 112 to the prompt server, so that the prompt server transmits the at least one first prompt message based on the identification information. Alternatively, the prompt server pre-stores association information between the smart glasses 111 and the smart mobile terminal 112. When transmitting the first user's speech, the smart glasses 111 also transmit their own identification information to the prompt server, so that the prompt server transmits the at least one first prompt message to the smart mobile terminal 112 based on the identification information and the association information. Alternatively, the prompt server may transmit the at least one first prompt message to the smart glasses 111, so that the smart glasses 111 forward the at least one first prompt message to the smart mobile terminal 112.

[0111] The smart mobile terminal 112 is configured to send the at least one first prompt information to the GAILLM server 120 .

[0112] The GAILLM server 120 is further configured to determine, using the generative artificial intelligence large language model and based on the at least one first prompt message, whether there is any information that needs to be supplemented or confirmed; if the information that needs to be supplemented or confirmed does not exist, obtain the at least one task execution command based on the at least one first prompt message using the generative artificial intelligence large language model, and send the at least one task execution command to the smart mobile terminal 112; if the information that needs to be supplemented or confirmed exists, obtain the at least one task execution command based on the at least one first prompt message using the generative artificial intelligence large language model, and send prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command to the smart mobile terminal 112. The prompt information of the information that needs to be supplemented or confirmed is generated by the GAILLM server 120 based on the determined information that needs to be supplemented or confirmed, and is configured to indicate or prompt what information needs to be supplemented or confirmed.

[0113] The smart mobile terminal 112 is further configured to execute at least one operation corresponding to the at least one task execution command.

[0114] The smart mobile terminal 112 is further configured to generate corresponding text according to the prompt information, convert the text into speech through the text-to-speech engine, and send the speech to the smart glasses 111. For example, the prompt information may be "Confirm which Peter in the contact list you are calling", and the corresponding text may be "Which Peter do you need to call? Peter (1), phone number 12345, Peter (2), phone number 23456, Peter (3), phone number 34567".

[0115] The smart glasses 111 are further configured to play the voice through a built-in speaker.

[0116] Optionally, in other embodiments of the present application, the smart glasses system 110 includes: smart glasses 111 and a smart mobile terminal 112, and the generative artificial intelligence large language model is configured on the smart mobile terminal 112.

[0117] The smart glasses 111 are configured to acquire the first user's voice through a built-in microphone and send the voice to the smart mobile terminal 112 via Bluetooth.

[0118] The smart mobile terminal 112 is configured to convert the first user's speech into first text using a speech-to-text engine and perform semantic parsing on the first text, obtain at least one task execution command based on the parsed semantics using the generative artificial intelligence large language model configured on the smart mobile terminal 112, and execute at least one operation corresponding to the at least one task execution command. The speech-to-text engine can be configured in the smart mobile terminal 112 or in a speech-to-text server. The smart mobile terminal 112 can send the first user's speech to the speech-to-text server 130 and use the speech-to-text server 130 to convert the first user's speech into the first text, or use the speech-to-text server 130 to convert the first user's speech into the first text and perform semantic parsing on the first text.

[0119] Furthermore, the smart mobile terminal 112 is also used to determine whether there is information that needs to be supplemented or confirmed based on the parsed semantics through the generative artificial intelligence big language model; when the information that needs to be supplemented or confirmed does not exist, the at least one task execution command is obtained through the generative artificial intelligence big language model based on the parsed semantics; when the information that needs to be supplemented or confirmed exists, the generative artificial intelligence big language model is used to generate a text of prompt information containing the information that needs to be supplemented or confirmed, and the text-to-speech engine is used to convert the text into speech, and the speech is sent to the smart glasses 111.

[0120] The smart glasses 111 are further configured to receive the voice via the Bluetooth, play the voice via the built-in speaker, acquire the second user's voice via the microphone, and send the second user's voice to the smart mobile terminal 112 via the Bluetooth.

[0121] The smart mobile terminal 112 is also used to use the speech-to-text engine to convert the second user's speech into a second text, perform semantic analysis on the second text, and determine whether there is information that needs to be supplemented or confirmed based on the semantics in the parsed second text (or all parsed semantics, such as the semantics in the first text and the semantics in the second text) through the generative artificial intelligence large language model; when the information that needs to be supplemented or confirmed does not exist, obtain the at least one task execution command through the generative artificial intelligence large language model based on all the parsed semantics; when the information that needs to be supplemented or confirmed exists, return to execute the step of generating a text of prompt information containing the information that needs to be supplemented or confirmed through the generative artificial intelligence large language model.

[0122] Furthermore, the smart mobile terminal 112 is also used to determine whether the smart mobile terminal 112 is locally configured with the generative artificial intelligence large language model based on the installation log or preset mark when receiving the first user voice. If the smart mobile terminal 112 is locally configured with the generative artificial intelligence large language model, the parsed semantics are input into the local generative artificial intelligence large language model to obtain at least one task execution command through the local generative artificial intelligence large language model; otherwise, at least one first prompt message is generated according to the parsed semantics, and the generated at least one first prompt message is sent to the GAILLM server 120.

[0123] Optionally, in other embodiments of the present application, the generative artificial intelligence large language model may also be configured in the smart glasses 111. The smart glasses 111 are also used to: obtain the first user voice, perform semantic analysis on the first user voice, input the parsed semantics into the generative artificial intelligence large language model to execute at least one task command through the generative artificial intelligence large language model, and execute at least one operation corresponding to the at least one task execution command. Specifically, the smart glasses 111 may use a local speech-to-text engine, or through a speech-to-text server 130, to convert the first user voice into a corresponding first text, and perform semantic analysis on the first text. Among them, the smart glasses 111 may be configured with a prompt generator for generating at least one first prompt message based on the parsed semantics, and the smart glasses 111 input the at least one first prompt message into the generative artificial intelligence large language model to execute at least one task command through the generative artificial intelligence large language model.

[0124] Furthermore, the smart glasses 111 are also used to: determine whether the generative artificial intelligence large language model is installed locally based on the installation log; when the generative artificial intelligence large language model is installed locally on the smart glasses 111, input the parsed semantics into the generative artificial intelligence large language model to execute at least one task command through the generative artificial intelligence large language model; when the generative artificial intelligence large language model is not installed locally on the smart glasses 111, generate at least one first prompt message based on the parsed semantics, and send the first prompt message to the GAILLM server 120 or the smart mobile terminal 112 to obtain the at least one task execution command through the generative artificial intelligence large language model on the GAILLM server 120 or the smart mobile terminal 112.

[0125] Alternatively, the smart glasses 111 are further used to: determine the installation location of the generative artificial intelligence large language model based on pre-stored model configuration information; when the generative artificial intelligence large language model is installed locally on the smart glasses 111, input the parsed semantics into the generative artificial intelligence large language model to execute at least one task command through the generative artificial intelligence large language model; when the generative artificial intelligence large language model is installed on the GAILLM server 120, generate at least one first prompt message based on the parsed semantics, and send the first prompt message to the GAILLM server 120, or forward the first prompt message to the GAILLM server 120 through the smart mobile terminal 112 to obtain the at least one task execution command through the generative artificial intelligence large language model on the GAILLM server 120; when the generative artificial intelligence large language model is installed on the smart mobile terminal 112, generate at least one first prompt message based on the parsed semantics, and send the first prompt message to the smart mobile terminal 112 to obtain the at least one task execution command through the generative artificial intelligence large language model on the smart mobile terminal 112.

[0126] Furthermore, in other embodiments of the present application, the smart glasses 111, the smart mobile terminal 112 or the prompt server 113 are also used to: after receiving the at least one task execution command, determine the executor of the target task based on the description information of the target task in the task execution command or the description information of the executor of the target task, and distribute each task execution command to the corresponding executor, so that each executor performs at least one operation associated with its corresponding task execution command.

[0127] See Figure 4, which is a schematic diagram of a first system framework based on the control system shown in Figures 1 and 3. As shown in Figure 4, the control system includes smart glasses 111, a smart mobile terminal 112 (e.g., a smartphone or smartwatch), and a GAILLM server 120. GAILLM is configured on the GAILLM server 120. The smart mobile terminal 112 is installed with an application (app) for wireless communication with the smart glasses 111 via, for example, Bluetooth. Optionally, the smart glasses 111 and the GAILLM server 120 wirelessly communicate via, for example, a 4G or 5G cellular network or Wi-Fi. The smart mobile terminal 112 and the GAILLM server 120 wirelessly communicate via, for example, a 4G or 5G cellular network or Wi-Fi.

[0128] Smart glasses 111 acquire a first user voice and transmit the first user voice to smart mobile terminal 112. Smart mobile terminal 112 converts the first user voice into first text using a speech-to-text engine, performs semantic analysis on the first text, generates at least one first prompt message based on the analyzed semantics, and transmits the at least one first prompt message to GAILLM server 120. GAILLM server 120 obtains at least one task execution command based on the at least one first prompt message through GAILLM and transmits the command to smart mobile terminal 112. Smart mobile terminal 112 executes at least one operation corresponding to the at least one task execution command.

[0129] See Figure 5, which is a schematic diagram of a second system framework based on the control system shown in Figures 1 and 3. As shown in Figure 5, the control system includes smart glasses 111 and a smartphone 112. GAILLM is embedded in smartphone 112. Smartphone 112 also has a mobile app installed for wireless communication with smart glasses 111 via Bluetooth.

[0130] A user's natural language voice (i.e., the voice command in audio format shown in FIG5 ), for example, "Call Peter," is transmitted via Bluetooth from smart glasses 11 to an app on smartphone 112. The app converts the user's voice into text using a speech-to-text engine and performs semantic analysis on the text. Based on the analyzed semantics, it generates a first prompt message, which is used to query the generative artificial intelligence large language model regarding the intent of the user's voice command. This first prompt message is then input into GAILLM, which generates a response. This response indicates "make a call" and includes a first task execution command. Based on this first task execution command, the app retrieves Peter's phone number from the smartphone's contact list and sends the result of this operation to GAILLM. This result may, for example, include all phone numbers associated with Peter listed in the contact list.

[0131] According to the operation result, if there is only one Peter in the contact list, GAILLM outputs a second task execution command to enable the APP to make a call based on Peter's phone number in the contact list.

[0132] If Peter does not exist in the contact list, GAILLM outputs a prompt message asking the user for Peter's phone number and a reply to the third task execution command, so that the APP generates a text containing the prompt message, such as "Please enter Peter's phone number", and converts the text into a synthesized speech in audio format through a text-to-speech engine and sends it to the smart glasses 111 for playback. Then the APP parses the user voice obtained by the smart glasses 111 to obtain the phone number answered by the user, and performs a call operation according to the phone number answered by the user according to the third task execution command.

[0133] If there are multiple Peters on the contact's phone number, such as Peter Zhang, Peter Wang, and Peter Li, GAILLM outputs a reply containing a prompt asking the user to confirm which Peter to call. This prompts the app to generate text containing the prompt. The text-to-speech engine converts this text into synthesized audio format and sends it to the smart glasses 111 for playback. After playing the synthesized audio, the smart glasses 111 capture the second user's voice (e.g., Peter Li) and send it to the app. The app converts this user's voice into text using the speech-to-text engine and performs semantic analysis on the text. Based on the analyzed semantics, it generates a second prompt message and inputs it into GAILLM. Based on this second prompt message, GAILLM determines the phone number of Peter Li to be called and outputs a fourth task instruction command, causing the app to execute the call according to the fourth task execution command.

[0134] See Figure 6, which is a schematic diagram of a third system framework based on the control system shown in Figures 1 and 3. As shown in Figure 6, the control system includes smart glasses 111, a smartphone 112, and a GAILLM server 120. GAILLM is deployed on the GAILLM server 120. Smartphone 112 also has a mobile app installed for wireless communication with smart glasses 111 via Bluetooth. This mobile app also communicates wirelessly with the GAILLM server 120 via Wi-Fi or a cellular network.

[0135] A user's natural language speech (i.e., the voice command in audio format in FIG6 ), for example, "Call Peter," is transmitted via Bluetooth from the smart glasses 11 to the app on the smartphone 112. The app converts the user's speech into text using a speech-to-text engine, performs semantic analysis on the text, generates prompt information based on the parsed semantics, and then sends this prompt information to the GAILLM server 120. The GAILLM server 120 inputs the prompt information into GAILLM, which generates a response and sends it to the app on the smartphone 112. The response includes prompt information for supplementary or confirmed information and a task execution command. The app generates text for the response, including the prompt information, and uses a text-to-speech engine to convert this text into synthesized audio format. The text is then played back by the smart glasses 111, prompting the user for more information or confirmation of the response. Simultaneously, the app performs at least one action specified by the task execution command in the response.

[0136] See Figure 7, which is a schematic diagram of a fourth system framework based on the control system shown in Figures 1 and 3. As shown in Figure 7, the control system includes smart glasses 111, a smartphone 112, and a cloud-based notification server 113 and GAILLM server 120. GAILLM is deployed on GAILLM server 120. Smartphone 112 also has a mobile app installed for wireless communication with smart glasses 111 via Bluetooth. This mobile app also communicates wirelessly with notification server 113 via Wi-Fi or a cellular network.

[0137] A natural language user voice (i.e., the voice command in audio format in FIG. 7 ), for example, "Call Peter," is transmitted from the smart glasses 11 to the mobile app on the smartphone 112 via Bluetooth. The mobile app converts the user voice into a first text using a speech-to-text engine and sends this first text (i.e., the voice command in text format in FIG. 7 ) to the prompt server 113. The prompt server 113 performs semantic analysis on the first text, generates prompt information based on the parsed semantics, and then sends this prompt information to the GAILLM server 120. The GAILLM server 120 inputs this prompt information into GAILLM to obtain a response, which it then sends to the prompt server 113. This response includes prompt information for information that requires supplementation or confirmation, as well as a task execution command. The prompt server 113 generates a second text (i.e., the text-based response in FIG. 7 ) for this response, containing the prompt information for supplementation or confirmation and the task execution command, and sends this second text to the mobile app. The mobile app converts the prompt information in the second text into synthesized audio using a text-to-speech engine and sends it to the smart glasses 111 for playback, asking the user for more information or confirmation of the response. At the same time, the mobile app executes at least one operation specified by the task execution command in the reply.

[0138] See Figure 8, which is a schematic diagram of a fifth system framework based on the control system shown in Figures 1 and 3. As shown in Figure 8, the control system includes smart glasses 111, a smartphone 112, and a cloud-based prompt server 113, a GAILLM server 120, a speech-to-text server 130, and a text-to-speech server 140. GAILLM is deployed within the GAILLM server 120. Smartphone 112 is installed with a mobile app for wireless communication with smart glasses 111 via Bluetooth. This mobile app also wirelessly communicates with prompt server 113, speech-to-text server 130, and text-to-speech server 140 via Wi-Fi or a cellular network.

[0139] A natural language user voice (i.e., the voice command in audio format in FIG8 ), for example, "Call Peter," is transmitted via Bluetooth from smart glasses 11 to the mobile app on smartphone 112. The mobile app converts the user voice into a first text via speech-to-text server 130 and sends the first text (i.e., the voice command in text format in FIG8 ) to the prompt server 113. The prompt server 113 performs semantic analysis on the first text, generates prompt information based on the parsed semantics, and then sends the prompt information to the GAILLM server 120. GAILLM server 120 inputs the prompt information into GAILLM to generate a response, which it then sends to the prompt server 113. The response includes prompt information for information that requires supplementation or confirmation, as well as a task execution command. The prompt server 113 generates a second text (i.e., the text response in FIG8 ) for the response, containing the prompt information for supplementation or confirmation and the task execution command, and sends the second text to the mobile app. The mobile app converts the prompt information in the second text into a synthesized audio format through the text-to-speech server 140 and sends it to the smart glasses 111 for playback, thereby asking the user for more information or confirming the reply. At the same time, the mobile app executes at least one operation specified by the task execution command in the reply.

[0140] Referring to FIG9 , FIG9 is a schematic diagram of a sixth system framework based on the control system shown in FIG1 and FIG3 . As shown in FIG9 , the control system includes: smart glasses 111 and a GAILLM server 120 in the cloud. GAILLM is configured in the GAILLM server 120. The smartphone 112 is installed with a speech-to-text engine, a text-to-speech engine, and a mobile phone app for wireless communication with the smart glasses 111 via Bluetooth. The difference between the system framework shown in FIG9 and the system framework shown in FIG8 is that in the system framework shown in FIG9 , the operations of speech-to-text, generating prompt information, and text-to-speech are all completed locally on the smartphone 112 by the mobile phone app in the smartphone 112.

[0141] See Figure 10, which is a schematic diagram of a seventh system framework based on the control system shown in Figures 1 and 3. As shown in Figure 10, the control system includes smart glasses 111, as well as a cloud-based notification server 113, a GAILLM server 120, a speech-to-text server 130, and a text-to-speech server 140. GAILLM is deployed within the GAILLM server 120. Smart glasses 111 communicate wirelessly with the notification server 113 via Wi-Fi or a cellular network.

[0142] A user wearing smart glasses 111 issues a voice command in natural language, for example, "Turn on the bedroom lights in 10 minutes." Smart glasses 111 capture the user command in the form of audio data and send it to a prompt server 113. Prompt server 113, via speech-to-text server 130, converts the audio data into a first text file, performs semantic analysis on the first text file, generates a prompt based on the parsed semantics, and then sends the prompt to GAILLM server 120. GAILLM server 120 inputs the prompt into GAILLM, which generates a response and sends it to prompt server 113. The response includes prompts for information that requires supplementation or confirmation, or a task execution command.

[0143] For example, when there is only one lamp in the bedroom, the reply includes a task execution command, so that the prompt server 113 executes an operation of sending a command to the bedroom lamp as an IoT device via WIFI or cellular network after 10 minutes according to the task execution command to control the bedroom lamp to turn on.

[0144] If there are multiple lights in the bedroom, the response includes a prompt to confirm or inquire about which light to turn on. Prompt server 113 generates text containing the prompt for the response. Text-to-speech server 140 converts the text into audio format. The converted audio is then sent to smart glasses 111 for playback, confirming or inquiring about which bedroom light to turn on. After playing the audio data, smart glasses 111 receive the user's voice in the form of audio data (e.g., "turn on the bedside lamp") and send it to prompt server 113. Prompt server 113, through speech-to-text server 130, converts the audio data into a second text file, performs semantic analysis on the second text file, generates prompt information based on the parsed semantics, and then sends the prompt information to GAILLM server 120. GAILLM server 120 inputs the prompt information into GAILLM to generate a response, which it then sends to prompt server 113. The reply includes a task execution command, so that the prompt server 113 executes an operation of sending a command to the bedside lamp as an IoT device via WIFI or cellular network 10 minutes later according to the task execution command to control the bedside lamp to turn on.

[0145] See Figure 11, which is a schematic diagram of an eighth system framework based on the control system shown in Figures 1 and 3. As shown in Figure 11, the control system includes smart glasses 111, a cloud-based notification server 113, and a GAILLM server 120. GAILLM is deployed on GAILLM server 120. A smartphone 112 is installed with a mobile app for wireless communication with smart glasses 111 via Bluetooth. This mobile app also wirelessly communicates with notification server 113 via Wi-Fi or a cellular network.

[0146] A user wearing smart glasses 111 issues a voice command in natural language, for example, "Turn on the bedroom light in 10 minutes." Smart glasses 111 capture the user command in the form of audio data and send it to smartphone 112. Smartphone 112 converts the user command into a text-based command and sends it to prompt server 113. Prompt server 113 performs semantic analysis on the text-based command, generates prompt information based on the parsed semantics, and then sends this prompt information to GAILLM server 120. GAILLM server 120 inputs this prompt information into GAILLM to generate a response, which it then sends to prompt server 113. This response includes prompt information for supplementary or confirmation information and a task execution command. Based on the task execution command, prompt server 113 executes an operation to send a command to the bedroom light, an IoT device, to turn on the light in 10 minutes. Simultaneously, prompt server 113 generates a text-based reply to the reply, including prompt information for supplementary or confirmation information, to be sent to the user, and sends this text-based reply to smartphone 112. The smartphone 112 converts the text-based reply to be sent to the user into voice and transmits the converted voice to the smart glasses 111 , which play the voice to confirm or consult the user for more information in the form of audio data.

[0147] In this embodiment, the smart glasses system utilizes a generative artificial intelligence large language model to implement device control based on natural language voice commands, thereby improving the convenience of device control based on the smart glasses system. In addition, due to the scalability and self-creativity of the generative artificial intelligence large language model, the intelligence and interactivity of the smart glasses system can be further improved.

[0148] Referring to FIG12 , FIG12 is a schematic diagram of the internal structure of the smart glasses provided in one embodiment of the present application. FIG13 is a schematic diagram of the external structure of the smart glasses provided in one embodiment of the present application. For ease of explanation, only the parts relevant to the embodiment of the present application are shown in the figure. As shown in FIG12 and FIG13 , the smart glasses 200 include: a frame 201, at least one temple 202, at least one microphone 203, at least one speaker 204, at least one processor 205, and at least one memory 206. FIG12 and FIG13 are only optimal examples. In actual applications, the smart glasses 200 may also have fewer or more components than those shown in FIG12 and FIG13 .

[0149] The frame 201 may be, for example, a front frame with lenses (eg, sunglass lenses, clear lenses, or corrective lenses). The at least one temple 202 may include, for example, a left temple 202A and a right temple 202B.

[0150] The temple 202 is connected to the frame 201, and the processor 205 is electrically connected to the microphone 203, the speaker 204, and the memory 206. The microphone 203, the speaker 204, the processor 205, and the memory 206 are disposed on at least one of the temples 202 and / or the frame 201. Preferably, the temple 202 is detachably connected to the frame 201.

[0151] The processor 205 includes a central processing unit (CPU) and a digital signal processor (DSP). The DSP is used to process the voice data acquired by the microphone 203. The CPU is preferably a microcontroller unit (MCU).

[0152] The memory 206 is a non-transitory memory, which may specifically include RAM (Random Access Memory) and a flash memory component. It stores one or more programs executable by the processor 205, each of which includes multiple instructions. The multiple instructions are used to acquire a first user's voice through the microphone 203, perform semantic analysis on the first user's voice, obtain at least one task execution command based on the parsed semantics using a generative artificial intelligence language model, and execute at least one operation corresponding to the at least one task execution command. The generative artificial intelligence language model may be configured locally on the smart glasses 200 or in other devices, such as a smart mobile terminal or other cloud server. The multiple instructions may obtain the at least one task execution command by inputting the parsed semantics into the local generative artificial intelligence language model or by sending the parsed semantics to another device configured with the generative artificial intelligence language model.

[0153] Optionally, in other embodiments of the present application, the generative artificial intelligence large language model is configured on a model server (the GAILLM server 120 in the control system as shown in Figure 1), the one or more programs also include a speech-to-text engine, and the smart glasses 200 also include a wireless communication component 207 electrically connected to the processor 205. The multiple instructions are also used to: convert the first user voice into a first text through the speech-to-text engine; perform semantic analysis on the first text, and generate at least one first prompt message based on the parsed semantics; send the at least one first prompt message to the model server through the wireless communication component 207, so as to obtain at least one task execution command based on the at least one first prompt message through the generative artificial intelligence large language model in the model server; and receive the at least one task execution command sent by the model server through the wireless communication component 207.

[0154] The wireless communication component 207 includes a wireless signal transceiver and its surrounding circuitry, and may be disposed within the inner cavity of the frame 201 and / or at least one of the temples 202. The wireless signal transceiver may utilize, but is not limited to, at least one of the following protocols for data transmission: WiFi (Wireless Fidelity), NFC (Near Field Communication), ZigBee, UWB (Ultra Wideband), RFID (Radio Frequency Identification), and cellular mobile communication protocols (e.g., 3G / 4G / 5G).

[0155] Optionally, in other embodiments of the present application, the generative artificial intelligence large language model is configured on a model server, and the smart glasses 200 also include a wireless communication component 207 electrically connected to the processor 205. The multiple instructions are also used to: send the first user voice to the model server through the wireless communication component 207, so that the first user voice is converted into a first text and the first text is semantically parsed through the speech-to-text engine in the model server, and the generative artificial intelligence large language model in the model server obtains the at least one task execution command according to the parsed semantics; and receive the at least one task execution command sent by the model server through the wireless communication component 207.

[0156] Optionally, the speech-to-text engine may be configured on the model server or on the speech-to-text server. The smart glasses 200 or the model server may also convert the first user's speech into the first text by interacting with data on the speech-to-text server.

[0157] Optionally, in other embodiments of the present application, the multiple instructions are also used to: receive the multiple task execution commands and execution sequence information of each task execution command sent by the model server through the wireless communication component 207; and execute operations corresponding to each task execution command according to the execution sequence information.

[0158] Optionally, in other embodiments of the present application, the model server sends the obtained multiple task execution commands to the smart glasses 200 one by one, and the multiple instructions are also used to: after executing the operation corresponding to a task execution command, send notification information to the model server, so that the model server sends the next task execution command to the smart glasses 200 according to the notification information.

[0159] Optionally, in other embodiments of the present application, the multiple instructions are also used to: receive a voice prompt message containing information that needs to be supplemented or confirmed, sent by the model server through the wireless communication component 207, and play the voice through the speaker; obtain a second user voice through the microphone 203, and send the second user voice to the model server through the wireless communication component 207, so that the model server obtains the at least one task execution command based on the first user voice and the second user voice through the generative artificial intelligence large language model.

[0160] Optionally, in other embodiments of the present application, the smart glasses 200 also include a wireless communication component 207 electrically connected to the processor 205, and the multiple instructions are also used to send control instructions to at least one device in the Internet of Things through the wireless communication component 207 according to the at least one task execution command, so as to control the at least one device to perform the operation specified by the at least one task execution command.

[0161] Optionally, in other embodiments of the present application, the smart glasses 200 also include a wireless communication component 207 electrically connected to the processor 205, and the generative artificial intelligence large language model is configured on the model server. The multiple instructions are also used to: send the first user voice to the prompt server through the wireless communication component 207, so that the prompt server converts the first user voice into a first text through a speech-to-text engine and performs semantic analysis on the first text, generates at least one first prompt message based on the parsed semantics, and sends the at least one first prompt message to the model server, wherein the model server obtains the at least one task execution command based on the at least one first prompt message through the generative artificial intelligence large language model and sends it to the prompt server; and receives the at least one task execution command sent by the prompt server through the wireless communication component 207.

[0162] Optionally, in other embodiments of the present application, the smart glasses 200 also include a wireless communication component 207 electrically connected to the processor 205, and the generative artificial intelligence large language model is configured on the model server, and the multiple instructions are also used to: send the first user voice to the prompt server through the wireless communication component 207, so that the prompt server converts the first user voice into a first text through a speech-to-text engine and performs semantic analysis on the first text, and generates at least one first prompt message based on the parsed semantics; receive the at least one first prompt message sent by the prompt server through the wireless communication component 207, and send the at least one first prompt message to the model server, so that the model server obtains the at least one task execution command based on the at least one first prompt message through the generative artificial intelligence large language model; and receive the at least one task execution command sent by the model server through the wireless communication component 207.

[0163] Optionally, in other embodiments of the present application, the generative artificial intelligence language model can also be configured in the smart glasses 200 or in a smart mobile terminal. The smart glasses 200 also include a Bluetooth component 208 electrically connected to the processor 205. The Bluetooth component 208 includes a Bluetooth signal transceiver and surrounding circuitry, and can be specifically disposed within the inner cavity of the frame 201 and / or at least one temple 202. The Bluetooth component 208 can be connected to a smart mobile terminal such as a smartphone or smartwatch for phone calls, music, and data communications.

[0164] The multiple instructions are also used to obtain at least one task execution command based on the parsed semantics through the generative artificial intelligence language model configured in the smart glasses 200.

[0165] The multiple instructions are also used to interact with the smart mobile terminal through the Bluetooth component 208, so as to forward the at least one first prompt information to the model server through the smart mobile terminal and receive the at least one task execution command sent by the model server, and obtain the at least one task execution command through the smart mobile terminal.

[0166] Optionally, in other embodiments of the present application, the smart glasses 200 further include an input device 209 electrically connected to the processor 205. The input device 209 includes a touch sensor and a button. The button may be a power button on the smart glasses 200.

[0167] Optionally, in other embodiments of the present application, the smart glasses 200 further include an indicator light and / or a buzzer electrically connected to the processor 205. The multiple instructions are further configured to output prompt information via the indicator light and / or the buzzer. The prompt information is configured to indicate the status of the smart glasses 200, including an operating state and an idle state. The operating state includes: a state in which voice pickup has started, a state in which voice pickup has ended, and a state in which voice processing has been performed. The indicator light may optionally be an LED (Light Emitting Diode).

[0168] Optionally, in other embodiments of the present application, the smart glasses 200 further include a data sensing component electrically connected to the processor 205, wherein the data sensing component includes at least one of a positioning component, an inertial measurement unit (IMU) sensor, a temperature sensor, a proximity sensor, a humidity sensor, an electronic compass, a timer, a camera, and a pedometer. The positioning component may be, but is not limited to, a positioning component based on the Global Positioning System (GPS) or Beidou satellites.

[0169] The plurality of instructions are further used to obtain the sensing data of the at least one device, and obtain the at least one task execution command based on the acquired sensing data and the parsed semantics through the generative artificial intelligence language model. Or,

[0170] The multiple instructions are also used to obtain sensory data from the at least one device, and to send the sensory data of the at least one device and the at least one prompt message to the model server or smart mobile terminal via the wireless communication component 207, so that the generative artificial intelligence large language model on the model server or the smart mobile terminal can obtain at least one task execution command based on the sensory data and the at least one prompt message. For example, if the first user's voice contains the semantics of "remind me when I am near the cinema", then the generative artificial intelligence large language model, based on the real-time positioning data sent by the smart glasses 200 and the first prompt message containing the semantics, can obtain a task execution command and send the task execution command to the smart glasses 200 when the user of the smart glasses 200 is about to arrive at the cinema, thereby instructing the smart glasses 200 to play the prompt voice message "You have arrived near the cinema".

[0171] Optionally, in other embodiments of the present application, the smart glasses 200 further include a voice biometrics module, which is used to identify the user of the smart glasses 200 using the acquired voiceprint of the user to enable the above-mentioned voice control function based on the smart glasses 200.

[0172] Furthermore, the smart glasses 200 also include a battery 210 for providing power support for the electronic components of the smart glasses 200, such as the microphone 203, the speaker 204, the processor 205, and the memory 206.

[0173] The electronic components of the smart glasses can be connected via a bus.

[0174] It should be noted that the relationships between the components of the aforementioned smart glasses can be either replacement or superposition. That is, a pair of smart glasses can be equipped with all of the components described in this embodiment, or, depending on the needs, selectively install a portion of the components. In the case of replacement, the smart glasses also include a peripheral connection interface, which can be, for example, at least one of a PS / 2 interface, a serial interface, a parallel interface, an IEEE1394 interface, and a USB (Universal Serial Bus) interface. The functionality of the replaced component can be implemented by a peripheral connected to the interface, such as an external speaker or sensor.

[0175] For details not covered in this embodiment, please refer to the relevant descriptions in the embodiments shown in Figures 1 to 3 and the following Figures 14 to 16, and will not be repeated here.

[0176] In this embodiment, the smart glasses utilize a generative artificial intelligence big language model to realize device control based on natural language voice commands, thereby improving the convenience of device control based on smart glasses. In addition, due to the scalability and self-creativity of the generative artificial intelligence big language model, the intelligence and interactivity of the smart glasses can be further improved.

[0177] See Figure 14, which is a flowchart illustrating an implementation of a natural language command control method based on a generative artificial intelligence large language model according to an embodiment of the present application. This method can be applied to a smart wearable device system, such as the smart glasses system 110 in the embodiments shown in Figures 1 and 3. The smart wearable device may include, but is not limited to, a smart helmet, smart headphones, smart earrings, a smart watch, and the smart glasses shown in Figures 12 and 13. As shown in Figure 14, the method includes the following steps:

[0178] S301: Acquire a first user voice, perform semantic analysis on the first user voice, and obtain a analysis result;

[0179] S302: using a generative artificial intelligence language model to obtain at least one task execution command according to the parsing result; and

[0180] S303: Execute at least one operation corresponding to the at least one task execution command.

[0181] Among them, the generative artificial intelligence large language model can be configured in any device in the smart wearable device system, such as the smart glasses 111, the smart mobile terminal 112 or the prompt server 113 in the smart glasses system 110 as shown in Figure 3, or it can also be configured in other cloud servers, such as the GAILLM server 120 shown in Figures 1 and 3.

[0182] Optionally, in other embodiments of the present application, the smart wearable device system includes a smart wearable device and a smart mobile terminal, the generative artificial intelligence large language model is configured on a model server, the steps of obtaining a first user voice, performing semantic parsing on the first user voice and obtaining a parsing result, using the generative artificial intelligence large language model, obtaining at least one task execution command according to the parsing result, and executing at least one operation corresponding to the at least one task execution command include:

[0183] Acquiring the first user's voice through the smart wearable device, converting the first user's voice into first text using a speech-to-text engine, performing semantic analysis on the first text, generating at least one first prompt message based on the analyzed semantics, and sending the first prompt message to the smart mobile terminal via Bluetooth;

[0184] Sending the at least one first prompt message to the model server through the smart mobile terminal, so as to obtain at least one task execution command according to the at least one first prompt message through the generative artificial intelligence large language model in the model server; and

[0185] The at least one task execution command sent by the model server is received through the smart mobile terminal, and at least one operation corresponding to the at least one task execution command is executed.

[0186] Optionally, in other embodiments of the present application, the smart wearable device system includes a smart wearable device and a smart mobile terminal, the generative artificial intelligence large language model is configured on a model server, the steps of obtaining a first user voice, performing semantic parsing on the first user voice and obtaining a parsing result, using the generative artificial intelligence large language model, obtaining at least one task execution command according to the parsing result, and executing at least one operation corresponding to the at least one task execution command include:

[0187] Acquire the first user's voice through the smart wearable device and send it to the smart mobile terminal via Bluetooth;

[0188] Using the smart mobile terminal, converting the first user's speech into first text using a speech-to-text engine and performing semantic analysis on the first text, generating at least one first prompt message based on the analyzed semantics, and sending the at least one first prompt message to the model server, so as to obtain at least one task execution command based on the at least one first prompt message through the generative artificial intelligence large language model in the model server; and

[0189] The at least one task execution command sent by the model server is received through the smart mobile terminal, and at least one operation corresponding to the at least one task execution command is executed.

[0190] Optionally, in other embodiments of the present application, the smart wearable device system includes a smart wearable device and a smart mobile terminal, the generative artificial intelligence large language model is configured in the smart mobile terminal, the steps of obtaining a first user voice, performing semantic parsing on the first user voice and obtaining a parsing result, using the generative artificial intelligence large language model, obtaining at least one task execution command according to the parsing result, and executing at least one operation corresponding to the at least one task execution command include:

[0191] Acquire the first user's voice through the smart wearable device and send it to the smart mobile terminal via Bluetooth; and

[0192] Through the smart mobile terminal, a speech-to-text engine is used to convert the first user's speech into a first text, and the first text is semantically parsed to obtain a parsing result. The generative artificial intelligence large language model is used to obtain at least one task execution command based on the parsing result, and at least one operation corresponding to the at least one task execution command is executed.

[0193] Optionally, in other embodiments of the present application, the smart mobile terminal generates a plurality of first prompt messages according to the parsed semantics, and sends the plurality of first prompt messages and information on the order in which the semantics corresponding to each of the first prompt messages appear in the first text to the model server. The model server obtains a plurality of task execution commands through the generative artificial intelligence large language model based on the plurality of first prompt messages and the order information, and sends the plurality of task execution commands and the execution order information of the plurality of task execution commands to the smart mobile terminal, and the execution order corresponds to the order;

[0194] The step of receiving, through the smart mobile terminal, the at least one task execution command sent by the model server and executing at least one operation corresponding to the at least one task execution command further includes:

[0195] The multiple task execution commands and the execution sequence information of the multiple task execution commands sent by the model server are received through the smart mobile terminal, and the operations corresponding to the respective task execution commands are executed according to the execution sequence information.

[0196] Optionally, in other embodiments of the present application, the smart mobile terminal generates multiple first prompt messages according to the parsed semantics, and sends the multiple first prompt messages to the model server one by one.

[0197] Optionally, in other embodiments of the present application, the model server determines, through the generative artificial intelligence large language model, based on the at least one first prompt information, whether there is information that needs to be supplemented or confirmed; when the information that needs to be supplemented or confirmed does not exist, the model server obtains the at least one task execution command through the generative artificial intelligence large language model based on the at least one first prompt information; when the information that needs to be supplemented or confirmed exists, the model server generates a text of the prompt information containing the information that needs to be supplemented or confirmed through the generative artificial intelligence large language model, and sends the text to the smart mobile terminal;

[0198] The method further includes:

[0199] Receiving the text sent by the model server through the smart mobile terminal, converting the text into speech using a text-to-speech engine, and sending the speech to the smart glasses;

[0200] Play the voice through the smart glasses, obtain the second user's voice, and send the second user's voice to the smart mobile terminal via the Bluetooth;

[0201] Through the smart mobile terminal, a speech-to-text engine is used to convert the second user's speech into a second text, and the second text is semantically parsed. A second prompt message is generated based on the semantics in the parsed second text, and the second prompt message is sent to the model server, so that the model server uses the generative artificial intelligence big language model to determine whether there is information that needs to be supplemented or confirmed based on all the received prompt information. When the information that needs to be supplemented or confirmed does not exist, the model server obtains the at least one task execution command based on all the received prompt information through the generative artificial intelligence big language model. When the information that needs to be supplemented or confirmed exists, the model server returns to the step of executing the text of the prompt message containing the information that needs to be supplemented or confirmed generated by the generative artificial intelligence big language model, and sending the text to the smart mobile terminal.

[0202] Optionally, in other embodiments of the present application, the steps of performing semantic parsing on the first text and obtaining a parsing result, using the generative artificial intelligence large language model to obtain at least one task execution command based on the parsing result, and executing at least one operation corresponding to the at least one task execution command include:

[0203] Performing semantic parsing on the first text and obtaining a first parsing result by the smart mobile terminal, and determining whether there is information that needs to be supplemented or confirmed based on the first parsing result by using the generative artificial intelligence large language model;

[0204] When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command by the smart mobile terminal according to the first analysis result and using the generative artificial intelligence large language model;

[0205] When there is information that needs to be supplemented or confirmed, the smart mobile terminal uses the generative artificial intelligence language model to generate a text prompt containing the information that needs to be supplemented or confirmed, uses a text-to-speech engine to convert the text into speech, and sends the speech to the smart wearable device via Bluetooth;

[0206] Play the voice through the smart wearable device, obtain the second user's voice, and send the second user's voice to the smart mobile terminal via the Bluetooth;

[0207] Using the smart mobile terminal, using a speech-to-text engine, converting the second user's speech into a second text, performing semantic parsing on the second text and obtaining a second parsing result, and using the generative artificial intelligence large language model to determine whether there is information that needs to be supplemented or confirmed based on the first parsing result and the second parsing result;

[0208] When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command by the smart mobile terminal based on the first analysis result and the second analysis result and using the generative artificial intelligence large language model; and

[0209] When there is information that needs to be supplemented or confirmed, return to the step of using the generative artificial intelligence large language model to generate a text of prompt information containing the information that needs to be supplemented or confirmed.

[0210] Optionally, in other embodiments of the present application, the smart wearable system includes a smart wearable device and a prompt server, and the steps of obtaining a first user voice, performing semantic parsing on the first user voice and obtaining a parsing result; using the generative artificial intelligence large language model to obtain at least one task execution command according to the parsing result; and executing at least one operation corresponding to the at least one task execution command include:

[0211] Acquire the first user's voice by using the built-in microphone of the smart wearable device, and send the first user's voice to the prompt server;

[0212] Sending the first user voice to a speech-to-text server through the prompt server, so as to convert the first user voice into a first text by using the speech-to-text server;

[0213] Through the prompt server, semantic parsing is performed on the first text returned by the speech-to-text server, and at least one first prompt message is generated according to the parsed semantics, and the at least one first prompt message is sent to the model server, so that the model server uses the generative artificial intelligence large language model to determine whether there is information that needs to be supplemented or confirmed based on the at least one first prompt message. When the information that needs to be supplemented or confirmed does not exist, the model server obtains the at least one task execution command based on the at least one first prompt message through the generative artificial intelligence large language model, and sends the at least one task execution command to the prompt server. When the information that needs to be supplemented or confirmed exists, the model server obtains the at least one task execution command based on the at least one first prompt message through the generative artificial intelligence large language model, and sends the prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command to the prompt server;

[0214] Sending, by the prompt server, a control instruction to at least one device in the Internet of Things according to the at least one task execution command, so as to control the at least one device to perform an operation specified by the at least one task execution command, and generating corresponding text according to the prompt information, and sending the text to a text-to-speech server, so that the text-to-speech server converts the text into speech;

[0215] Sending the voice returned by the text-to-speech server to the smart wearable device through the prompt server; and

[0216] Play the voice through the smart wearable device.

[0217] Optionally, in other embodiments of the present application, the smart wearable device system includes a smart wearable device, a smart mobile terminal, and a prompt server, the generative artificial intelligence large language model is configured in the smart mobile terminal, the steps of obtaining a first user voice, performing semantic parsing on the first user voice and obtaining a parsing result, using the generative artificial intelligence large language model, obtaining at least one task execution command according to the parsing result, and executing at least one operation corresponding to the at least one task execution command specifically include:

[0218] Acquire the first user's voice through the smart wearable device, and send the first user's voice to the prompt server through a wireless network;

[0219] converting the first user's speech into a first text by using a speech-to-text engine, performing semantic analysis on the first text, generating at least one first prompt message according to the analyzed semantics, and sending the at least one first prompt message to the smart wearable device via the wireless network;

[0220] Sending the at least one first prompt message to the smart mobile terminal via Bluetooth through the smart wearable device; and

[0221] Through the smart mobile terminal, using the generative artificial intelligence large language model, at least one task execution command is obtained according to the at least one first prompt information, and at least one operation corresponding to the at least one task execution command is executed.

[0222] Optionally, in other embodiments of the present application, the smart wearable device system includes a smart wearable device, a smart mobile terminal, and a prompt server, the generative artificial intelligence large language model is configured on the model server, the first user voice is obtained, the first user voice is semantically parsed and a parsing result is obtained, the generative artificial intelligence large language model is used to obtain at least one task execution command according to the parsing result, and the steps of executing at least one operation corresponding to the at least one task execution command specifically include:

[0223] Acquire the first user's voice through the smart wearable device, and send the first user's voice to the prompt server through a wireless network;

[0224] converting the first user's speech into a first text by using a speech-to-text engine, performing semantic analysis on the first text, generating at least one first prompt message according to the analyzed semantics, and sending the at least one first prompt message to the smart wearable device via the wireless network;

[0225] Sending the at least one first prompt message to the smart mobile terminal via Bluetooth through the smart wearable device;

[0226] Sending the at least one first prompt message to the model server through the smart mobile terminal, so that the model server uses the generative artificial intelligence large language model to obtain at least one task execution command according to the at least one first prompt message; and

[0227] The at least one task execution command sent by the model server is received through the smart mobile terminal, and at least one operation corresponding to the at least one task execution command is executed.

[0228] Optionally, in other embodiments of the present application, the smart wearable device system includes a smart wearable device, a smart mobile terminal, and a prompt server, the generative artificial intelligence large language model is configured on the model server, the first user voice is obtained, the first user voice is semantically parsed and a parsing result is obtained, the generative artificial intelligence large language model is used to obtain at least one task execution command according to the parsing result, and the steps of executing at least one operation corresponding to the at least one task execution command specifically include:

[0229] Acquire the first user's voice through the smart wearable device, and send the first user's voice to the prompt server through a wireless network;

[0230] converting the first user's speech into a first text by using a speech-to-text engine, performing semantic analysis on the first text, generating at least one first prompt message according to the analyzed semantics, and sending the at least one first prompt message to the smart wearable device via the wireless network;

[0231] Sending, by the smart wearable device, the at least one first prompt message to the model server via a wireless network, so that the model server uses the generative artificial intelligence large language model to obtain at least one task execution command according to the at least one first prompt message;

[0232] Receiving, by the smart wearable device, the at least one task execution command sent by the model server using a wireless network, and sending the at least one task execution command to the smart mobile terminal using Bluetooth; and

[0233] At least one operation corresponding to the at least one task execution command is executed through the smart mobile terminal.

[0234] The working mechanism of the above-mentioned control method will be described below in conjunction with Figures 15 and 16. As shown in Figures 15 and 16, boxes 401 to 413, first, the user's voice will be converted into text by the "speech-to-text" engine. The user's voice in text format will be used to generate prompt information to guide the "Generative Artificial Intelligence Large Language Model" on how to analyze and respond to the user's voice. The prompt information is generated based on the semantics of the parsed text. For example, the prompt information will indicate the intention of the "Generative Artificial Intelligence Large Language Model" to respond to the user's voice. The "Generative Artificial Intelligence Large Language Model" generates a response based on the prompt information.

[0235] As shown in FIG15 , the reply may be to instruct the smart glasses system to ask the user for more information or to confirm the information, or it may be to instruct the corresponding device to perform an operation corresponding to the prompt information, such as: making a call, accessing a contact list, accessing a calendar, sending an email, sending a text message, searching the Internet, calling a network service, positioning and navigating, calling a mobile application SDK (such as iOS SDK, Android SDK), calling a third-party application SDK (such as OpenRice, Cathay), launching other applications, etc., or it may be to instruct the corresponding device to perform the corresponding operation while instructing the smart glasses system to ask the user for more information or to confirm the information.

[0236] As shown in blocks 404 and 413 in Figure 16, if the response instructs the user to perform a corresponding operation, the corresponding operation is performed. If the response instructs the user to query information or confirm information, the text of the prompt information containing the information to be queried or confirmed is converted into speech and played. If the response not only instructs the user to perform a corresponding operation but also instructs the user to query information or confirm information, the corresponding operation is first performed, and then it is confirmed whether the response also instructs the user to query information or confirm information, that is, whether the task is completed. If the response also instructs the user to query information or confirm information, indicating that the task is not completed, the process returns to block 404: the corresponding prompt voice is generated and played.

[0237] Furthermore, if the reply indicates asking the user for more information, the user voice command is obtained again after playing the voice, and the process continues in this order until the end.

[0238] For example, if the user's voice command is "I want to call Simon," the smart glasses system generates a first prompt message to query the generative AI language model about the intent of the user's voice command. The generative AI language model outputs a response indicating the intent is "make a call" and includes a first task execution command to obtain Simon's phone number. Based on this first task execution command, the smart glasses system retrieves Simon's phone number from the smartphone's contact list and sends the result to the generative AI language model. If there is only one Simon in the contact list, the generative AI language model outputs a second task execution command, instructing the smart glasses system to call Simon's phone number. If there is no Simon in the contact list, the generative AI language model outputs a response message containing a prompt message asking the user for Simon's phone number and a third task execution command. Based on this response, the smart glasses system then performs a call based on the phone number provided by the user, based on the third task execution command. If there are multiple Simons on the contact's phone, the generative artificial intelligence large language model output includes a prompt message to the user to confirm which Simon needs to be called and a reply to the fourth task execution command, so that the smart glasses system confirms to the user which Simon needs to be called in the form of voice based on the reply, and executes the call operation according to the phone number confirmed by the user based on the fourth task execution command.

[0239] For details not covered in this embodiment, please refer to the relevant descriptions in the embodiments shown in Figures 1 to 13 above, and will not be repeated here.

[0240] In this embodiment, the smart wearable device system utilizes a generative artificial intelligence big language model to realize device control based on natural language voice commands, thereby improving the convenience of device control based on the smart wearable device system. In addition, due to the scalability and self-creativity of the generative artificial intelligence big language model, the intelligence and interactivity of the smart wearable device system can be further improved.

[0241] The present application also provides a non-transitory computer-readable storage medium, which can be provided in the smart glasses or smart wearable devices in the above-mentioned embodiments. The non-transitory computer-readable storage medium can be the memory 206 in the embodiment shown in Figure 12. The computer-readable storage medium stores a computer program, which, when executed by the processor, implements the natural language command control method based on the generative artificial intelligence large language model described in the above-mentioned embodiments. Furthermore, the computer-storable medium can also be various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk, or an optical disk.

[0242] In the several embodiments provided herein, it should be understood that the disclosed smart glasses, control systems, and control methods can be implemented in other ways. For example, multiple modules or components can be combined or integrated into another system, or some features can be omitted or not implemented. Furthermore, the connections or direct or communicative connections shown or discussed can be indirect or communicative connections through some interface, device, or module, and can be electrical, mechanical, or other forms.

[0243] It should be noted that for the aforementioned method embodiments, for ease of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0244] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0245] The above is a description of the smart glasses, control system, and control method provided in this application. For those skilled in the art, based on the ideas of the embodiments of this application, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A natural language command control system based on a generative artificial intelligence large language model, characterized in that: The system includes a smart glasses system and a model server, wherein a generative artificial intelligence large language model is configured on the model server; The smart glasses system is used to obtain a first user voice, perform semantic analysis on the first user voice, generate at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the model server; The model server is used to obtain at least one task execution command and send it to the smart glasses system according to the at least one first prompt information sent by the smart glasses system through the generative artificial intelligence large language model; The smart glasses system is further configured to execute at least one operation corresponding to the at least one task execution command.

2. The control system according to claim 1, characterized in that: The smart glasses system includes smart glasses and a smart mobile terminal; The smart glasses are used to acquire the first user's voice through a built-in microphone and send it to the smart mobile terminal through Bluetooth; The smart mobile terminal is used to convert the first user speech into a first text through a speech-to-text engine and perform semantic analysis on the first text, generate the at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the model server; The model server is further configured to obtain, according to the at least one first prompt message sent by the smart mobile terminal, the at least one task execution command through the generative artificial intelligence large language model and send the command to the smart mobile terminal; The intelligent mobile terminal is further used to execute at least one operation corresponding to the at least one task execution command.

3. The control system according to claim 2, characterized in that: The intelligent mobile terminal is further used to generate a plurality of first prompt information according to the parsed semantics, and send the plurality of first prompt information and the order in which the semantics corresponding to each of the first prompt information appear in the first text to the model server; The model server is further used to obtain multiple task execution commands through the generative artificial intelligence large language model according to the multiple first prompt information and the sequence information sent by the smart mobile terminal, and send the multiple task execution commands and the execution sequence information of the multiple task execution commands to the smart mobile terminal, and the execution sequence corresponds to the sequence; The intelligent mobile terminal is further used to execute operations corresponding to each of the task execution commands according to the execution sequence information.

4. The control system according to claim 2, characterized in that: The intelligent mobile terminal is further used to generate multiple first prompt information according to the parsed semantics, and send the multiple first prompt information to the model server one by one according to the order in which the semantics corresponding to each first prompt information appear in the first text.

5. The control system according to claim 2, characterized in that: The model server is also used for: Determining, by means of the generative artificial intelligence large language model, whether there is information that needs to be supplemented or confirmed according to the at least one first prompt information; When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command through the generative artificial intelligence large language model according to the at least one first prompt information; When the information that needs to be supplemented or confirmed exists, a text containing prompt information of the information that needs to be supplemented or confirmed is generated through the generative artificial intelligence large language model, and the text is sent to the smart mobile terminal; Receiving the second prompt information sent by the smart mobile terminal, and determining whether there is the information that needs to be supplemented or confirmed based on all the received prompt information through the generative artificial intelligence large language model; When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command through the generative artificial intelligence large language model according to all the received prompt information; as well as When the information that needs to be supplemented or confirmed exists, return to the step of generating a text of prompt information containing the information that needs to be supplemented or confirmed by the generative artificial intelligence large language model, and sending the text to the smart mobile terminal; The smart mobile terminal is further used to convert the text into speech using a text-to-speech engine, and send the speech to the smart glasses; The smart glasses are further configured to receive the voice via the Bluetooth, play the voice via a built-in speaker, acquire the second user's voice via the microphone, and send the second user's voice to the smart mobile terminal via the Bluetooth; The smart mobile terminal is also used to use the speech-to-text engine to convert the second user's speech into a second text, perform semantic analysis on the second text, generate the second prompt information according to the semantics in the parsed second text, and send the second prompt information to the model server.

6. The control system according to claim 1, characterized in that: The smart glasses system includes smart glasses and a prompt server, and the control system also includes a speech-to-text server and a text-to-speech server; The smart glasses are used to obtain the first user's voice through a built-in microphone and send the first user's voice to the prompt server; The prompt server is used to send the first user voice to the voice-to-text server; The speech-to-text server is configured to convert the first user speech into a first text and send the first text to the prompt server; The prompt server is further used to perform semantic analysis on the first text, generate the at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the model server; The model server is further used to determine whether there is information that needs to be supplemented or confirmed based on the at least one first prompt information through the generative artificial intelligence large language model; When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command through the generative artificial intelligence large language model according to the at least one first prompt information, and sending the at least one task execution command to the prompt server; When the information that needs to be supplemented or confirmed exists, obtaining the at least one task execution command according to the at least one first prompt information through the generative artificial intelligence large language model, and sending the prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command to the prompt server; The prompt server is further configured to send a control instruction to at least one device in the Internet of Things according to the at least one task execution command to control the at least one device to perform the operation specified by the at least one task execution command, and to generate a corresponding text according to the prompt information, and send the text to the text-to-speech server; The text-to-speech server is used to convert the text into speech and send the speech to the prompt server; The prompt server is further used to send the voice to the smart glasses; The smart glasses are also used to play the voice through built-in speakers.

7. The control system according to claim 1, characterized in that: The smart glasses system includes smart glasses, a smart mobile terminal and a prompt server; The smart glasses are used to acquire the first user's voice through a built-in microphone and send the first user's voice to the smart mobile terminal; The intelligent mobile terminal is used to convert the first user voice into a first text and send the first text to the prompt server; The prompt server is further used to perform semantic analysis on the first text, generate the at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the model server; The model server is further used to determine whether there is information that needs to be supplemented or confirmed based on the at least one first prompt information through the generative artificial intelligence large language model; When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command through the generative artificial intelligence large language model according to the at least one first prompt information, and sending the at least one task execution command to the prompt server; When the information that needs to be supplemented or confirmed exists, obtaining the at least one task execution command according to the at least one first prompt information through the generative artificial intelligence large language model, and sending the prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command to the prompt server; The prompt server is further configured to send a control instruction to at least one device in the Internet of Things according to the at least one task execution command, so as to control the at least one device to perform the operation specified by the at least one task execution command, and to generate a corresponding text according to the prompt information, and send the text to the smart mobile terminal; The intelligent mobile terminal is further used to convert the text into speech and send the speech to the smart glasses; The smart glasses are also used to play the voice through built-in speakers.

8. The control system according to claim 1, characterized in that: The smart glasses system includes smart glasses and a smart mobile terminal, the control system also includes a prompt server, the prompt server is configured with a speech-to-text engine, and the smart mobile terminal is configured with a text-to-speech engine; The smart glasses are used to obtain the first user's voice through a built-in microphone, and send the first user's voice to the prompt server through a wireless network; The prompt server is configured to convert the first user speech into a first text through the speech-to-text engine and perform semantic analysis on the first text, generate the at least one first prompt information according to the analyzed semantics, and send the at least one first prompt information to the smart mobile terminal; The smart mobile terminal is used to send the at least one first prompt information to the model server; The model server is further used to determine whether there is information that needs to be supplemented or confirmed based on the at least one first prompt information through the generative artificial intelligence large language model; When the information that needs to be supplemented or confirmed does not exist, the at least one task execution command is obtained through the generative artificial intelligence large language model according to the at least one first prompt information, and the at least one task execution command is sent to the smart mobile terminal; when the information that needs to be supplemented or confirmed exists, the at least one task execution command is obtained through the generative artificial intelligence large language model according to the at least one first prompt information, and the prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command are sent to the smart mobile terminal; The intelligent mobile terminal is further used to execute at least one operation corresponding to the at least one task execution command; The smart mobile terminal is further used to generate corresponding text according to the prompt information, convert the text into speech through the text-to-speech engine, and send the speech to the smart glasses; The smart glasses are also used to play the voice through built-in speakers.

9. The control system according to claim 1, characterized in that: The smart glasses system includes smart glasses and a smart mobile terminal, and the generative artificial intelligence large language model is configured on the smart mobile terminal; The smart glasses are used to acquire the first user's voice through a built-in microphone and send it to the smart mobile terminal through Bluetooth; The smart mobile terminal is used to convert the first user speech into a first text through a speech-to-text engine and perform semantic analysis on the first text, obtain at least one task execution command according to the analyzed semantics through the generative artificial intelligence large language model, and execute at least one operation corresponding to the at least one task execution command; The intelligent mobile terminal is further used to determine whether there is information that needs to be supplemented or confirmed based on the parsed semantics through the generative artificial intelligence large language model; When the information that needs to be supplemented or confirmed does not exist, the at least one task execution command is obtained through the generative artificial intelligence large language model according to the parsed semantics; when the information that needs to be supplemented or confirmed exists, a text containing prompt information of the information that needs to be supplemented or confirmed is generated through the generative artificial intelligence large language model, the text is converted into speech using a text-to-speech engine, and the speech is sent to the smart glasses; The smart glasses are further configured to receive the voice via the Bluetooth, play the voice via a built-in speaker, acquire the second user's voice via the microphone, and send the second user's voice to the smart mobile terminal via the Bluetooth; The smart mobile terminal is further used to use the speech-to-text engine to convert the second user's speech into a second text, perform semantic analysis on the second text, and determine whether the information that needs to be supplemented or confirmed exists according to the semantics in the parsed second text through the generative artificial intelligence large language model; When the information that needs to be supplemented or confirmed does not exist, the at least one task execution command is obtained through the generative artificial intelligence big language model according to all the parsed semantics; when the information that needs to be supplemented or confirmed exists, the step of generating a text of prompt information containing the information that needs to be supplemented or confirmed through the generative artificial intelligence big language model is returned to be executed.

10. A smart glasses for natural language commands based on a generative artificial intelligence large language model, characterized in that: The smart glasses include: a frame, temples, a microphone, a processor and a memory; The temple is connected to the frame, and the processor is electrically connected to the microphone and the memory; The memory stores one or more programs executable by the processor, wherein the one or more programs include multiple instructions, and the multiple instructions are used to: Acquiring a first user's voice through the microphone; Performing semantic analysis on the first user's speech; Obtain at least one task execution command based on the parsed semantics through a generative artificial intelligence large language model; and At least one operation corresponding to the at least one task execution command is executed.

11. The smart glasses according to claim 10, wherein: The generative artificial intelligence large language model is configured on a model server, the one or more programs further include a speech-to-text engine, the smart glasses further include a wireless communication component electrically connected to the processor, and the plurality of instructions are further used to: Converting the first user speech into a first text by the speech-to-text engine; Performing semantic analysis on the first text, and generating at least one first prompt information according to the analyzed semantics; Sending the at least one first prompt information to the model server through the wireless communication component, so as to obtain at least one task execution command according to the at least one first prompt information through the generative artificial intelligence large language model in the model server; as well as The at least one task execution command sent by the model server is received through the wireless communication component.

12. The smart glasses according to claim 10, wherein: The generative artificial intelligence large language model is configured on a model server, the smart glasses further include a wireless communication component electrically connected to the processor, and the plurality of instructions are further used to: The first user voice is sent to the model server through the wireless communication component, so that the first user voice is converted into a first text and the first text is semantically parsed through the speech-to-text engine in the model server, and the at least one task execution command is obtained according to the parsed semantics through the generative artificial intelligence large language model in the model server; as well as The at least one task execution command sent by the model server is received through the wireless communication component.

13. The smart glasses according to claim 12, wherein: The plurality of instructions are also used to: Receiving, via the wireless communication component, a plurality of the task execution commands and execution order information of each of the task execution commands sent by the model server; and According to the execution order information, operations corresponding to each of the task execution commands are executed.

14. The smart glasses according to claim 12, wherein: The model server sends the obtained multiple task execution commands to the smart glasses one by one, and the multiple instructions are also used to: After executing an operation corresponding to a task execution command, a notification message is sent to the model server, so that the model server sends the next task execution command to the smart glasses according to the notification message.

15. The smart glasses according to claim 12, wherein: The smart glasses further include a speaker electrically connected to the processor, and the plurality of instructions are further used to: Receiving, through the wireless communication component, a voice message containing prompt information that needs to be supplemented or confirmed, sent by the model server, and playing the voice message through the speaker; The second user voice is acquired through the microphone, and the second user voice is sent to the model server through the wireless communication component, so that the model server obtains the at least one task execution command according to the first user voice and the second user voice through the generative artificial intelligence large language model.

16. The smart glasses according to claim 10, wherein: The smart glasses also include a wireless communication component electrically connected to the processor, and the multiple instructions are also used to send control instructions to at least one device in the Internet of Things through the wireless communication component according to the at least one task execution command, so as to control the at least one device to perform the operation specified by the at least one task execution command.

17. The smart glasses according to claim 10, wherein: The smart glasses further include a wireless communication component electrically connected to the processor, the generative artificial intelligence large language model is configured on a model server, and the plurality of instructions are further used to: The first user voice is sent to the prompt server through the wireless communication component, so that the prompt server converts the first user voice into a first text through a speech-to-text engine and performs semantic analysis on the first text, generates at least one first prompt message according to the analyzed semantics, and sends the at least one first prompt message to the model server, wherein the model server obtains the at least one task execution command through the generative artificial intelligence large language model according to the at least one first prompt message and sends it to the prompt server; and The at least one task execution command sent by the prompt server is received through the wireless communication component.

18. The smart glasses according to claim 10, wherein: The smart glasses further include a wireless communication component electrically connected to the processor, the generative artificial intelligence large language model is configured on a model server, and the plurality of instructions are further used to: The first user voice is sent to a prompt server through the wireless communication component, so that the prompt server converts the first user voice into a first text through a speech-to-text engine and performs semantic analysis on the first text, and generates at least one first prompt information according to the analyzed semantics; Receiving, through the wireless communication component, the at least one first prompt message sent by the prompt server, and sending the at least one first prompt message to the model server, so that the model server obtains the at least one task execution command through the generative artificial intelligence large language model according to the at least one first prompt message; as well as The at least one task execution command sent by the model server is received through the wireless communication component.

19. A natural language command control method based on a generative artificial intelligence large language model, characterized in that: Applied to a smart wearable device system, the method comprises: Acquire a first user voice, perform semantic analysis on the first user voice and obtain a analysis result; Using the generative artificial intelligence large language model, according to the parsing result, obtain at least one task execution command; and At least one operation corresponding to the at least one task execution command is executed.

20. The control method according to claim 19, characterized in that: The smart wearable device system includes a smart wearable device and a smart mobile terminal, the generative artificial intelligence large language model is configured on a model server, the first user voice is obtained, the first user voice is semantically parsed and a parsing result is obtained, at least one task execution command is obtained according to the parsing result by using the generative artificial intelligence large language model, and at least one operation corresponding to the at least one task execution command is executed, including: Acquire the first user voice through the smart wearable device, convert the first user voice into a first text using a speech-to-text engine, perform semantic analysis on the first text, generate at least one first prompt message according to the analyzed semantics, and send the first prompt message to the smart mobile terminal through Bluetooth; Sending the at least one first prompt information to the model server through the smart mobile terminal, so as to obtain at least one task execution command according to the at least one first prompt information through the generative artificial intelligence large language model in the model server; and The at least one task execution command sent by the model server is received through the intelligent mobile terminal, and at least one operation corresponding to the at least one task execution command is executed.

21. The control method according to claim 19, characterized in that: The smart wearable device system includes a smart wearable device and a smart mobile terminal, the generative artificial intelligence large language model is configured on a model server, the first user voice is obtained, the first user voice is semantically parsed and a parsing result is obtained, at least one task execution command is obtained according to the parsing result by using the generative artificial intelligence large language model, and at least one operation corresponding to the at least one task execution command is executed, including: Acquire the first user's voice through the smart wearable device and send it to the smart mobile terminal via Bluetooth; By means of the smart mobile terminal, the first user speech is converted into a first text by using a speech-to-text engine and the first text is semantically parsed, at least one first prompt message is generated according to the parsed semantics, and the at least one first prompt message is sent to the model server, so as to obtain at least one task execution command according to the at least one first prompt message through the generative artificial intelligence large language model in the model server; and The at least one task execution command sent by the model server is received through the intelligent mobile terminal, and at least one operation corresponding to the at least one task execution command is executed.

22. The control method according to claim 19, characterized in that: The smart wearable device system includes a smart wearable device and a smart mobile terminal, the generative artificial intelligence large language model is configured in the smart mobile terminal, the first user voice is obtained, the first user voice is semantically parsed and a parsing result is obtained, at least one task execution command is obtained according to the parsing result by using the generative artificial intelligence large language model, and at least one operation corresponding to the at least one task execution command is executed, including: Acquire the first user's voice through the smart wearable device, and send it to the smart mobile terminal through Bluetooth; and Through the smart mobile terminal, a speech-to-text engine is used to convert the first user voice into a first text, and the first text is semantically parsed and a parsing result is obtained. The generative artificial intelligence large language model is used to obtain at least one task execution command based on the parsing result, and at least one operation corresponding to the at least one task execution command is executed.

23. The control method according to claim 21, characterized in that: The smart mobile terminal generates a plurality of first prompt messages according to the parsed semantics, and sends the plurality of first prompt messages and the sequence information of the semantics corresponding to each of the first prompt messages in the first text to the model server; the model server obtains a plurality of task execution commands through the generative artificial intelligence large language model according to the plurality of first prompt messages and the sequence information, and sends the plurality of task execution commands and the execution sequence information of the plurality of task execution commands to the smart mobile terminal, wherein the execution sequence corresponds to the sequence; The receiving, by the smart mobile terminal, the at least one task execution command sent by the model server, and executing at least one operation corresponding to the at least one task execution command further includes: The multiple task execution commands and the execution sequence information of the multiple task execution commands sent by the model server are received through the intelligent mobile terminal, and the operations corresponding to the respective task execution commands are performed according to the execution sequence information.

24. The control method according to claim 21, characterized in that: The smart mobile terminal generates multiple pieces of the first prompt information according to the parsed semantics, and sends the multiple pieces of the first prompt information to the model server one by one.

25. The control method according to claim 21, characterized in that: The model server determines whether there is information that needs to be supplemented or confirmed according to the at least one first prompt information through the generative artificial intelligence big language model; when there is no information that needs to be supplemented or confirmed, the model server obtains the at least one task execution command through the generative artificial intelligence big language model according to the at least one first prompt information; when there is information that needs to be supplemented or confirmed, the model server generates a text of the prompt information containing the information that needs to be supplemented or confirmed through the generative artificial intelligence big language model, and sends the text to the smart mobile terminal; The method further comprises: Receiving the text sent by the model server through the smart mobile terminal, converting the text into speech using a text-to-speech engine, and sending the speech to the smart glasses; Playing the voice through the smart glasses, acquiring the second user's voice, and sending the second user's voice to the smart mobile terminal through the Bluetooth; and Through the smart mobile terminal, using a speech-to-text engine, the second user voice is converted into a second text, the second text is semantically parsed, and second prompt information is generated according to the semantics in the parsed second text, and the second prompt information is sent to the model server, so that the model server uses the generative artificial intelligence big language model to determine whether there is the information that needs to be supplemented or confirmed based on all the received prompt information. When the information that needs to be supplemented or confirmed does not exist, the model server obtains the at least one task execution command through the generative artificial intelligence big language model based on all the received prompt information. When the information that needs to be supplemented or confirmed exists, the model server returns to execute the step of generating a text containing the prompt information of the information that needs to be supplemented or confirmed through the generative artificial intelligence big language model, and sending the text to the smart mobile terminal.

26. The control method according to claim 22, characterized in that: The semantically parsing the first text and obtaining a parsing result, using the generative artificial intelligence large language model to obtain at least one task execution command according to the parsing result, and executing at least one operation corresponding to the at least one task execution command includes: By means of the smart mobile terminal, semantic analysis is performed on the first text to obtain a first analysis result, and by means of the generative artificial intelligence large language model, it is determined whether there is information that needs to be supplemented or confirmed according to the first analysis result; When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command through the intelligent mobile terminal according to the first analysis result and using the generative artificial intelligence large language model; When there is the information that needs to be supplemented or confirmed, the smart mobile terminal uses the generative artificial intelligence large language model to generate a text containing the prompt information of the information that needs to be supplemented or confirmed, uses a text-to-speech engine to convert the text into speech, and sends the speech to the smart wearable device via Bluetooth; Play the voice through the smart wearable device, obtain the second user's voice, and send the second user's voice to the smart mobile terminal through the Bluetooth; By means of the smart mobile terminal, the second user's voice is converted into a second text by using a speech-to-text engine, the second text is semantically parsed and a second parsing result is obtained, and by means of the generative artificial intelligence large language model, it is determined whether the information that needs to be supplemented or confirmed exists according to the first parsing result and the second parsing result; When the information that needs to be supplemented or confirmed does not exist, obtaining the at least one task execution command by the smart mobile terminal according to the first analysis result and the second analysis result and using the generative artificial intelligence large language model; and When the information that needs to be supplemented or confirmed exists, return to execute the step of using the generative artificial intelligence large language model to generate a text of prompt information containing the information that needs to be supplemented or confirmed.

27. The control method according to claim 19, characterized in that: The smart wearable system includes a smart wearable device and a prompt server, wherein the first user voice is obtained, semantic analysis is performed on the first user voice and an analysis result is obtained; and at least one task execution command is obtained according to the analysis result by using the generative artificial intelligence large language model; and executing at least one operation corresponding to the at least one task execution command, including: Acquiring the first user's voice by using the built-in microphone of the smart wearable device, and sending the first user's voice to the prompt server; Sending the first user voice to a voice-to-text server through the prompt server, so that the first user voice is converted into a first text by the voice-to-text server; and The prompt server performs semantic analysis on the first text returned by the speech-to-text server, generates at least one first prompt message according to the analyzed semantics, and sends the at least one first prompt message to the model server, so that the model server uses the generative artificial intelligence large language model to determine whether there is information that needs to be supplemented or confirmed according to the at least one first prompt message, wherein when there is no information that needs to be supplemented or confirmed, the model server obtains the at least one task execution command according to the at least one first prompt message through the generative artificial intelligence large language model, and sends the at least one task execution command to the prompt server; when there is information that needs to be supplemented or confirmed, the model server obtains the at least one task execution command according to the at least one first prompt message through the generative artificial intelligence large language model, and sends the prompt information of the information that needs to be supplemented or confirmed and the at least one task execution command to the prompt server; Sending, by the prompt server, a control instruction to at least one device in the Internet of Things according to the at least one task execution command, so as to control the at least one device to perform the operation specified by the at least one task execution command, and generating corresponding text according to the prompt information, and sending the text to a text-to-speech server, so as to convert the text into speech by using the text-to-speech server; Sending the voice returned by the text-to-speech server to the smart wearable device through the prompt server; and The voice is played through the smart wearable device.

28. The control method according to claim 19, characterized in that: The smart wearable device system includes a smart wearable device, a smart mobile terminal, and a prompt server, the generative artificial intelligence large language model is configured in the smart mobile terminal, the first user voice is obtained, the first user voice is semantically parsed and the parsing result is obtained, the generative artificial intelligence large language model is used, at least one task execution command is obtained according to the parsing result, and at least one operation corresponding to the at least one task execution command is executed, including: Acquire the first user voice through the smart wearable device, and send the first user voice to the prompt server through a wireless network; By means of the prompt server, converting the first user speech into a first text by using a speech-to-text engine, performing semantic analysis on the first text, generating at least one first prompt message according to the analyzed semantics, and sending the at least one first prompt message to the smart wearable device via the wireless network; Sending the at least one first prompt information to the smart mobile terminal via Bluetooth through the smart wearable device; and Through the intelligent mobile terminal, using the generative artificial intelligence large language model, at least one task execution command is obtained according to the at least one first prompt information, and at least one operation corresponding to the at least one task execution command is executed.

29. The control method according to claim 19, characterized in that: The smart wearable device system includes a smart wearable device, a smart mobile terminal and a prompt server, the generative artificial intelligence large language model is configured on the model server, the first user voice is obtained, the first user voice is semantically parsed and the parsing result is obtained, the generative artificial intelligence large language model is used, at least one task execution command is obtained according to the parsing result, and at least one operation corresponding to the at least one task execution command is executed, including: Acquire the first user voice through the smart wearable device, and send the first user voice to the prompt server through a wireless network; By means of the prompt server, converting the first user speech into a first text by using a speech-to-text engine, performing semantic analysis on the first text, generating at least one first prompt message according to the analyzed semantics, and sending the at least one first prompt message to the smart wearable device via the wireless network; Sending the at least one first prompt information to the smart mobile terminal via Bluetooth through the smart wearable device; Sending the at least one first prompt information to the model server through the smart mobile terminal, so that the model server uses the generative artificial intelligence large language model to obtain at least one task execution command according to the at least one first prompt information; and The at least one task execution command sent by the model server is received through the intelligent mobile terminal, and at least one operation corresponding to the at least one task execution command is executed.

30. The control method according to claim 19, characterized in that: The smart wearable device system includes a smart wearable device, a smart mobile terminal and a prompt server, the generative artificial intelligence large language model is configured on the model server, the first user voice is obtained, the first user voice is semantically parsed and the parsing result is obtained, the generative artificial intelligence large language model is used, at least one task execution command is obtained according to the parsing result, and at least one operation corresponding to the at least one task execution command is executed, including: Acquire the first user voice through the smart wearable device, and send the first user voice to the prompt server through a wireless network; By means of the prompt server, converting the first user speech into a first text by using a speech-to-text engine, performing semantic analysis on the first text, generating at least one first prompt message according to the analyzed semantics, and sending the at least one first prompt message to the smart wearable device via the wireless network; Sending the at least one first prompt information to the model server through the wireless network by the smart wearable device, so that the model server uses the generative artificial intelligence large language model to obtain at least one task execution command according to the at least one first prompt information; Receiving, by the smart wearable device, the at least one task execution command sent by the model server using a wireless network, and sending the at least one task execution command to the smart mobile terminal using Bluetooth; and At least one operation corresponding to the at least one task execution command is executed through the intelligent mobile terminal.

Images