Electronic system and control method thereof

Abstract

An electronic system and a control method thereof are provided. The electronic system comprises a projection device and at least one peripheral device. The control method of the electronic system comprises the steps that an original instruction is sent to a natural language model through a projection device, and the projection device is provided with a communication interface and is in communication connection with at least one peripheral device through the communication interface; in response to the original instruction corresponding to the operation of the at least one peripheral device, generating at least one standard instruction based on the original instruction by means of the natural language model; and controlling the at least one peripheral device to execute an operation corresponding to the original instruction by the projection device according to the at least one standard instruction. According to the electronic system and the control method thereof, the projection device can be used as a central control device to control the peripheral device connected with the projection device.

Description

Technical Field

[0001] The invention relates to a control mechanism, and more particularly to an electronic system capable of controlling peripheral devices and a control method thereof. Background Art

[0002] The current control method of the projection device is that the user manually operates a remote control of the projection device or operates a human-machine interface on the projection device to operate related projection settings of the projection device. Therefore, the traditional control method of the projection device is quite inconvenient.

[0003] When the user wants to control the peripheral device coupled to the projection device, the user needs to confirm the type and connection status of the peripheral device connected to the projection device before manual control can be performed. Moreover, each function operation of the peripheral device needs to be operated separately, so the traditional method of controlling the peripheral device is quite inconvenient.

[0004] The "background technology" section is only used to help understand the content of the present invention. Therefore, the content disclosed in the "background technology" section may contain some known technologies that are not known to those skilled in the art. The content disclosed in the "background technology" section does not mean that the content or the problems to be solved by one or more embodiments of the present invention have been known or recognized by those skilled in the art before the application of the present invention. Summary of the invention

[0005] The invention provides an electronic system and a control method thereof, which can control peripheral devices connected thereto by using a projection device as a central control device.

[0006] Other purposes and advantages of the present invention can be further understood from the technical features disclosed in the present invention.

[0007] To achieve one or part or all of the above-mentioned purposes or other purposes, the present invention proposes a method for controlling an electronic system, wherein the electronic system includes a projection device and at least one peripheral device, and the method includes the following steps: sending an original instruction to a natural language model by the projection device, wherein the projection device has a communication interface and is communicatively connected with at least one peripheral device via the communication interface; in response to the original instruction corresponding to the operation of at least one peripheral device, generating at least one standard instruction based on the original instruction by the natural language model; and controlling at least one peripheral device to perform the operation corresponding to the original instruction according to the at least one standard instruction by the projection device.

[0008] To achieve one or part or all of the above purposes or other purposes, the present invention proposes a control electronic system. The electronic system includes: at least one peripheral device and a projection device for receiving an original instruction. The projection device includes: a communication interface, which is connected to the at least one peripheral device for communication; and a processor, which is coupled to the communication interface. The processor is configured to perform: in response to the original instruction corresponding to the operation of the at least one peripheral device, receiving at least one standard instruction corresponding to the original instruction through the communication interface; and controlling at least one peripheral device to perform the operation corresponding to the original instruction according to the at least one standard instruction.

[0009] Based on the above, the electronic system and control method of the present invention can identify the original command of the user to know the operation of the peripheral device corresponding to the original command, and then generate the corresponding standard command, and the projection device controls the peripheral device to perform the corresponding operation based on the standard command. Accordingly, the projection device can control the peripheral device connected to it. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a schematic diagram of an electronic system according to a first embodiment of the present invention.

[0011] Figure 2 is a schematic diagram of an electronic system according to a second embodiment of the present invention.

[0012] Figure 3 is a flow chart of a method for controlling an electronic system according to a second embodiment of the present invention.

[0013] Figure 4 is a schematic diagram of an electronic system according to a third embodiment of the present invention.

[0014] Figure 5 is a flow chart of a method for controlling an electronic system according to a third embodiment of the present invention.

[0015] Figure 6 FIG. 4 is a schematic diagram of a peripheral device definition process according to an embodiment of the present invention.

[0016] Figure 7 is a schematic diagram of a voice control process according to an embodiment of the present invention.

[0017] Description of reference numerals:

[0018] 100: Projection device

[0019] 110: Processor

[0020] 120: Communication interface

[0021] 130: Storage

[0022] P, P_1~P_N: Peripheral devices

[0023] 200: Natural Language Model

[0024] 400: Cloud Server

[0025] 610: BT connector

[0026] 620: HDMI connector

[0027] 630: Wi-Fi connector

[0028] 640: USB connector

[0029] 800: Sound installation

[0030] S305~S315: Steps

[0031] S505~S335: Steps

[0032] S701~S721: steps. DETAILED DESCRIPTION

[0033] The above-mentioned other technical contents, features and effects of the present invention will be clearly presented in the following detailed description of a preferred embodiment with reference to the accompanying drawings. The directional terms mentioned in the following embodiments, such as up, down, left, right, front or back, etc., are only the directions with reference to the accompanying drawings. Therefore, the directional terms used are used to illustrate and not to limit the present invention.

[0034] Figure 1 is a schematic diagram of an electronic system according to a first embodiment of the present invention. Figure 1 , the electronic system includes a peripheral device P and a projection device 100. The projection device 100 includes a processor 110, a communication interface 120 and a storage 130. The processor 110 is coupled to the communication interface 120 and the storage 130.

[0035] The processor 110 is, for example, a central processing unit (CPU), a physical processing unit (PPU), a programmable microprocessor, an embedded control chip, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or other similar devices.

[0036] The communication interface 120 includes at least one connector to connect at least one peripheral device. Figure 1Taking one peripheral device P as an example, the number of peripheral devices P may be one or more. The communication interface 120 may include at least one of a Bluetooth (BT) connector, a High Definition Multimedia Interface (HDMI) port, a Local Area Network (LAN) port, a network card connector, an audio input / output port, and a Universal Serial Bus (USB) port. The peripheral device P is, for example, an electronic device connected via Bluetooth, an electronic device connected via HDMI, an electronic device connected via Wi-Fi, or an electronic device connected via USB.

[0037] The electronic device connected by Bluetooth adopts the Bluetooth communication protocol, and the electronic device is, for example, a Bluetooth microphone, a Bluetooth speaker, a Bluetooth speaker, a Bluetooth amplifier, a Bluetooth headset, and other electronic devices. The electronic device connected by HDMI is, for example, a set-top box with HDMI, a multimedia player, a computer, a smart phone, a game console, an audio, an amplifier, a television, and other electronic devices. The electronic device connected by Wi-Fi is an electronic device connected to the projection device 100 using the Wi-Fi communication protocol, or an electronic device with Wi-Fi function, for example, an electronic device with a wireless network card, a router, or a switch. The electronic device connected by USB is, for example, a set-top box with a USB port, a multimedia player, a computer, a smart phone, a game console, an audio, an amplifier, and other electronic devices. The peripheral device P can also be a projection device 100 connected to other household devices (for example, air conditioners, lights, etc.) through a local area network, Bluetooth, etc., so that the projection device 100 can be used as a central control device of the smart home Internet of Things for control.

[0038] The peripheral device P of this embodiment may also include an input device, such as a sound receiving device (microphone), a keyboard, a touch panel, a mouse or a remote control, etc. The input device is used to receive original instructions from the user. The input device is coupled to the communication interface 120, and the communication interface 120 is configured to receive original instructions from the input device. For example, the communication interface 120 is connected to the sound receiving device, and the original instructions (voice instructions) received by the sound receiving device are transmitted to the processor 110 via the communication interface 120. For example, the communication interface 120 is connected to the keyboard, and the original instructions (text instructions) received by the keyboard are transmitted to the processor 110 via the communication interface 120. For example, the communication interface 120 is connected to an electronic device connected by Wi-Fi, and the original instructions (voice instructions or text instructions) received by the electronic device are transmitted to the processor 110 via the communication interface 120.

[0039] The storage 130 may be implemented by any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk or other similar devices or a combination of these devices. The storage 130 includes one or more program code segments, which are executed by the processor 110 after being installed to implement the following method of controlling the electronic system.

[0040] In this embodiment, the processor 110 responds to the original instruction corresponding to the operation of the peripheral device P, receives at least one standard instruction corresponding to the original instruction through the communication interface 120, and controls the peripheral device P to execute the operation corresponding to the original instruction according to the at least one standard instruction.

[0041] In one embodiment, the electronic system may further include an environment sensing device, such as a light sensor and a decibel meter. The environment sensing device is used to sense changes in the surrounding environment, and the projection device 100 may receive information input by the environment sensing device as reference information for the electronic system to control the projection device 100 and / or the peripheral device P. The light sensor is used to sense the ambient brightness, and the decibel meter is used to sense the ambient volume. The electronic system may use the environment sensing device to assist in adjusting the brightness, color gamut, and volume of the projection device 100 and / or the peripheral device P.

[0042] Figure 2 is a schematic diagram of an electronic system according to a second embodiment of the present invention. Figure 2 The electronic system of the second embodiment includes a projection device 100, at least one peripheral device P_1-P_N, and a natural language model 200. The projection device 100 of the first embodiment can be used in the second embodiment. The projection device 100 can communicate with the natural language model 200 in a wireless or wired manner via the communication interface 120.

[0043] In this embodiment, the natural language model 200 is, for example, a chatbot (ChatBot) with a machine learning algorithm. The chatbot is, for example, any pre-trained chatbot such as Chat Generative Pre-trained Transformer (ChatGPT), Microsoft Bing, Google Bard, or ERNIEBot, or a dedicated chatbot trained with data in a specific field. The natural language model 200 can also be a language model trained separately. The natural language model 200 can be used to perform natural language processing (Natural Language Processing, NLP), natural language processing and understanding, dialogue management, speech-to-text, and text-to-speech, etc. The natural language model 200 can recognize multiple languages ​​and multiple accents. The natural language model 200 can be set in a cloud server. In one embodiment, the natural language model 200 may also be disposed in the projection device 100 , and the natural language model 200 is coupled to the processor 110 of the projection device 100 .

[0044] Figure 3 is a flow chart of a method for controlling an electronic system according to a second embodiment of the present invention. Figure 3 In step S305, the projection device 100 sends the original instruction to the natural language model 200. Then, in step S310, in response to the original instruction corresponding to the operation of at least one of the peripheral devices P_1 to P_N, the natural language model 200 generates at least one standard instruction based on the original instruction. In step S315, the projection device 100 controls at least one of the peripheral devices P_1 to P_N to perform the operation corresponding to the original instruction according to the at least one standard instruction. For example, in controlling the peripheral device P_1, the projection device 100 transmits the corresponding control signal to the control circuit of the peripheral device P_1 through the processor 110, and the control circuit performs the corresponding operation.

[0045] In one embodiment, the original instruction can control the projection device 100 and the peripheral devices P_1~P_N at the same time. In response to the original instruction corresponding to the operation of at least one of the projection device 100 and the peripheral devices P_1~P_N at the same time, the natural language model 200 generates at least one standard instruction based on the original instruction. The projection device 100 controls at least one of the projection device 100 and the peripheral devices P_1~P_N to perform the operation corresponding to the original instruction according to the at least one standard instruction. The original instruction issued by the user can control only the operation of at least one of the peripheral devices P_1~P_N, or can control the operation of the projection device 100 and at least one of the peripheral devices P_1~P_N at the same time.

[0046] In one embodiment, the natural language model 200 may generate at least one standard instruction based on the original instruction, the projector rule instruction and at least one peripheral device rule instruction. The projector rule instruction is used to make the standard instruction generated by the natural language model 200 conform to the operation executable by the projection device 100, and the peripheral device rule instruction is used to make the standard instruction generated by the natural language model 200 conform to the operation executable by at least one of the peripheral devices P_1 to P_N. In this embodiment, the projector rule instruction and the peripheral device rule instruction may be stored in the storage 130 of the projection device 100. The peripheral device rule instruction includes at least one of the product type of one of the corresponding peripheral devices and the description of the communication interface 120 to which the corresponding one of the peripheral devices is connected (e.g., the connection method).

[0047] The processor 110 of the projection device 100 may convert the received at least one standard instruction into a control code that can be recognized and executed by the projection device 100. Alternatively, the natural language model 200 or the cloud server may convert the received at least one standard instruction into a control code that can be recognized and executed by the projection device 100.

[0048] Figure 4 is a schematic diagram of an electronic system according to a third embodiment of the present invention. Figure 4, the electronic system of the third embodiment includes a projection device 100, peripheral devices P_1~P_N, a natural language model 200 and a cloud server 400. The projection device 100 of the first and second embodiments and the natural language model 200 of the second embodiment can be used in the third embodiment. The projection device 100, the natural language model 200 and the cloud server 400 can communicate with each other by wired and / or wireless communication methods. The wired communication method is, for example, a cable. The wireless communication method includes, for example, Wi-Fi, Bluetooth, etc. For example, the projection device 100, the natural language model 200 and the cloud server 400 can be connected by network transmission. The natural language model 200 can be set in the cloud server 400 or a third-party cloud server, and the cloud server 400 and the third-party cloud server can include a processor, a communication device and a storage device. The storage device is used, for example, to store a chat robot with a machine learning algorithm, and the processor is used, for example, to execute the above algorithm. The communication device is used to communicate with the projection device 100 and the natural language model 200.

[0049] Figure 5 is a flow chart of a method for controlling an electronic system according to a third embodiment of the present invention. Figure 5 In step S505 , the projection device 100 obtains the device description of at least one of the peripheral devices P_1 -P_N connected to the communication interface 120 , and transmits the device description to the cloud server 400 .

[0050] The device description of the peripheral devices P_1~P_N can be a device name, an on-screen display (OSD) name, a general architecture name, etc. The device description can have the characteristics of the brand or device. The device description of an electronic device connected via HDMI is, for example, "XXXBD", "AndroidTV", "TVDongle", "AppleTV", "GameStation". The device description of an electronic device connected via Bluetooth or Wi-Fi is, for example, a device model or a string containing a brand, such as "BEX1850MH" (Bluetooth headset), "MITVS26" (Bluetooth speaker). In one embodiment, the device description can also be displayed on the display interface (such as on-screen display, OSD) of the projection device 100 to let the user know the peripheral devices P_1~P_N currently connected to the projection device 100.

[0051] When the peripheral devices P_1-P_N are connected to the projection device 100 by wire or wireless means, the peripheral devices P_1-P_N can provide the device descriptions of the peripheral devices P_1-P_N to the projection device 100 by means of connection signals. The projection device 100 can be set to read the device descriptions of the peripheral devices P_1-P_N at a predetermined time point (for example, when the projection device 100 is turned on, a new peripheral device is detected, at a fixed time interval, etc.) and provide them to the cloud server 400. One peripheral device has a corresponding device description.

[0052] After the projection device 100 transmits the device description to the cloud server 400, in step S510, the cloud server 400 transmits the device description and the projector rule instruction to the natural language model 200 in response to receiving the device description from the projection device 100. The projector rule instruction is, for example, stored in the cloud server 400. The projector rule instruction is used to limit the natural language model 200 to generate a peripheral device rule instruction for the device description after receiving the device description. After receiving the device description and the projector rule instruction, the natural language model 200 generates a peripheral device rule instruction based on the device description and the projector rule instruction. In step S515, the natural language model 200 transmits the peripheral device rule instruction to the cloud server 400.

[0053] In order to enable the electronic system to control the peripheral devices P_1-P_N, it is necessary to enable the natural language model 200 to generate at least one standard instruction corresponding to the operation executable by at least one of the peripheral devices P_1-P_N based on the peripheral device rule instruction of at least one of the peripheral devices P_1-P_N. Since the peripheral devices P_1-P_N connected to the projection device 100 may change at any time, for example, the user plugs in and out different electronic devices connected by HDMI according to needs, or connects to different electronic devices via Bluetooth or Wi-Fi, etc., the device description of the peripheral devices P_1-P_N currently connected to the projection device 100 needs to be transmitted to the natural language model 200 each time the original instruction is sent. Accordingly, the natural language model 200 can provide standard instructions that can be used to execute the operations of the peripheral devices P_1-P_N connected to the projection device 100, avoiding the natural language model 200 from providing invalid instructions that cannot be executed.

[0054] When the natural language model 200 identifies the device description, the device description may be a product code, number or serial number of a peripheral device. The projector rule instruction can be used to require the natural language model 200 to search for the peripheral devices P_1~P_N corresponding to the device description through its internal database or the network, and obtain the corresponding peripheral devices P_1~P_N The software and hardware configurations and executable operations have. The natural language model 200 can generate peripheral device rule instructions based on the obtained software and hardware configurations and executable operations. For example, the device description is "XX_PS5", and the natural language model 200 learns through the query that the corresponding peripheral device P_1 is a "game console", and associates this peripheral device P_1 with the relevant operations of the "game console" and the connection port to which this peripheral device P_1 is connected. Therefore, even if the original instruction does not explicitly require the execution of an action on a peripheral device connected to a certain connection method in the projection device 100, as long as the original instruction is related to the "game console", when the natural language model 200 converts the original instruction into a standard instruction, the standard instruction will be a default instruction for controlling the peripheral device P_1 connected to the connection port associated with the device description "XX_PS5".

[0055] In one embodiment, the projector rule instruction provides a predetermined format of the peripheral device rule instruction. When the natural language model 200 recognizes the device description, the product features corresponding to the device description are filled into the variable part of the predetermined format to form the peripheral device rule instruction.

[0056] In one embodiment, after the natural language model 200 transmits the peripheral device rule instruction to the cloud server 400, the cloud server 400 associates the peripheral device rule instruction with the corresponding projection device 100 and stores it, or the cloud server 400 associates the peripheral device rule instruction with the user account of the corresponding projection device 100 and stores it. In other embodiments, the cloud server 400 may also associate information such as a product serial number, an Internet Protocol (IP) address, and a Media Access Control Address (MAC) address with the peripheral device rule instruction. The information is used to indicate the source of the device description. Accordingly, when the cloud server 400 subsequently receives an original instruction issued by the same projection device 100, the cloud server 400 may transmit the original instruction, the projector rule instruction, and the peripheral device rule instruction associated with the projection device 100 to the natural language model 200.

[0057] In one embodiment, the device description may be a description with a fixed format for the natural language model 200 to recognize. The fixed format may refer to a description that is common in the industry and applicable to a variety of products. For example, the natural language model 200 determines the type of each peripheral device P_1 to P_N and the operations that can be performed by each peripheral device based on the device description, and generates peripheral device rule instructions for each peripheral device P_1 to P_N based on the result of the determination. One peripheral device corresponds to one peripheral device rule instruction.

[0058] In one embodiment, the instructions for enabling the natural language model 200 to recognize the device description to generate the peripheral device rule instructions may not be included in the projector rule instructions, and may be implemented by other rule instructions.

[0059] After the user inputs the original command to the projection device 100, in step S520, the projection device 100 transmits the original command to the cloud server 400. In step S525, in response to receiving the original command from the projection device 100, the cloud server 400 transmits the original command, the projector rule command, and the peripheral device rule command to the natural language model 200, so that the natural language model 200 can analyze the original command and generate a standard command corresponding to the original command and that can be interpreted by the projection device 100 based on the original command, the projector rule command, and the peripheral device rule command.

[0060] In this embodiment, the natural language model 200 can generate different standard instructions according to the type of the original instruction. The types of the original instruction include single control instructions, multiple control instructions, compound control instructions, etc. The single control instruction is an instruction for which the original instruction only requires adjustment of one parameter of the projection device 100 or the peripheral devices P_1~P_N or only includes one operation. For example, the original instruction is a single control instruction such as lowering the volume of the peripheral devices P_1~P_N or switching the connected peripheral devices P_1~P_N. The natural language model 200 can generate standard instructions corresponding to the single operation. Multiple control instructions are instructions for which the original instruction requires multiple single operations. For example, the original instruction can be multiple control instructions such as lowering the volume of the peripheral devices P_1~P_N and switching the connected peripheral devices P_1~P_N. The natural language model 200 can generate standard instructions corresponding to multiple single operations. The compound control instruction is an instruction for which the original instruction requires adjustment of multiple parameters of the projection device 100 and / or the peripheral devices P_1~P_N. The original instruction is, for example, a composite control instruction for adjusting the visual effect of the projection image, such as “please improve the color of the image.” For example, the natural language model 200 generates multiple standard instructions for adjusting multiple parameters such as the color gamut, brightness, and sharpness of the projection image.

[0061] The projector rule instruction is also used to limit the natural language model 200 to only output instructions that can be interpreted by the projection device 100 or can be converted into control codes of the projection device 100. For example, the projector rule instruction can limit the natural language model 200 to only output standard instructions that can be executed by the projection device 100. Standard instructions can be, for example, "power on", "power off", "volume up", "volume down", "connect HDMI1", "connect HDMI2", "turn on the audio", "switch HDMI source", etc. or their codes, and require the natural language model 200 to parse the semantics of the received original instructions and summarize the original instructions into any of the above standard instructions, or summarize them into other standard instructions.

[0062] The projector rule instructions and the peripheral device rule instructions may include a set of tens, hundreds or thousands of characters. The projector rule instructions added each time the cloud server 400 sends the original instruction to the natural language model 200 may be the same. The cloud server 400 may also optimize the projector rule instructions at any time, for example, increase or decrease the number of conditions of the projector rule instructions to achieve faster feedback speed, lower cost or more accurate feedback.

[0063] In step S530, the natural language model 200 transmits the standard command to the cloud server 400. The cloud server 400 converts the standard command into a corresponding control code. Thereafter, in step S535, the cloud server 400 transmits the control code to the projection device 100, so that the projection device 100 executes the control code to control at least one of the peripheral devices P_1 to P_N, or the projection device 100 controls at least one of the peripheral devices P_1 to P_N and the operation of the projection device 100 at the same time according to the control code. In addition, as shown in step S535', the natural language model 200 may transmit the standard command to the projection device 100, and the projection device 100 itself converts the standard command into a corresponding control code to execute the corresponding operation.

[0064] In one embodiment, the processor 110 of the projection device 100 can display a control interface in the display interface of the projection device 100, and the user can input the original command through the control interface. The original command can be a text command or a voice command. For example, the control interface includes an option to activate the recording function of the sound receiving device (such as a microphone) of the projection device 100. When the sound receiving device is activated, the control interface can receive a voice signal through the sound receiving device. After receiving the voice signal, the projection device 100 converts the voice signal into an original command in text form through the processor 110. Alternatively, the control interface includes an option to activate manual input (handwriting board, touch pad or keyboard input) to receive the original command in text form input by the user.

[0065] In other embodiments, the voice signal may be converted into an original command in text form by a terminal device (such as a smart phone) paired with the projection device 100. For example, after the projection device 100 receives the voice signal, the projection device 100 transmits the voice signal to the terminal device paired with it, and then the terminal device converts the voice signal into an original command in text form, and transmits the original command to the projection device 100. Alternatively, the voice signal may be received by a sound receiving device (such as a microphone) of the terminal device, and after the terminal device converts the voice signal into an original command in text form, the original command is transmitted to the projection device 100, and the projection device 100 uploads it to the cloud server 400, or the terminal device is coupled to the cloud server 400, and the terminal device can directly upload the original command to the cloud server 400.

[0066] In other embodiments, after receiving the voice signal, the projection device 100 or the terminal device may transmit the voice signal to the cloud server 400, so that the cloud server 400 converts the voice signal into an original command in text form. After the cloud server 400 receives the voice signal, the voice recognition model of the cloud server 400 may convert the voice signal in natural language form into an original command in text form.

[0067] In other embodiments, the original instruction may be a voice signal in the form of natural language input by the user. The voice signal may be directly uploaded to the natural language model 200 , and the natural language model 200 recognizes the original instruction in the form of voice.

[0068] Figure 6 is a schematic diagram of a peripheral device definition process according to an embodiment of the present invention. Figure 6 In this embodiment, the communication interface 120 includes, for example, four connectors, namely, a BT connector 610, an HDMI connector 620, a Wi-Fi connector 630, and a USB connector 640. When the projection device 100 is powered on, the processor 110 executes a monitoring program in the projection device 100 to scan the BT connector 610, the HDMI connector 620, the Wi-Fi connector 630, and the USB connector 640, and obtains the device descriptions of the peripheral devices P_1 to P_N through the connection signals of each connection method.

[0069] For the BT connector 610, the processor 110 of the projection device 100 can obtain the device description by the name when the Bluetooth communication protocol sends a connection request. For the HDMI connector 620, the processor 110 of the projection device 100 can obtain the device description by the Consumer Electronics Control (CEC) protocol or the Monitor Control Command Set (MCCS) protocol. For the Wi-Fi connector 630, the processor 110 can obtain the device description when the Wi-Fi protocol sends a connection request. For the USB connector 640, the processor 110 can obtain the device description when the USB protocol sends a connection request. The peripheral devices P_1~P_N all have corresponding description texts in the handshake packet that establishes a link with the projection device 100. The processor 110 can capture these description texts as device descriptions.

[0070] After obtaining the device description, the projection device 100 uploads the device description to the cloud server 400. At the same time, the monitoring program of the projection device 100 will detect in real time whether there is any change in the connection of the BT connector 610, the HDMI connector 620, the Wi-Fi connector 630, and the USB connector 640. If there is a change, the updated device description is re-obtained and uploaded to the cloud server 400 for updating. If the monitoring program detects that one of the peripheral devices is disconnected, the projection device 100 can send an update command to the cloud server 400 to delete the device description corresponding to the disconnected peripheral device in the cloud server 400.

[0071] After receiving the device description, the cloud server 400 transmits the device description and the projector rule command to the natural language model 200. The device description allows the natural language model 200 to identify the product type of the peripheral devices P_1-P_N, generate corresponding peripheral device rule commands, and transmit the peripheral device rule commands back to the cloud server 400 for storage.

[0072] The peripheral device rule instructions saved in the cloud server 400 can be overwritten when a new peripheral device rule instruction is received. For example, when the peripheral devices P_1-P_N are changed, the projection device 100 will send the device description to the cloud server 400 again, and the cloud server 400 will send the device description and the projector rule instruction to the natural language model 200 again, so that the natural language model 200 generates a new peripheral device rule instruction and sends it back to the cloud server 400 to overwrite the old peripheral device rule instruction.

[0073] The natural language model 200 can generate a single peripheral device rule instruction or a combination of multiple peripheral device rule instructions. The combination of multiple peripheral device rule instructions, for example, generates a respective peripheral device rule instruction for each peripheral device (i.e., each device description). Therefore, when the projection device 100 is connected to multiple peripheral devices P_1~P_N, multiple peripheral device rule instructions will be generated. The natural language model 200 can generate standard instructions for the projection device 100 and the peripheral devices P_1~P_N based on the intention of the original instruction and according to the projector rule instruction and the peripheral device rule instruction. Therefore, a cross-device control result can be achieved by one original instruction.

[0074] For example, the projection device 100 is connected to a game console of brand A via the HDMI connector 620, and is connected to a sound device of brand B via the BT connector 610. The processor 110 of the projection device 100 can obtain the device description of the game console as "A_GameStation" through the CEC protocol, and obtain the device description of the sound device as "B_SoundBar" through the Bluetooth communication protocol. The processor 110 can transmit the device descriptions "A_GameStation" and "B_SoundBar" to the cloud server 400 via the communication interface 120, and the cloud server 400 then transmits the device descriptions and the projector rule instructions to the natural language model 200.

[0075] The natural language model 200 recognizes the device description "A_GameStation", obtains information such as the product type, port type, and executable operations (for example, standby, startup, and switching modes) of "A_GameStation" from the network, and generates peripheral device rule instructions corresponding to "A_GameStation" (hereinafter referred to as "peripheral rule A"). Peripheral rule A is used to limit the natural language model 200 to generate standard instructions only for operations that can be executed by the game console of brand A. The natural language model 200 recognizes the device description "B_SoundBar", and its internal database includes information such as the product type and executable operations corresponding to "B_SoundBar", and generates peripheral device rule instructions corresponding to "B_SoundBar" (hereinafter referred to as "peripheral rule B"). Peripheral rule B is used to limit the natural language model 200 to generate standard instructions only for operations that can be executed by the audio device of brand B.

[0076] Next, the natural language model 200 returns the surrounding rules A and the surrounding rules B to the cloud server 400. The cloud server 400 is used to store the user account, IP address or product serial number corresponding to the projection device 100. When the cloud server 400 receives the original command, the cloud server 400 transmits the original command, the stored projector rule command, and the surrounding rules A and B to the natural language model 200. The natural language model 200 generates at least one standard command based on the requirements for the projection device 100, the game console of brand A, and the audio device of brand B in the original command.

[0077] Figure 7 2 is a schematic diagram of a voice control process according to an embodiment of the present invention. Before executing the voice control process, the peripheral device definition process may be executed at least once to enable the natural language model 200 to generate peripheral device rule instructions, and then the process of controlling the peripheral devices can be implemented.

[0078] Please refer to Figure 7 In step S701, the projection device 100 receives a voice signal. In step S703, the projection device 100 converts the voice signal into an original instruction in text form, or directly uses the voice signal as the original instruction. In step S705, the projection device 100 transmits the original instruction to the cloud server 400. In step S707, the cloud server 400 transmits the original instruction, the projector rule instruction, and the peripheral device rule instruction to the natural language model 200. In step S709, the natural language model 200 generates a standard instruction for the operation corresponding to the original instruction based on the original instruction, the projector rule instruction, and the peripheral device rule instruction.

[0079] Assuming that the original instruction is "I want to play video games", the natural language model 200 can analyze the following points: (1) the input signal source of the projection device 100 needs to be switched to one of the peripheral devices P_1-P_N currently connected to the HDMI connector 620 (e.g., a game console); (2) one of the peripheral devices P_1-P_N currently connected to the BT connector 610 needs to be activated (e.g., the audio device 800); (3) the playback mode of the projection device 100 should be changed to "game mode". Therefore, the natural language model 200 will generate the following standard instructions based on the projector rule instruction and the peripheral device rule instruction: a first standard instruction to switch the signal input source to the HDMI source; a second standard instruction to activate the audio device 800 and adjust the volume or channel to a value corresponding to the game mode; and a third standard instruction to switch the projection mode of the projection device 100 to the game mode.

[0080] In step S711, the natural language model 200 transmits the standard command to the cloud server 400. In step S713, the cloud server 400 converts the standard command into a control code executable by the projection device 100, and in step S715, transmits the control code to the projection device 100.

[0081] In step S717, the projection device 100 switches the HDMI source to the game console based on the first standard instruction. For example, assuming that the HDMI switch includes multiple HDMI input ports (respectively connected to multiple peripheral devices P_1~P_N) and an HDMI output port (the output multimedia signal can be provided to the display interface of the projection device 100), the projection device 100 transmits a switching signal to the HDMI switch by the processor 110 based on the first standard instruction, so that the control circuit of the HDMI switch selects the HDMI input port to be connected by the switch on the switch, so as to receive the multimedia signal provided by one of the corresponding peripheral devices P_1~P_N from the connected HDMI input port. The first standard instruction corresponds to the operation of controlling the projection device 100, and the first standard instruction is converted from the original instruction and the projector rule instruction. In other embodiments, if the original instruction includes a specific game among multiple games in the game console, the natural language model 200 can also generate a standard instruction corresponding to the specific game.

[0082] In step S719, the projection device 100 activates the audio device 800 based on the second standard instruction, and drives the audio device 800 to adjust the volume. For example, the projection device 100 transmits a start signal to the audio device 800 based on the second standard instruction by the processor 110 to wake up or activate the audio device 800. Then, the processor 110 transmits a corresponding control signal to the control circuit of the audio device 800 based on the second standard instruction, and the control circuit adjusts the volume or channel to a value corresponding to the game mode. The second standard instruction corresponds to the operation of controlling one of the peripheral devices P_1~P_N (for example, the audio device 800), and the second standard instruction is, for example, converted from at least the original instruction and the peripheral device rule instruction.

[0083] In step S721, the projection device 100 switches its projection mode to the game mode according to the third standard instruction to apply the parameter settings of the game mode (e.g., brightness, color, etc.). For example, the projection device 100 selects the game mode from multiple projection modes based on the third standard instruction by the processor 110, and reads the parameter settings of the game mode, and then adjusts the settings of the corresponding light source module, display module, etc. of the projection device 100 based on the parameter settings. The third standard instruction corresponds to the operation of controlling the projection device 100, and the third standard instruction is, for example, converted from the original instruction and the projector rule instruction.

[0084] In summary, the electronic system and control method of the present invention can identify the peripheral devices connected to the projection device, and provide the peripheral device related information to the natural language model, so that the natural voice model generates corresponding standard instructions, and then the projection device is used as the central console of the peripheral devices based on the standard instructions to control the peripheral devices. Accordingly, even if the original instruction issued by the user does not clearly specify the object to be controlled, the electronic system can still achieve the desired result of the user by simultaneously controlling multiple peripheral devices related to the requirements of the original instruction through the recognition of the natural language model. In addition, the user does not need to remember the cumbersome function operation, the physical connection of multiple external peripheral devices or the specified IP address, system name, device code and other information, and can control the projection device in the most convenient way. Even for users who are operating the projection device for the first time or are not familiar with the operation of its peripheral devices, they can use the peripheral devices without obstacles and adjust the parameter settings to the best viewing, sound and other effects.

[0085] However, the above is only a preferred embodiment of the present invention, and it cannot be used to limit the scope of the implementation of the present invention. That is, all simple equivalent changes and modifications made according to the claims and the content of the invention of the present invention are still within the scope of the patent of the present invention. In addition, any embodiment or claim of the present invention does not need to achieve all the purposes, advantages or features disclosed by the present invention. In addition, the abstract and title (invention name) are only used to assist in the retrieval of patent documents, and are not used to limit the scope of rights of the present invention. In addition, the terms "first", "second", etc. mentioned in this specification or claims are only used to name the name of the element or distinguish different embodiments or scopes, and are not used to limit the upper or lower limit on the number of elements.

Claims

1. A method for controlling an electronic system, characterized in that: The electronic system includes a projection device and at least one peripheral device, and the method includes the following steps: Sending an original command to a natural language model via the projection device, wherein the projection device has a communication interface and is communicatively connected with the at least one peripheral device via the communication interface; In response to the original instruction corresponding to the operation of the at least one peripheral device, generating at least one standard instruction based on the original instruction by the natural language model; as well as The projection device controls the at least one peripheral device to execute the operation corresponding to the original instruction according to the at least one standard instruction.

2. The method for controlling an electronic system according to claim 1, characterized in that: The method for controlling an electronic system further comprises the following steps: In response to the original instruction corresponding to the operation of the projection device and the at least one peripheral device, generating the at least one standard instruction based on the original instruction by the natural language model; as well as The projection device controls the projection device and the at least one peripheral device to execute the operation corresponding to the original instruction according to the at least one standard instruction.

3. The method for controlling an electronic system according to claim 2, characterized in that: The step of generating the at least one standard instruction based on the original instruction by using the natural language model comprises: The at least one standard instruction is generated by the natural language model based on the original instruction, the projector rule instruction and the at least one peripheral device rule instruction, wherein the projector rule instruction is used to make the at least one standard instruction generated by the natural language model conform to the operation executable by the projection device, and the at least one peripheral device rule instruction is used to make the at least one standard instruction generated by the natural language model conform to the operation executable by the at least one peripheral device.

4. The method for controlling an electronic system according to claim 3, characterized in that: The method for controlling an electronic system further comprises the following steps: Obtaining at least one device description of at least one peripheral device connected to the communication interface by the projection device, and transmitting the at least one device description to a cloud server, wherein the cloud server stores the projector rule instruction; By the cloud server, in response to receiving the at least one device description from the projection device, transmitting the at least one device description and the projector rule instruction to the natural language model; as well as Generate the at least one peripheral device rule instruction based on the at least one device description and the projector rule instruction by using the natural language model, and transmit the at least one peripheral device rule instruction to the cloud server; The step of generating the at least one standard instruction based on the original instruction by means of the natural language model comprises: In response to receiving the original instruction from the projection device, the cloud server transmits the original instruction, the projector rule instruction and the at least one peripheral device rule instruction to the natural language model, so that the natural language model generates the at least one standard instruction.

5. The method for controlling an electronic system according to claim 4, characterized in that: The method for controlling an electronic system further comprises the following steps: In response to the projection device detecting a change in at least one peripheral device connected to the communication interface, the projection device obtains at least one updated device description and transmits it to the cloud server, so that the natural language model generates corresponding at least one peripheral device rule instruction based on the at least one updated device description.

6. The method for controlling an electronic system according to claim 4, characterized in that: After the step of transmitting the at least one peripheral device rule instruction to the cloud server by using the natural language model, the method further includes the following steps: The cloud server associates the at least one peripheral device rule instruction with the projection device and stores the instruction; or The cloud server associates the at least one peripheral device rule instruction with the user account of the projection device and stores the instruction.

7. The method for controlling an electronic system according to claim 3, characterized in that: The at least one peripheral device rule instruction includes at least one of a product type of the at least one peripheral device and a description of the communication interface to which the at least one peripheral device is connected.

8. The method for controlling an electronic system according to claim 1, characterized in that: The method for controlling an electronic system further comprises the following steps: In response to receiving a voice signal, converting the voice signal into the original instruction in a text form by the projection device; or In response to receiving the voice signal, converting the voice signal into the original instruction in text form by a terminal device paired with the projection device; or In response to receiving the voice signal, the projection device or the terminal device transmits the voice signal to a cloud server, so that the cloud server converts the voice signal into the original instruction in the text form.

9. The method for controlling an electronic system according to claim 1, characterized in that: The method for controlling an electronic system further comprises the following steps: In response to receiving a voice signal, the projection device transmits the voice signal to the natural language model, and uses the voice signal as the original instruction.

10. The method for controlling an electronic system according to claim 1, characterized in that: The method for controlling an electronic system further comprises the following steps: The projection device converts the at least one standard instruction into at least one corresponding control code to execute the at least one control code to control the at least one peripheral device, wherein the at least one control code corresponds to the operation required by the original instruction, or The cloud server converts the at least one standard instruction into the corresponding at least one control code, and transmits the at least one control code to the projection device, so that the projection device executes the at least one control code to control the at least one peripheral device.

11. An electronic system, characterized in that: The electronic system includes at least one peripheral device and a projection device, wherein: The projection device is used to receive an original instruction, and the projection device includes a communication interface and a processor, wherein: The communication interface is communicatively connected to the at least one peripheral device; and The processor is coupled to the communication interface, and the processor is configured to execute: In response to the original command corresponding to the operation of the at least one peripheral device, receiving at least one standard command corresponding to the original command through the communication interface; and According to the at least one standard instruction, the at least one peripheral device is controlled to execute the operation corresponding to the original instruction.

12. The electronic system according to claim 11, characterized in that: The processor is also configured to perform: In response to the original command corresponding to the operation of the projection device and the at least one peripheral device, receiving the at least one standard command corresponding to the original command via the communication interface; as well as According to the at least one standard instruction, the projection device and the at least one peripheral device are controlled to execute the operation corresponding to the original instruction.

13. The electronic system according to claim 12, characterized in that: The electronic system also includes a natural language model, and the projection device is connected to the natural language model via the communication interface. The natural language model is used to generate the at least one standard instruction based on the original instruction, the projector rule instruction and the at least one peripheral device rule instruction, wherein the projector rule instruction is used to make the at least one standard instruction generated by the natural language model conform to the operation executable by the projection device, and the at least one peripheral device rule instruction is used to make the at least one standard instruction generated by the natural language model conform to the operation executable by the at least one peripheral device.

14. The electronic system according to claim 13, characterized in that: The electronic system further includes a cloud server, and the projection device is connected to the cloud server via the communication interface, wherein: The processor is configured to execute: obtaining, through the communication interface, at least one device description of the at least one peripheral device connected to the communication interface; The cloud server is provided with the projector rule instruction, and the cloud server is configured to execute: In response to receiving the at least one device description from the projection device, transmitting the at least one device description and the projector rule instruction to the natural language model; and The natural language model is configured to perform: In response to receiving the at least one device description and the projector rule command, generating the at least one peripheral device rule command based on the at least one device description and the projector rule command, and transmitting the at least one peripheral device rule command to the cloud server.

15. The electronic system according to claim 14, characterized in that: The cloud server is configured to perform: In response to receiving the original instruction from the projection device, transmitting the original instruction, the projector rule instruction, and the at least one peripheral device rule instruction to the natural language model; The natural language model is configured to perform: In response to receiving the original command, the projector rule command and the at least one peripheral device rule command, the at least one standard command is generated based on the original command, the projector rule command and the at least one peripheral device rule command.

16. The electronic system according to claim 14, characterized in that: The processor is configured to perform: In response to detecting a change in at least one peripheral device connected to the communication interface, an updated at least one device description is obtained through the communication interface and transmitted to the cloud server, so that the natural language model generates corresponding at least one peripheral device rule instruction according to the updated at least one device description.

17. The electronic system according to claim 14, characterized in that: The cloud server is configured to perform: Associating the at least one peripheral device rule instruction with the projection device and then storing the instruction; or The at least one peripheral device rule instruction is associated with the user account of the projection device and then stored.

18. The electronic system according to claim 13, characterized in that: The at least one peripheral device rule instruction includes at least one of a product type of the at least one peripheral device and a description of the communication interface to which the at least one peripheral device is connected.

19. The electronic system according to claim 13, characterized in that: The processor is configured to perform: In response to receiving a voice signal, the voice signal is transmitted to the natural language model via the communication interface, and the voice signal is used as the original instruction.

20. The electronic system according to claim 11, characterized in that The processor is configured to perform: In response to receiving a voice signal, converting the voice signal into the original instruction in text form; or In response to receiving the voice signal, transmitting the voice signal to a cloud server so that the cloud server converts the voice signal into the original instruction in text form; or The electronic system further includes a terminal device, which is coupled to the projection device and is configured to execute: In response to receiving the voice signal, converting the voice signal into the original instruction in text form; or In response to receiving the voice signal, the voice signal is transmitted to a cloud server, so that the cloud server converts the voice signal into the original instruction in the text form.

21. The electronic system of claim 11, characterized in that: The processor is configured to perform: converting the at least one standard instruction into at least one corresponding control code to execute the at least one control code to control the at least one peripheral device, wherein the at least one control code corresponds to the operation required by the original instruction, or The electronic system also includes a cloud server, which is configured to execute: converting the at least one standard instruction into the corresponding at least one control code, and transmitting the at least one control code to the projection device, so that the projection device executes the at least one control code to control the at least one peripheral device.

Images