Electronic system and its control method

Abstract

To provide an electronic system and a control method thereof controlling connected peripheral devices by using a projection device as a central control device.SOLUTION: An electronic system includes at least one peripheral device, and a projection device having a communication interface and connecting the at least one peripheral device via the communication interface. A control method of the electronic system includes transmitting an original command to a natural language model via the projection device, generating at least one standard command based on the original command via the natural language model according to a fact that the original command corresponds to the operation of the at least one peripheral device, and controlling the at least one peripheral device to perform an operation corresponding to the original command via the projection device based on the at least one standard command.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a control mechanism, and more particularly, to an electronic system capable of controlling peripheral devices and a control method thereof.

Background Art

[0002] The current control method of a projection device is to operate the remote controller of the projection device through manual operation by the user, or to operate the projection settings related to the projection device by operating the human-machine interface on the projection device. Therefore, the conventional control method of the projection device is quite inconvenient.

[0003] The "Background Art" paragraph only helps to understand the content of the present invention. Therefore, the content disclosed in the "Background Art" paragraph may include some content that does not constitute known technology to those skilled in the art. In the content disclosed in the "Background Art" paragraph, it does not represent that the problems to be solved by the content or one or more embodiments of the present invention have been well-known or recognized by those skilled in the art before the filing of the present invention.

Summary of the Invention

Problems to be Solved by the Invention

[0004] When the user wants to control the peripheral devices connected to the projection device, the user can perform manual control only after confirming the types and connection status of the peripheral devices connected to the projection device by himself / herself. In addition, the respective function operations of the peripheral devices need to be operated independently and separately. Therefore, the conventional control of the peripheral devices is quite inconvenient.

Means for Solving the Problems

[0005] The present invention provides an electronic system capable of controlling connected peripheral devices and a control method thereof by using a projection device as a central control device.

[0006] Other objectives and advantages of the present invention can be further understood from the technical features disclosed in the present invention.

[0007] To achieve some or all of the above-described objectives, or other objectives, the present invention provides a method for controlling an electronic system. The electronic system includes a projection device and at least one peripheral device. The method includes transmitting an original command to a natural language model via the projection device, wherein the projection device has a communication interface and is connected via the communication interface to at least one peripheral device; generating at least one standard command via the natural language model based on the original command in accordance with the fact that the original command corresponds to an operation of at least one peripheral device; and controlling the at least one peripheral device via the projection device to perform an operation corresponding to the original command based on the at least one standard command.

[0008] To achieve some or all of the above-described objectives, or other objectives, the present invention provides an electronic system. The electronic system includes at least one peripheral device and a projection device for receiving original instructions. The projection device includes a communication interface connected to the at least one peripheral device and a processor coupled to the communication interface. The processor is configured to receive at least one standard instruction via the communication interface in response to the original instruction corresponding to an operation of at least one peripheral device, and to control the at least one peripheral device to perform an operation corresponding to the original instruction based on the at least one standard instruction. [Effects of the Invention]

[0009] Based on the above, the electronic system and its control method of the present invention recognize the user's original command, know the operation of the peripheral device corresponding to the original command, generate a corresponding standard command, and control the peripheral device to execute the corresponding operation based on the standard command via the projection device. Based on this, the projection device can be configured to control peripheral devices connected to the projection device. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram of an electronic system according to a first embodiment of the present invention. [Figure 2] This is a schematic diagram of an electronic system according to a second embodiment of the present invention. [Figure 3] This is a flowchart of a method for controlling an electronic system according to a second embodiment of the present invention. [Figure 4] This is a schematic diagram of an electronic system according to a third embodiment of the present invention. [Figure 5] This is a flowchart of a method for controlling an electronic system according to a third embodiment of the present invention. [Figure 6] This is a schematic diagram of the definition flow of peripheral devices according to one embodiment of the present invention. [Figure 7] This is a schematic diagram of a voice control flow according to one embodiment of the present invention. [Modes for carrying out the invention]

[0011] The aforementioned and other technical features, characteristics, and effects of the present invention will be clearly expressed through the following detailed description of preferred embodiments accompanied by the accompanying drawings. Directional terms such as up, down, left, right, front, or back, as used in the following embodiments, are merely references to the directions in the accompanying drawings. Therefore, the directional terms used are for illustrative purposes only and do not limit the present invention.

[0012] Figure 1 is a schematic diagram of an electronic system according to a first embodiment of the present invention. Referring to Figure 1, the electronic system includes peripheral equipment P and a projection device 100. The projection device 100 includes a processor 110, a communication interface 120, and storage 130. The processor 110 is coupled to the communication interface 120 and the storage 130. The projection device 100 is, for example, a projector.

[0013] In this embodiment, the number of processors 110 may be one or more. The processors 110 are, for example, a central processing unit (CPU), a physics processing unit (PPU), a programmable microprocessor (Microprocessor), an embedded control chip, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or other similar devices.

[0014] The communication interface 120 connects to at least one peripheral device by including at least one connector. Although Figure 1 illustrates this with one peripheral device P, the number of peripheral devices P may be one or more. The communication interface 120 may include at least one of the following: a Bluetooth (BT®) connector, a High Definition Multimedia Interface (HDMI®) port, a Local Area Network (LAN) connection port, a network card connector, an audio input / output port, and a Universal Serial Bus (USB) port. Peripheral devices P may be, for example, electronic devices that connect via Bluetooth, HDMI, Wi-Fi, or USB.

[0015] Electronic devices that connect via Bluetooth (peripheral devices P) employ the Bluetooth communication protocol, and examples of such devices include Bluetooth microphones, Bluetooth speakers, Bluetooth speaker boxes, Bluetooth amplifiers, and Bluetooth earphones. Electronic devices that connect via HDMI (peripheral devices P) include examples of set-top boxes, multimedia players, computers, smartphones, game consoles, audio equipment, amplifiers, and televisions equipped with HDMI. Electronic devices that connect via Wi-Fi (peripheral devices P) are electronic devices that connect to the projection device 100 using the Wi-Fi communication protocol, or electronic devices equipped with Wi-Fi functionality, such as wireless network cards, routers, and switches. Electronic devices that connect via USB (peripheral devices P) include examples of set-top boxes, multimedia players, computers, smartphones, game consoles, audio equipment, and amplifiers equipped with USB ports. Peripheral devices P may also be other household appliances (e.g., air conditioners, lights, etc.) to which the projection device 100 connects via a local area network, Bluetooth, etc., and control is performed by making the projection device 100 the central control unit of the smart home IoT.

[0016] The peripheral device P in this embodiment may further include an input device, such as a sound pickup device (microphone), keyboard, touch panel, mouse, or remote controller, which is used to receive original commands from the user. The input device is coupled to a communication interface 120, which is configured to receive original commands from the input device. For example, the communication interface 120 is connected to a sound pickup device, and the original commands (voice commands) received by the sound pickup device are transmitted to the processor 110 via the communication interface 120. For example, the communication interface 120 is connected to a keyboard, and the original commands (text commands) received by the keyboard are transmitted to the processor 110 via the communication interface 120. For example, the communication interface is connected to an electronic device that connects via Wi-Fi, and the original commands (voice commands or text commands) received by this electronic device are transmitted to the processor 110 via the communication interface 120.

[0017] The storage 130 may be implemented by employing any fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk, or other similar device, or a combination thereof. The storage 130 includes one or more program code snippets, which, after being installed, are executed via the processor 110 to implement a method for controlling the following electronic system.

[0018] In this embodiment, the processor 110 receives at least one standard command corresponding to the original command via the communication interface 120 in response to the original command corresponding to the operation of the peripheral device P, and the processor 110 controls the peripheral device P to perform the operation corresponding to the original command based on the at least one standard command.

[0019] In one embodiment, the electronic system may further include an environmental sensing device, which may be, for example, an optical sensor or a decibel meter. The environmental sensing device is configured to sense changes in the surrounding environment, and the projection device 100 may receive the information input via the environmental sensing device as reference information for the electronic system to control the projection device 100 and / or the peripheral device P. The optical sensor is configured to sense, for example, the brightness of the environment, and the decibel meter is configured to sense, for example, the volume of the environment. The electronic system can assist in adjusting the brightness, color gamut, volume, etc. of the projection device 100 and / or the peripheral device P via the environmental sensing device.

[0020] FIG. 2 is a schematic diagram of an electronic system according to a second embodiment of the present invention. Referring to FIG. 2, the electronic system of the second embodiment includes a projection device 100, at least one peripheral device P_1 to P_N, and a natural language model 200. The projection device 100 of the first embodiment can be used in the second embodiment. The projection device 100 may be communicatively connected to the natural language model 200 wirelessly or wired via a communication interface 120.

[0021] In this embodiment, the natural language model 200 is, for example, a chatbot equipped with a machine learning algorithm. The chatbot may be a pre-trained chatbot such as a Chat Generative Pre-trained Transformer (ChatGPT), Microsoft Bing, Google Bard, or Ernie Bot, or a dedicated chatbot trained on data in a specific field. The natural language model 200 may also be a separately trained language model. The natural language model 200 may be configured to perform natural language processing (NLP), natural language processing and understanding, dialogue management, speech-to-text, and text-to-speech. The natural language model 200 can distinguish between multiple languages ​​and many accents. The natural language model 200 may be hosted on a cloud server. In one embodiment, the natural language model 200 is provided within the projection device 100, and the natural language model 200 may be coupled with the processor 110 of the projection device 100.

[0022] FIG. 3 is a flowchart of a method for controlling an electronic system according to a second embodiment of the present invention. Referring to FIG. 3, at step S305, an original command is transmitted to the natural language model 200 via the projection device 100. Next, at step S310, in response to the original command corresponding to an operation of at least one of the peripheral devices P_1 to P_N, at least one standard command is generated based on the original command via the natural language model 200. At step S315, via the projection device 100, at least one of the peripheral devices P_1 to P_N is controlled to perform an operation corresponding to the original command based on the at least one standard command. For example, regarding the control of the peripheral device P_1, the projection device 100 transmits a corresponding control signal to the control circuit (processor) of the peripheral device P_1 via the processor 110, and the control circuit (processor) of the peripheral device P_1 performs the corresponding operation.

[0023] In one embodiment, the original command may control the projection device 100 and the peripheral devices P_1 to P_N (simultaneously). In response to the original command corresponding to an operation of at least one of the projection device 100 and the peripheral devices P_1 to P_N (simultaneously), at least one standard command is generated based on the original command via the natural language model 200. Via the projection device 100, at least one of the projection device 100 and the peripheral devices P_1 to P_N is controlled to perform an operation corresponding to the original command based on the at least one standard command. The original command issued by the user may control only the operation of at least one of the peripheral devices P_1 to P_N, or may also control the operation of at least one of the peripheral devices among the projection device 100 and the peripheral devices P_1 to P_N (simultaneously).

[0024] In one embodiment, the natural language model 200 may generate at least one standard command based on the original command, the projector rule command, and at least one peripheral rule command. The projector rule command is used to conform the standard command generated by the natural language model 200 to an operation that the projection device 100 can perform, and the peripheral rule command is used to conform the standard command generated by the natural language model 200 to an operation that at least one of the peripheral devices P_1 to P_N can perform. In this embodiment, the projector rule command and the peripheral rule command may be stored in the storage 130 of the projection device 100. The peripheral rule command includes at least one of the product type of the peripheral device to which it corresponds and a description (e.g., connection method) of the communication interface 120 to which the peripheral device connects.

[0025] The processor 110 of the projection device 100 may convert at least one (received) standard command into a control code that the projection device 100 can identify and execute. Alternatively, the processor may convert at least one (received) standard command into a control code that the projection device 100 can identify and execute via a natural language model 200 or a cloud server.

[0026] Figure 4 is a schematic diagram of an electronic system according to a third embodiment of the present invention. Referring to Figure 4, the electronic system of the third embodiment includes a projection device 100, peripheral devices P_1 to P_N, a natural language model 200, and a cloud server 400. The projection device 100 and the natural language model 200 of the first and second embodiments can be used in the third embodiment. Communication between the projection device 100, the natural language model 200, and the cloud server 400 may be via wired and / or wireless communication methods. A wired communication method is, for example, a cable. A wireless communication method includes, for example, Wi-Fi, Bluetooth, etc. For example, the projection device 100, the natural language model 200, and the cloud server 400 may be connected via network transmission. The natural language model 200 may be installed on the cloud server 400 and / or a third-party cloud server, and the cloud server 400 and / or a third-party cloud server may include one processor (or more processors), a communication device, and a storage device. The storage device of the cloud server 400 and / or a third-party cloud server is configured to store, for example, a chatbot (natural language model 200) equipped with a machine learning algorithm, and the processor of the cloud server 400 and / or a third-party cloud server is used, for example, to execute the algorithm described above. The communication device of the cloud server 400 and / or a third-party cloud server is used to establish a communication connection with the projection device 100 and the natural language model 200.

[0027] Figure 5 is a flowchart of a method for controlling an electronic system according to a third embodiment of the present invention. Referring to Figure 5, in step S505, the device description of at least one of the peripheral devices P_1 to P_N connected to the communication interface 120 via the projection device 100 is obtained, and the device description is sent to the cloud server 400.

[0028] The device descriptions for peripheral devices P_1 to P_N may be the device name, on-screen display (OSD) name, general-purpose framework name, etc. The device descriptions may have brand or device characteristics. The device descriptions for electronic devices connected via HDMI (peripheral devices P_1 to P_N) are, for example, "XXXBD", "Android TV", "TVDongle", "Apple TV", and "GameStation". The device descriptions for electronic devices connected via Bluetooth are, for example, strings including the device model or brand, such as "BEX1850MH" (Bluetooth earphones) and "MITVS26" (Bluetooth audio). In one embodiment, the device descriptions may be displayed on the display interface (e.g., On-screen display, OSD) of the projection device 100 to inform the user of the peripheral devices P_1 to P_N currently connected to the projection device 100.

[0029] If peripheral devices P_1 to P_N are connected to the projection device 100 via wired or wireless connection, peripheral devices P_1 to P_N may provide their device descriptions to the projection device 100 via connection signals. The projection device 100 may read the device descriptions of peripheral devices P_1 to P_N and provide them to the cloud server 400 at predetermined times (for example, when the projection device 100 is started up, when a new peripheral device is detected, at regular intervals, etc.). Each peripheral device has two corresponding device descriptions.

[0030] After the projection device 100 sends a device description to the cloud server 400, in step S510, the cloud server 400, upon receiving the device description from the projection device 100, sends the device description and projector rule commands to the natural language model 200. The projector rule commands are stored, for example, in the cloud server 400. The projector rule commands are configured to restrict the natural language model 200 to generate peripheral device rule commands based on the device description after receiving it. After receiving the device description and projector rule commands, the natural language model 200 generates peripheral device rule commands based on the device description and projector rule commands. In step S515, the natural language model 200 sends the peripheral device rule commands to the cloud server 400.

[0031] In order for the electronic system to control peripheral devices P_1 to P_N, the natural language model 200 needs to generate at least one standard command corresponding to an operation that at least one of the peripheral devices P_1 to P_N can perform, based on a peripheral device rule command from at least one of the peripheral devices P_1 to P_N. Since the peripheral devices P_1 to P_N connected to the projection device 100 may change from time to time, for example, when the user plugs in and unplugs different HDMI-connected electronic devices as needed, or connects to different electronic devices via Bluetooth or Wi-Fi, the device descriptions of the peripheral devices P_1 to P_N currently connected to the projection device 100 need to be sent to the natural language model 200 each time an original command is sent. Based on this, the natural language model 200 can provide standard commands used to perform operations on the peripheral devices P_1 to P_N connected to the projection device 100, and can avoid providing invalid commands that the natural language model 200 cannot perform.

[0032] When the natural language model 200 recognizes a device description, the device description may be the product code, product number, or serial number of a peripheral device. The projector rule directive may be configured to request the natural language model 200 to search its internal database or network for peripheral devices P_1 to P_N corresponding to the device description and to retrieve the software configuration, hardware configuration, and executable operations of the corresponding peripheral devices P_1 to P_N. The natural language model 200 may generate a peripheral rule directive based on the retrieved software configuration, hardware configuration, and executable operations. For example, if the device description is "XX_PS5", the natural language model 200, through the search, learns that the corresponding peripheral device P_1 is a "game console", and associates this peripheral device P_1 with the associated operations of the "game console" and the connection port to which this peripheral device P_1 is connected. Therefore, even if the original command does not explicitly specify which peripheral device the operation should be performed on using which connection method of the projection device 100, if the original command is associated with "game console," when the natural language model 200 converts the original command into a standard command, the standard command becomes a pre-configured command that controls peripheral device P_1 connected to the connection port associated with the device description "XX_PS5."

[0033] In one embodiment, the projector rule command provides a reserved format for the peripheral device rule command, and when the natural language model 200 recognizes a device description, the natural language model 200 writes the product features corresponding to the device description into the variable portion of the reserved format to form the peripheral device rule command.

[0034] In one embodiment, after the natural language model 200 transmits a peripheral device rule command to the cloud server 400, the cloud server 400 may store the peripheral device rule command in association with the corresponding projection device 100, or the cloud server 400 may store the peripheral device rule command in association with the corresponding user account of the projection device 100. In another embodiment, the cloud server 400 may also associate the peripheral device rule command with information such as the product serial number, Internet Protocol (IP) address, and Media Access Control Address (MAC) address. This information is used to indicate the source of the device description. Based on this, when the cloud server 400 later receives an original command issued from the same projection device 100, the cloud server 400 transmits the original command, the projector rule command, and the peripheral device rule command associated with the projection device 100 to the natural language model 200.

[0035] In one embodiment, the device description may be a fixed-format description that is recognized by the natural language model 200. The fixed format refers to a description that is common in the industry and used across multiple products. For example, the natural language model 200 determines the type of each peripheral device P_1 to P_N and the operations that each can perform based on the device description, and generates peripheral device rule commands for each peripheral device P_1 to P_N based on the determination result. One peripheral device corresponds to one peripheral device rule command.

[0036] In one embodiment, the instructions used to generate peripheral device rule instructions by causing the natural language model 200 to recognize the device description do not have to be included in the projector rule instructions and may be implemented via other rule instructions.

[0037] After the user inputs the original command to the projection device 100, in step S520, the projection device 100 sends the original command to the cloud server 400. In step S525, upon receiving the original command from the projection device 100, the cloud server 400 sends the original command, projector rule command, and peripheral device rule command to the natural language model 200 to analyze the original command, and based on the original command, projector rule command, and peripheral device rule command, generates a standard command that corresponds to the original command and can be deciphered by the projection device 100.

[0038] In this embodiment, the natural language model 200 may generate different standard commands based on the type of original command. The types of original commands include single-item control commands, multi-item control commands, and composite control commands. A single-item control command is one in which the request corresponding to the original command requires only the adjustment of one parameter of the projection device 100 or peripheral devices P_1 to P_N, or includes only the command for one operation. For example, the original command is a single-item control command that requires only, for example, to lower the volume of peripheral devices P_1 to P_N, or to switch the connected peripheral devices P_1 to P_N. The natural language model 200 may generate standard commands corresponding to single-item operations. A multi-item control command is one in which the request corresponding to the original command includes commands for multiple single-item operations. For example, the original command is a multi-item control command that concerns, for example, lowering the volume of peripheral devices P_1 to P_N and switching the connected peripheral devices P_1 to P_N. The natural language model 200 may generate standard commands corresponding to multiple single-item operations. A composite control command is one in which the request corresponding to the original command includes the adjustment of multiple parameters of the projection device 100 and / or peripheral devices P_1 to P_N. The original command is, for example, a composite control command that requires adjustment of the visual effects of the projection screen, for example, the original command is "Improve the screen color." For example, the natural language model 200 generates multiple standard commands that adjust multiple parameters of the projection screen, such as color gamut, brightness, and sharpness.

[0039] Projector rule commands are further used to restrict the natural language model 200 to output only commands that the projection device 100 can ultimately decipher, or commands that can be converted into control codes for the projection device 100. For example, projector rule commands may restrict the natural language model 200 to output only standard commands that the projection device 100 can execute, such as "start," "shutdown," "increase volume," "decrease volume," "connect HDMI1," "connect HDMI2," "turn on audio," "switch HDMI source," etc., or their codes, and the natural language model 200 is required to analyze the semantics of the received original command and then reductively convert the original command into one of the above standard commands, or into another standard command.

[0040] The projector rule commands and peripheral device rule commands may be sets of, for example, tens, hundreds, or thousands of characters. The projector rule commands added each time the cloud server 400 sends original commands to the natural language model 200 may be the same. The cloud server 400 may optimize the projector rule commands from time to time, for example by increasing or decreasing the number of conditions in the projector rule commands to achieve a faster feedback rate or more accurate feedback.

[0041] In step S530, the natural language model 200 sends a standard command to the cloud server 400. The cloud server 400 converts the standard command into a corresponding control code. Then, in step S535, the cloud server 400 sends the control code to the projection device 100, causing the projection device 100 to execute the control code to control at least one of the peripheral devices P_1 to P_N, or the projection device 100 simultaneously controls the operation of at least one of the peripheral devices P_1 to P_N and the projection device 100 based on the control code. Alternatively, as shown in step S535', the natural language model 200 may send a standard command to the projection device 100, and the projection device 100 itself may convert the standard command into a corresponding control code to execute the corresponding operation.

[0042] In one embodiment, the processor 110 of the projection device 100 may display a control interface on the display interface of the projection device 100, and the user may input original commands into the control interface. The original commands may be text commands or voice commands. For example, the control interface may include a selection item that can activate the recording function of the sound pickup device (e.g., microphone) of the projection device 100, and after the sound pickup device is activated, the control interface may receive a voice signal via the sound pickup device. After receiving the voice signal, the projection device 100 converts the voice signal into an original command in text format via the processor 110. Alternatively, the control interface may include a selection item that can activate manual input (handwriting pad, touchpad, or keyboard input) to receive an original command in text format entered by the user.

[0043] In other embodiments, the audio signal may be converted into a text-based original command via a terminal device (e.g., a smartphone) paired with the projection device 100. For example, after receiving the audio signal via the projection device 100, the audio signal is transmitted to the paired terminal device via the projection device 100, the audio signal is then converted into a text-based original command via the terminal device, and the original command is transmitted back to the projection device 100. Alternatively, the audio signal may be received using the sound-collecting device (e.g., a microphone) of the terminal device, the audio signal may be converted into a text-based original command, the original command is transmitted back to the projection device 100, and uploaded to the cloud server 400 via the projection device 100. In other cases, the terminal device and the cloud server 400 are connected, and the terminal device may directly upload the original command to the cloud server 400.

[0044] In another embodiment, the projection device 100 or terminal device may, after receiving the audio signal, transmit the audio signal to the cloud server 400, causing the cloud server 400 to convert the audio signal into an original command in text format. After the cloud server 400 receives the audio signal, it may convert the audio signal in natural language format into an original command in text format via its speech recognition model.

[0045] In another embodiment, the original command may be an audio signal in natural language format input by the user, which can be directly uploaded to the natural language model 200, and the original command in audio format can be recognized via the natural language model 200.

[0046] Figure 6 is a schematic diagram of the definition flow of peripheral devices according to one embodiment of the present invention. Referring to Figure 6, in this embodiment, the communication interface 120 includes four connectors, for example, a BT connector 610, an HDMI connector 620, a Wi-Fi connector 630, and a USB connector 640. After the projection device 100 is started up, the processor 110 may execute a monitoring program within the projection device 100 to scan the BT connector 610, HDMI connector 620, Wi-Fi connector 630, and USB connector 640, and obtain device descriptions of peripheral devices P_1 to P_N via the connection signals of each connection method. The monitoring program is stored in the storage 130.

[0047] For the BT connector 610, the processor 110 of the projection device 100 may obtain the device description via the name used when issuing a connection request using the Bluetooth communication protocol. For the HDMI connector 620, the processor 110 of the projection device 100 may obtain the device description via the Consumer Electronics Control (CEC) protocol or the Monitor Control Command Set (MCCS) protocol. For the Wi-Fi connector 630, the processor 110 may obtain the device description when issuing a connection request via the Wi-Fi protocol. For the USB connector 640, the processor 110 may obtain the device description when issuing a connection request via the USB protocol, and each of the peripheral devices P_1 to P_N equally contains the relevant description text in the handshake packet that establishes a link with the projection device 100, and the processor 110 may use these description texts as the device description.

[0048] After obtaining the device description, the projection device 100 uploads the device description to the cloud server 400. Simultaneously, the monitoring program of the projection device 100 may detect in real time whether there has been a change in the connection of the BT connector 610, HDMI connector 620, Wi-Fi connector 630, and USB connector 640. If there is a change, the updated device description is obtained again and uploaded to the cloud server 400. If the monitoring program detects that the connection of one of the peripheral devices has been disconnected, the projection device 100 may send an update command to the cloud server 400 to delete the device description corresponding to the disconnected peripheral device in the cloud server 400.

[0049] After the cloud server 400 receives the device description, the device description and projector rule commands are sent to the natural language model 200. The device description causes the natural language model 200 to identify the product types of peripheral devices P_1 to P_N and generate corresponding peripheral device rule commands, which are then sent to the cloud server 400 and stored there.

[0050] Peripheral device rule commands stored in the cloud server 400 may be overwritten when a new peripheral device rule command is received. For example, if there is a change in peripheral devices P_1 to P_N, the projection device 100 may send a device description to the cloud server 400 again. The cloud server 400 then sends the device description and projector rule command to the natural language model 200 again, causing the natural language model 200 to generate a new peripheral device rule command and send it to the cloud server 400, thereby overwriting the old peripheral device rule command.

[0051] The natural language model 200 may generate a single peripheral rule command or a combination of multiple peripheral rule commands. A combination of multiple peripheral rule commands generates, for example, a separate peripheral rule command for each peripheral device (i.e., each device description). Therefore, if multiple peripheral devices P_1 to P_N are connected to the projection device 100, multiple peripheral rule commands may be generated. Based on the intent of the original command, the natural language model 200 may simultaneously generate standard commands for the projection device 100 and peripheral devices P_1 to P_N based on the projector rule command and the peripheral rule command. Therefore, control results across devices can be achieved through a single original command.

[0052] For example, the projection device 100 is connected to a game console of brand A via an HDMI connector 620 and to an audio device of brand B via a BT connector 610. The processor 110 of the projection device 100 obtains the device description "A_GameStation" of the game console via the CEC protocol and the device description "B_SoundBar" of the audio device via the Bluetooth communication protocol. The processor 110 may then send the device descriptions "A_GameStation" and "B_SoundBar" to the cloud server 400 via the communication interface 120, and the device descriptions and projector rule commands are sent to the natural language model 200 via the cloud server 400.

[0053] The natural language model 200 recognizes the device description "A_GameStation" and retrieves information such as the product type, connection port type, and executable operations (e.g., standby, start, mode switching) of "A_GameStation" from the network, and generates peripheral rule directives (hereinafter abbreviated as "Peripheral Rule A") corresponding to "A_GameStation". Peripheral Rule A is used to restrict the natural language model 200 to generate standard directives only for operations that game consoles of brand A can execute. The natural language model 200 also recognizes the device description "B_SoundBar" and its internal database contains information such as the product type and executable operations corresponding to "B_SoundBar", and generates peripheral rule directives (hereinafter abbreviated as "Peripheral Rule B") corresponding to "B_SoundBar". Peripheral Rule B is used to restrict the natural language model 200 to generate standard directives only for operations that sound devices of brand B can execute.

[0054] Next, the natural language model 200 sends peripheral rule A and peripheral rule B to the cloud server 400. The cloud server 400 is configured to store the user account, IP address, or product serial number corresponding to the projector 100. When the cloud server 400 receives the original command, it sends the original command, the already stored projector rule command, and peripheral rules A and B to the natural language model 200. The natural language model 200 generates at least one standard command based on the requirements of the projector 100, the game console of brand A, and the sound device of brand B in the original command.

[0055] Figure 7 is a schematic diagram of a voice control flow according to one embodiment of the present invention. Before executing the voice control flow, at least one peripheral definition flow may be executed to cause the natural language model 200 to generate peripheral rule commands, thereby enabling a flow that controls the peripheral device only afterward.

[0056] Referring to Figure 7, in step S701, the projection device 100 receives an audio signal. In step S703, the projection device 100 converts the audio signal into an original command in text format, or uses the audio signal directly as the original command. In step S705, the projection device 100 sends the original command to the cloud server 400. In step S707, the cloud server 400 sends the original command, projector rule command, and peripheral device rule command to the natural language model 200. In step S709, the natural language model 200 generates a standard command for the operation corresponding to the original command based on the original command, projector rule command, and peripheral device rule command.

[0057] Assuming the original command is "I want to play a game," the natural language model 200 can analyze the following points: (1) The input signal source of the projector 100 needs to be switched to one of the peripheral devices P_1~P_N currently connected to the HDMI connector 620 (e.g., a game console). (2) One of the peripheral devices P_1~P_N currently connected to the BT connector 610 (e.g., an audio device 800) needs to be activated. (3) The projection mode of the projector 100 should be changed to "game mode." Therefore, based on the projector rule command and the peripheral device rule command, the natural language model 200 generates standard commands such as a first standard command to switch the signal input source to the HDMI source, a second standard command to activate the audio device 800 and adjust the volume or channel to a value corresponding to game mode, and a third standard command to switch the projection mode of the projector 100 to game mode.

[0058] In step S711, the natural language model 200 sends a standard command to the cloud server 400. In step S713, the cloud server 400 converts the standard command into a control code that the projection device 100 can execute, and in step S715, the control code is sent to the projection device 100.

[0059] In step S717, the projection device 100 switches the HDMI source to the game console based on a first standard command. For example, assuming that the HDMI switch includes multiple HDMI input ports (each connected to multiple peripheral devices P_1 to P_N) and one HDMI output port (the output media signal may be provided to the display interface of the projection device 100), the projection device 100 transmits a switch signal to the HDMI switch via the processor 110 based on the first standard command, causing the control circuit of the HDMI switch to select the HDMI input port to be connected by the switch, thereby receiving the multimedia signal provided by the corresponding one of the peripheral devices P_1 to P_N from the connected HDMI input port. The first standard command corresponds to an operation that controls the projection device 100, and the first standard command is, for example, a conversion based on the original command and the projector rule command. In another embodiment, if the original command includes one specific game among multiple games in the game console, the natural language model 200 may also generate a standard command corresponding to the adjustment to the specific game.

[0060] In step S719, the projection device 100 activates the sound device 800 based on the second standard command and drives the sound device 800 to adjust the volume. For example, the projection device 100 activates or restarts the sound device 800 by transmitting an activation signal to the sound device 800 via the processor 110 based on the second standard command. The processor 110 then transmits a corresponding control signal to the control circuit of the sound device 800 based on the second standard command, and the control circuit adjusts the volume or channel to a value corresponding to the game mode. The second standard command corresponds to an operation that controls one of the peripheral devices P_1 to P_N (e.g., the sound device 800), and the second standard command is, for example, a translation based on at least the original command and the peripheral device rule command.

[0061] In step S721, the projection device 100 switches its projection mode to game mode based on the third standard command and applies the parameter settings for game mode (e.g., brightness, color, etc.). For example, the projection device 100 selects game mode from multiple projection modes based on the third standard command via the processor 110, reads the parameter settings for game mode, and then adjusts the settings of the corresponding components of the projection device 100, such as the light source module and display module, based on the parameter settings. The third standard command corresponds to an operation that controls the projection device 100, and the third standard command is, for example, a conversion based on the original command and the projector rule command.

[0062] In summary, the electronic system and control method of the present invention recognize peripheral devices connected to the projection device, provide information about the peripheral devices to a natural language model, generate standard commands corresponding to the natural language model, control the peripheral devices based on the standard commands via the projection device, and allow the projection device to be used as a central control console for the peripheral devices. Based on this, even if the original command issued by the user does not clearly specify the object to be controlled, the electronic system can achieve the user's desired result by simultaneously controlling multiple peripheral devices related to the request of the original command through identification by the natural language model. Furthermore, the user does not need to remember complex function operations, the physical connections of multiple external peripheral devices, or information such as specified IP addresses, system names, and device codes, and can control the projection device in the most convenient way. As a result, even users operating the projection device for the first time or users unfamiliar with operating peripheral devices can use the peripheral devices without difficulty and adjust parameter settings for the best viewing, sound, and other effects.

[0063] The above-described embodiments are merely preferred embodiments of the present invention and do not limit the scope of implementation of the present invention. All simple equivalent modifications and alterations made in accordance with the claims and content of the invention remain within the scope of the patent of the present invention. No embodiment or claim of the present invention may necessarily achieve all the purposes, advantages, or features disclosed herein. Furthermore, the abstract and title (title of the invention) are used only to assist in searching patent documents and are not intended to limit the scope of the rights of the present invention. In addition, terms such as "first," "second," etc., as used herein or in the claims, are used only to name elements or to distinguish different embodiments or scopes and are not used to limit the upper or lower limits on the quantity of elements. [Industrial applicability]

[0064] The electronic system and control method of the present invention can be applied to a projection device to control peripheral devices connected to it. [Explanation of symbols]

[0065] 100: Projection device 110: Processor 120: Communication Interface 130: Storage P, P_1~P_N: Peripherals 200: Natural Language Models 400: Cloud Server 610: BT connector 620: HDMI connector 630: Wi-Fi connector 640: USB connector 800: Sound equipment S305~S315: Step S505~S335: Step S701~S721: Step

Claims

1. A method for controlling an electronic system including a projection device and at least one peripheral device, A step of transmitting an original command to a natural language model via the projection device, wherein the projection device is equipped with a communication interface and communicates with at least one peripheral device via the communication interface, The steps include generating at least one standard command based on the original command via the natural language model, in accordance with the fact that the original command corresponds to the operation of at least one peripheral device, A step of controlling at least one peripheral device by executing at least one control code via the projection device, wherein the at least one control code corresponds to executing the operation corresponding to the original command, and the at least one control code is generated based on the at least one standard command via the projection device or a cloud server connected to the projection device. including Characterized by, A method for controlling electronic systems.

2. The steps include generating at least one standard command based on the original command via the natural language model, in accordance with the fact that the original command corresponds to the operation of the projection device and at least one peripheral device, A step of controlling the projection device and the at least one peripheral device via the projection device to perform the operation corresponding to the original command based on the at least one standard command. Includes Characterized by, A method for controlling the electronic system described in claim 1.

3. The step of generating the at least one standard command based on the original command via the natural language model is: To generate the at least one standard command based on the original command, the projector rule command, and the at least one peripheral device rule command via the natural language model. Includes, The projector rule command is used to make the at least one standard command generated by the natural language model conform to an operation that the projection device can perform. The at least one peripheral rule directive is used to conform the at least one standard directive generated by the natural language model to an operation that the at least one peripheral device can perform. Characterized by, A method for controlling the electronic system described in claim 2.

4. The steps include obtaining at least one device description of at least one peripheral device connected to the communication interface via the projection device, and transmitting the at least one device description to the cloud server, wherein the projector rule command is stored in the cloud server. The steps include: transmitting the at least one device description and the projector rule command to the natural language model in response to receiving the at least one device description from the projection device via the cloud server; The steps include generating the at least one peripheral device rule command based on the at least one device description and the projector rule command via the natural language model, and sending the at least one peripheral device rule command to the cloud server. It further includes, The step of generating the at least one standard command based on the original command via the natural language model is: Upon receiving the original command from the projection device via the cloud server, the original command, the projector rule command, and the at least one peripheral device rule command are transmitted to the natural language model, causing the natural language model to generate the at least one standard command. including Characterized by, A method for controlling the electronic system described in claim 3.

5. In response to the projection device detecting that the at least one device description of the at least one peripheral device connected to the communication interface has changed, the projection device obtains the updated at least one device description and transmits it to the cloud server, thereby causing the natural language model to generate a corresponding at least one peripheral rule command based on the updated at least one device description. Includes Characterized by, A method for controlling the electronic system described in claim 4.

6. After the step of sending the at least one peripheral rule command to the cloud server via the natural language model, The steps of associating the at least one peripheral device rule command with the projection device via the cloud server and then storing it in the cloud server, Or, The step of associating the account of the user of the projection device with the at least one peripheral device rule command via the cloud server and then storing it on the cloud server. Includes Characterized by, A method for controlling the electronic system described in claim 4.

7. The at least one peripheral rule directive includes at least one of the product type of the at least one peripheral and the description of the communication interface to which the at least one peripheral connects. Characterized by, A method for controlling the electronic system described in claim 3.

8. The step of converting the audio signal into the original command in text form via the projection device in response to the sound receiving device receiving the audio signal. Or, In response to the projection device receiving the audio signal, the projection device converts the audio signal into the original command in text format via a terminal device paired with the projection device. Or, The projection device or the terminal device, upon receiving the audio signal, transmits the audio signal to the cloud server via the projection device or the terminal device, causing the cloud server to convert the audio signal into the original command in text format. Includes Characterized by, A method for controlling the electronic system described in claim 1.

9. The step of converting the audio signal into the original command by transmitting the audio signal to the natural language model via the projection device in response to the sound receiving device receiving the audio signal. Includes Characterized by, A method for controlling the electronic system described in claim 1.

10. It is an electronic system, At least one peripheral device, Projection device and Includes, The projection device is used to receive original commands, and the projection device includes a communication interface and a processor. The communication interface is connected to the at least one peripheral device. The processor is coupled to the communication interface, and the processor is In accordance with the fact that the original command corresponds to the operation of the at least one peripheral device, the communication interface receives at least one standard command corresponding to the at least one original command. The system controls the at least one peripheral device by executing at least one control code, the at least one control code corresponds to executing the operation corresponding to the original command, and the at least one control code is generated based on the at least one standard command via the projection device or a cloud server connected to the projection device. so Characterized by, Electronic systems.

11. The aforementioned processor, In accordance with the fact that the original command corresponds to the operation of the projection device and the at least one peripheral device, the at least one standard command corresponding to the original command is received via the communication interface. Based on the at least one standard command, the projector and the at least one peripheral device are controlled to perform the operation corresponding to the original command. so Characterized by, The electronic system according to claim 10.

12. Further including natural language models, The projection device is connected to the natural language model via the communication interface, and the natural language model is used to generate at least one standard command based on the original command, the projector rule command, and at least one peripheral device rule command. The projector rule command is used to make the at least one standard command generated by the natural language model conform to an operation that the projection device can perform. The at least one peripheral rule directive is used to conform the at least one standard directive generated by the natural language model to an operation that the at least one peripheral device can perform. Characterized by, The electronic system according to claim 11.

13. The aforementioned cloud server further includes, The projection device connects to the cloud server via the communication interface, The processor is configured to obtain, via the communication interface, at least one device description of the at least one peripheral device connected to the communication interface. The cloud server stores the projector rule commands, and the cloud server is configured to transmit the at least one device description and the projector rule commands to the natural language model in response to receiving the at least one device description from the projection device. The natural language model is configured to receive the at least one device description and the projector rule command, generate the at least one peripheral device rule command based on the at least one device description and the projector rule command, and send the at least one peripheral device rule command to the cloud server. Characterized by, The electronic system according to claim 12.

14. The cloud server is configured to transmit the original command, the projector rule command, and the at least one peripheral device rule command to the natural language model in response to receiving the original command from the projection device. The natural language model is configured to generate the at least one standard command based on the original command, the projector rule command, and the at least one peripheral rule command, in response to receiving the original command, the projector rule command, and the at least one peripheral rule command. Characterized by, The electronic system according to claim 13.

15. The processor is configured to detect a change in the device description of at least one peripheral device connected to the communication interface, and in response to this, retrieve the updated device description via the communication interface and send it to the cloud server, thereby causing the natural language model to generate a corresponding at least one peripheral rule command based on the updated device description. Characterized by, The electronic system according to claim 13.

16. The aforementioned cloud server is The at least one peripheral device rule command is associated with the projection device and stored in the cloud server. Or, The at least one peripheral device rule command is associated with the user account of the projection device and stored on the cloud server. so Characterized by, The electronic system according to claim 13.

17. The at least one peripheral rule directive includes at least one of the product type of the at least one peripheral and the description of the communication interface to which the at least one peripheral is connected. Characterized by, The electronic system according to claim 12.

18. The processor is configured to convert the audio signal into the original command by transmitting the audio signal to the natural language model via the communication interface in response to the sound acquisition device receiving the audio signal. Characterized by, The electronic system according to claim 12.

19. The aforementioned processor, In response to receiving an audio signal, the sound receiving device converts the audio signal into the original command in text format. Or, Upon receiving the audio signal, the processor transmits the audio signal to the cloud server, causing the cloud server to convert the audio signal into the original command in text format. Structured in such a way Or, Further including terminal devices, The terminal device is coupled to the projection device, and the terminal device, Upon receiving the audio signal, the terminal device converts the audio signal into the original command in text format. Or, Upon receiving the audio signal, the terminal device transmits the audio signal to the cloud server, causing the cloud server to convert the audio signal into the original command in text format. so Characterized by, The electronic system according to claim 10.

Images