Autonomous electromagnetic intelligent agent system based on programmable metasurface

Abstract

The invention discloses an autonomous electromagnetic intelligent agent system based on a programmable metasurface, and belongs to the technical field of intelligent robots. The system comprises a cerebellum module based on a programmable metasurface and a brain module based on a high-capacity basic model, and has human-level intelligent autonomous beam control capability; the cerebellum module uses a programmable metasurface to execute an electromagnetic wave manipulation task; the brain module is used for generating a high-level task strategy, decomposing a high-level task and issuing an obtained executable subtask sequence to the cerebellum module; the brain module and the cerebellum module work on the two hosts respectively, and data transmission is carried out between the brain module and the cerebellum module through a network communication protocol. The autonomous electromagnetic intelligent agent system exists in an intelligent scene in the form of an intelligent housekeeper, and can be widely applied to various intelligent scenes such as intelligent families, intelligent factories and intelligent communities.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent robot technology, and relates to artificial intelligence, giant body intelligence, robot and electromagnetic perception cross technology, large language model technology, and human-computer interaction technology, and specifically to an electromagnetic metasurface intelligent body system based on a programmable metasurface. Background Art

[0002] With the rapid development of modern information technology, autonomous intelligent products driven by embodied intelligence technology, large language model technology, Internet of Things technology, 5G communication technology, and artificial intelligence technology have gradually been applied to people's daily lives. Therefore, with the gradual implementation of these technologies, autonomous intelligent bodies will also promote smart scenes such as smart homes and smart factories to gradually enter our real lives. On the other hand, due to the continuous technological improvement and functional enhancement of smart devices, how to build an autonomous intelligent body as the smart official of a smart home or factory, so that it can communicate with users through language and can autonomously understand the environment, and even users only need simple gestures to make the autonomous body understand the user's needs, has become an urgent need for the current evolving intelligent environment.

[0003] Existing technologies are still at the stage of voice control of various devices, and some devices can only perform simple human body detection on users in local areas. Existing technologies are difficult to achieve full coverage of perception and control of the entire smart scene, especially in some private areas, and are also unable to perceive and understand the user's intentions anytime and anywhere, and even provide effective help to users when they are in dangerous situations.

[0004] Programmable metasurfaces are ultra-thin engineering structures that have been recognized as an important medium for flexible manipulation of electromagnetics and have been valuable applications in many fields such as perception, control, communication, and computing. Electromagnetic sensing technology based on programmable metasurfaces can achieve real-time, all-weather intelligent perception of users' postures, behaviors, and health status in indoor environments, and microwave-based sensing technology will not cause the problem of easily invading user privacy like visual perception. In addition, programmable metasurfaces can also achieve remote and precise control of wireless devices in the environment, allowing smart devices to respond quickly without causing the risk of user data leakage like traditional communication methods.

[0005] Recently, large-capacity foundation models (LFMs), especially large language models (LLMs), have achieved remarkable success in achieving human-level intelligence, such as ChatGPT, BERT, Wenxinyiyan, etc. LLM technology will bring great impetus to the development of service-oriented autonomous agents. Autonomous agents supported by large language models will serve as intelligent butlers in future intelligent scenarios. Users can interact with autonomous agents anytime and anywhere. After understanding the user's intention, the autonomous agent will dispatch specific execution agencies to complete the specific tasks assigned by the user.

[0006] Autonomous agents that combine programmable metasurfaces with large language models will be able to play a variety of roles in future smart environments, such as senior caregivers, family doctors, security guards, chefs, etc. No matter where the user is in the environment, the smart butler can perceive the user's behavior and status in real time through the programmable metasurface, and make correct decisions with the powerful reasoning and understanding capabilities of the large language model, and then decompose the task and execute it step by step. Therefore, the autonomous electromagnetic agent system based on programmable metasurfaces will become an effective way to perceive complex environments and perform complex tasks in the next generation of smart homes or smart factories in the future. Summary of the invention

[0007] In order to address the above-mentioned deficiencies in the prior art, the present invention provides an autonomous electromagnetic intelligent agent system based on a programmable metasurface, called metaAgent.

[0008] The metaAgent system of the present invention consists of two parts: one part is a cerebellum module based on a programmable metasurface, which uses the programmable metasurface to perform specific electromagnetic wave manipulation tasks; the other part is a brain module based on a large-capacity basic model LFMs, which is used to generate high-level task strategies, decompose complex tasks into a series of simple executable subtask sequences, and send the executable subtask sequences to the cerebellum module. In the present invention, the programmable metasurface used by the metaAgent system is called a semantically programmable metasurface (SPMs), which is different from the traditional programmable metasurface. The semantically programmable metasurface performs beam control through a semantic coding mode. In the present invention, the semantic coding mode reflects that the brain of the metaAgent system uses high-level natural language to formulate prompts during the reasoning process, rather than low-level binary digital sequences in traditional programmable metasurfaces. According to multimodal input data (including text, voice, image, microwave signal), the brain module of the metaAgent will initiate an action plan, including commanding each SPMs in the cerebellum module to obtain supplementary perception data, command robot entities, etc. Therefore, the autonomous electromagnetic intelligent system metaAgent of the present invention will exist in the form of a smart housekeeper in the smart scene. The user (metaAgent user) and metaAgent can communicate freely through language. At the same time, when the user has dangerous behaviors such as falling, metaAgent will also actively perform behavioral posture recognition and health sign detection, quickly perceive the user's current dangerous situation (such as user falling, rapid heartbeat, etc.), and take corresponding actions according to the actual situation, thereby helping the user out of danger. The present invention can be widely used in various smart scenes in the future (such as smart homes, smart factories, smart communities, etc.).

[0009] The technical solution adopted by the autonomous electromagnetic intelligent body system of the present invention is as follows:

[0010] An autonomous electromagnetic agent system metaAgent based on a programmable metasurface includes: a brain module for formulating high-level strategies and a cerebellum module for performing specific electromagnetic manipulation tasks; the metaAgent system is based on programmable metasurface beam manipulation technology and autonomous agent technology, and has autonomous beam manipulation capabilities with human-level intelligence; the details are as follows:

[0011] The metaAgent brain module includes: memory submodule, perception expert submodule, planning expert submodule, grounding expert submodule and encoding expert submodule. Among them, the memory submodule consists of an action library and a memory library; the action library is used to store the device library composed of all devices that the metaAgent can control and the skill library composed of all skills that it can perform; the memory library is used to store all historical records during the operation of the metaAgent system and the environmental information corresponding to the operating environment, mainly including the visual semantic map of the user's environment, which is saved in the form of a knowledge graph; the perception expert submodule continuously collects multimodal data as input, including the sound recorded by the microphone, the image recorded by the camera, the text entered by the keyboard, and the microwave signal collected by the software defined radio (USRP), which is processed by the system-integrated deep neural network model (a specific type of model is selected for a specific task), so as to summarize the output semantic results for the planning expert submodule. The perception expert submodule selects appropriate deep neural networks or other signal processing methods to analyze and understand the input information. In terms of image and sound processing, models pre-trained on large-scale data sets can be used. For example, the Xunfei API can identify the speaker's identity and speech content, and the stereo depth camera can achieve multi-camera fusion for human key point detection. The mmetaAgent system's microwave perception of the environment is based on multiple programmable metasurfaces working in different frequency bands of the cerebellum module, which can be used to achieve various perception tasks (such as skeleton key point detection, user positioning, behavior recognition, breathing and heartbeat monitoring, etc.). Microwave perception uses programmable metasurfaces to detect the environment and interprets the measured signals using artificial neural networks trained through supervised learning. The above-mentioned perception methods (including visual perception, speech perception, and microwave perception) are also shared with the encoding expert submodule. Microwave perception can be used as a supplement to visual perception when optical perception is ineffective (such as at corners or behind opaque layers) or unavailable (such as for privacy reasons). On the other hand, microwaves have the ability to penetrate perception, so they can be safely applied in some areas where it is not suitable to deploy visual sensors due to user privacy. The perception expert submodule synthesizes the understanding of multimodal input data in natural language and passes it to the planning expert submodule. The planning expert submodule uses the knowledge provided by the memory submodule to analyze the natural language input from the perception expert submodule or the encoding expert submodule and decompose the natural language input into a series of executable subtask sequences. The executable subtask sequence to be executed is then passed to the grounding expert submodule in the form of natural language. The planning expert submodule is implemented based on the LLM that provides some contextual demonstrations. In addition, the planning expert submodule is improved based on the human feedback stored in the memory bank by the memory submodule. The encoding expert submodule takes as input a triple consisting of the task sequence generated by the planning expert submodule, the function of the required action generated by the grounding expert module, and the natural language device.In particular, the coding expert submodule checks whether the various required subtasks can be performed in parallel or need to be performed sequentially. The coding expert submodule then writes python code and runs it automatically on the host. The output of the coding expert submodule is to generate two outputs in natural language: one is the "external" output, which is used to communicate with human users through a chat tool; the other is the "internal" output, which is provided to the planning expert submodule and the grounding expert submodule for consideration. The "internal" output is a key part of the metaAgent system's reasoning in natural language through discussions with the above-mentioned multiple experts (perception experts, planning experts, grounding experts, and coding experts), and the autonomous self-organization skills of the metaAgent system are built on this basis. The coding expert submodule is designed based on the LLM with some context demonstrations.

[0012] The cerebellum module of metaAgent includes: programmable metasurfaces working in different frequency bands, Field Programmable Gate Array (FPGA) control module, Universal Software Radio Peripheral (USRP) module, transmitting antenna and receiving antenna, visual sensor module, text input module, and voice module; among them, the programmable metasurface is composed of 32*24 independently controllable control units (the structure of the control unit is mainly composed of a rectangular metal resonant patch, a phase bias line, a dielectric substrate and a metal reflector), working in the 2.4GHz, 5.5GHz and 9.7GHz frequency bands. Each programmable metasurface is controlled by an FPGA control module, and the USRP module is responsible for real-time collection of microwave data through the transceiver antenna. The USRP module is connected to the host, and all control algorithms of the metaAgent system run on the host. The visual perception module is composed of multiple ZED2 depth cameras, which are used to collect image information of the environment; the text input module is composed of a keyboard connected to the host, which is used to collect text input information from the user; the voice module is composed of a microphone and a speaker, and the user can interact with the metaAgent in real time through voice.

[0013] The brain module and cerebellum module of metaAgent work on two hosts respectively, and data is transmitted between them through the network communication protocol.

[0014] By using the above-mentioned autonomous electromagnetic intelligent agent system based on programmable metasurface, flexible interaction between metaAent and the user can be realized. When the autonomous electromagnetic intelligent agent system works, it may include the following steps:

[0015] 1) The brain module of the autonomous electromagnetic intelligent body system receives the sensor model data transmitted by the cerebellum module;

[0016] 2) The perception expert submodule of the brain module receives data in multiple modalities (text, voice, image, microwave), analyzes the data in multiple modalities through the corresponding deep learning network model, and summarizes the semantic information in a specific format as the output result;

[0017] 3) The planning expert submodule receives the semantic information output by the perception expert submodule, decomposes the specific tasks according to the knowledge of the memory submodule, and generates a series of executable subtask sequences;

[0018] 4) The grounding expert submodule assigns corresponding executable equipment and corresponding skills to each subtask based on the knowledge of the memory submodule;

[0019] 5) The coding expert submodule writes Python code that can be executed on the host according to the subtask sequence and the assigned executable devices and skills;

[0020] 6) The cerebellum module receives the Python code sent by the brain module and executes it on the host;

[0021] 7) The semantically programmable hypersurface of the cerebellum module executes the corresponding Python code to complete the corresponding task;

[0022] 8) The cerebellum module outputs the task execution results of the semantic programmable hypersurface to the user and also feeds back to the brain module;

[0023] 9) The brain module updates the memory submodule.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] The present invention proposes an autonomous electromagnetic intelligent body system based on a programmable metasurface; the present invention has the following advantages:

[0026] (i) The present invention designs an intelligent agent for autonomous electromagnetic manipulation that combines a programmable metasurface with a large language model, which can interact with users flexibly and freely anytime and anywhere through natural language.

[0027] (ii) The present invention utilizes the powerful wide field of view and penetrable sensing capabilities of the programmable metasurface to achieve all-round perception and control of users, robots and environments in smart scenes, including key point detection of the human body's three-dimensional skeleton, breathing and heartbeat detection, object positioning, signal enhancement and robot motion control.

[0028] (III) The metaAgent system framework proposed in the present invention is scalable and can be customized according to the specific needs of specific intelligent scenarios.

[0029] (IV) The metaAgent system proposed in the present invention has data security and can fully protect the user's personal privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a structural block diagram of the autonomous electromagnetic intelligent body system based on programmable metasurface of the present invention.

[0031] Figure 2 It is a schematic diagram of the working process of the autonomous electromagnetic intelligent body system based on the programmable metasurface in an embodiment of the present invention.

[0032] Figure 3 It is a visualization interface designed for an autonomous electromagnetic intelligent system based on a programmable metasurface in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention will be further described below by way of embodiments in conjunction with the accompanying drawings, but the scope of the present invention is not limited in any way.

[0034] The present invention provides an autonomous electromagnetic intelligent body system based on a programmable metasurface, the system comprising: a brain module based on a large-capacity basic model and a cerebellum module based on a programmable metasurface. The present invention uses the programmable metasurface as a medium for manipulating electromagnetic waves in the environment, and formulates high-level strategies for the brain module constructed with the help of LFMs, thereby controlling the cerebellum module based on the programmable metasurface to interact with people, robots and objects in the environment, thereby autonomously completing a series of complex human-computer interaction tasks.

[0035] The autonomous electromagnetic intelligent body system based on programmable metasurface provided by the present invention consists of only two core parts. The brain module is designed based on a large-capacity basic model, including a memory module, a perception expert, a planning expert, a grounding expert and a coding expert; the cerebellum module is designed based on a programmable metasurface, including a variety of perception modules and programmable metasurfaces of different working frequency bands; the brain module is responsible for autonomously understanding user needs and formulating strategies based on multi-modal perception data input, decomposing tasks into a series of subtasks and entrusting them to the cerebellum for execution; the cerebellum module is responsible for receiving a series of task sequences from the brain and using the programmable metasurface to perform specific electromagnetic manipulation tasks.

[0036] The memory module consists of an action library and a memory library; the action library is used to store the device library consisting of all devices that the system can control and the skill library consisting of all skills that it can perform; the memory library is used to store all historical records during the operation of the system and the environmental information corresponding to the operating environment, mainly including the visual semantic map of the environment, which is saved here in the form of a knowledge graph.

[0037] The perception expert module consists of multiple deep learning models, which are used to convert perception data of multiple modalities (such as text, sound, vision, microwave, etc.) into semantic results of language modality.

[0038] The planning expert module is designed based on a large language model and uses the knowledge provided by the memory module to analyze the natural language input from the perception expert or encoding expert and decompose it into a series of executable subtasks.

[0039] The grounding expert module is designed based on a large language model and is responsible for receiving the subtask list from the planning expert and assigning appropriate skills and related equipment to each subtask.

[0040] The coding expert module is designed based on a large language model. It is responsible for writing Python code that can be run on the host based on the three types of knowledge input in natural language from the previous expert: expected goals, corresponding action skills, and equipment.

[0041] The cerebellum module consists of multiple semantically programmable metasurfaces, a microphone module, a camera module, a text input module, etc.

[0042] The semantic programmable metasurface consists of a programmable metasurface, a microcontroller, a universal software radio peripheral (USRP), a transceiver antenna, and a host. The programmable metasurface consists of 32*24 independently controllable units, operating in the 2.4GHz, 5.5GHz, and 9.7GHz frequency bands; the microcontroller is connected to the programmable metasurface and is composed of a field programmable gate array (FPGA) module; the universal software radio peripheral is connected to the transceiver antenna and is connected to the host through a network port, and the semantic programmable metasurface control algorithm runs on the host;

[0043] In the autonomous electromagnetic intelligent body system based on programmable metasurface, further, the programmable metasurface includes but is not limited to programmable metasurfaces operating in the 2.4 GHz, 5.5 GHz and 9.7 GHz frequency bands.

[0044] Perception experts, planning experts, grounding experts, and coding experts form an expert group chat to develop high-level strategies in the form of natural language discussions.

[0045] When the autonomous electromagnetic intelligent system based on programmable metasurface works, the brain perception expert conveys the task to the planning expert by analyzing the environmental data or user instructions. The planning expert is responsible for task decomposition and the grounding expert allocates the corresponding equipment and skills. Then the coding expert generates the corresponding code, which is finally handed over to the programmable metasurface of the cerebellum for execution, thus realizing human-computer interaction. It includes the following steps:

[0046] 1) The brain receives the sensor model data transmitted by the cerebellum;

[0047] 2) Perception experts receive data in multiple modalities (text, speech, image, microwave), analyze the data through the corresponding deep learning network model, and summarize the output results into semantic information in a specific format;

[0048] 3) The planning expert will receive the semantic information output by the perception expert, decompose the specific task according to the knowledge of the memory module, and generate a series of executable subtask sequences;

[0049] 4) The grounding expert assigns corresponding executable equipment and corresponding skills to each subtask based on the knowledge of the memory module;

[0050] 5) The coding expert writes Python code that can be run on the host according to the subtask sequence and the assigned equipment and skills;

[0051] 6) The cerebellum receives the Python code sent by the brain and executes it on the host;

[0052] 7) The semantically programmable hypersurface executes the corresponding code to complete the corresponding task;

[0053] 8) The cerebellum outputs the task execution results to the user and also feeds back to the brain;

[0054] 9) The brain updates memory modules.

[0055] In the above steps 4), 5), and 6), the planning expert, the grounding expert, and the coding expert use the same memory module; the skills corresponding to the programmable metasurface stored in the memory module include but are not limited to: beam focusing, human 3D skeleton recognition, user positioning, user behavior recognition, user health detection, object positioning, robot positioning, communication link enhancement, robot control, etc.; for the cerebellum module ((ID meta ,C)) can be expressed by the function π:

[0056] (ID meta ,C)=π(l,other inputs;KG) (1)

[0057] Among them, ID meta is the index of the programmable hypersurface; C is the encoding mode; l is the natural language prompt based on the stored knowledge; other inputs represents other inputs; KG represents the knowledge graph; the function π converts the natural language prompt l based on the stored knowledge (in the form of the knowledge graph (KG)) and other inputs into the index ID of the selected programmable hypersurface meta and spatial (or spatiotemporal) coding mode C.

[0058] Embodiment 1

[0059] In this embodiment, the autonomous electromagnetic intelligent body system architecture based on the programmable metasurface is as follows: Figure 1 The workflow of the autonomous electromagnetic intelligent system based on programmable metasurface is shown in Figure 2 As shown in Figure 2. Among them, the brain module of metaAgent is as follows Figure 1 As shown in the upper left corner, the cerebellum module of metaAgent is as follows Figure 1 Shown below left. Figure 3 It is a visualization interface designed for the autonomous electromagnetic intelligent body system based on programmable metasurface of the present invention.

[0060] In this embodiment, the interaction process between the user and the metaAgent system is as follows: Figure 2 The details are as follows:

[0061] 1) The user inputs a command via voice: "Please detect Alice's breathing rate!";

[0062] 2) metaAgent first starts the perception environment (perception expert submodule) to analyze multiple modal data and obtains the user task language description;

[0063] 3) After receiving the task language description from the perception expert submodule, the planning expert submodule of metaAgent decomposes it into a subtask sequence: 1) locate Alice; 2) measure Alice's breathing rate.

[0064] 4) The ground expert submodule assigns corresponding equipment and skills to the subtask sequence: 1) Position Alice (equipment: 2.4GHz-SPM, skill: user positioning); 2) Measure Alice's breathing rate (equipment: 5.5GHz-SPM, skill: h breathing detection);

[0065] 5) The coding expert submodule writes the corresponding Python code based on the above results (corresponding equipment and skills);

[0066] 6) The cerebellum module receives the Python code and executes it;

[0067] 7) If the code is executed successfully, the result is output to the user through the microphone, and the result is passed to the planning expert submodule for task summary and analysis. If the planning expert submodule believes that there is no need for the next step, it outputs "END" and ends this interaction.

[0068] 8) If the code is written incorrectly, it will be returned to the coding expert submodule for rewriting.

[0069] Finally, it should be noted that the purpose of publishing the embodiments is to help further understand the present invention, but those skilled in the art can understand that various substitutions and modifications are possible without departing from the scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the contents disclosed in the embodiments, and the scope of protection claimed by the present invention shall be subject to the scope defined in the claims.

Claims

1. An autonomous electromagnetic intelligent agent system based on a programmable metasurface, characterized in that: The system includes a cerebellum module based on a programmable metasurface and a brain module based on a large-capacity basic model. The system has autonomous beam steering capabilities with human-level intelligence. The cerebellum module uses a programmable metasurface to perform electromagnetic wave manipulation tasks; The brain module is used to generate high-level task strategies and decompose high-level tasks to send the resulting executable subtask sequences to the cerebellum module; The brain module and cerebellum module work on two hosts respectively, and data is transmitted between them through a network communication protocol; The brain module includes: memory submodule, perception expert submodule, planning expert submodule, grounding expert submodule and encoding expert submodule; among them, The memory submodule consists of an action library and a memory library; the action library is used to store the device library consisting of all devices that the autonomous electromagnetic intelligent system can control and the skill library consisting of all skills that it can perform; the memory library is used to store all historical records during the operation of the system and the environmental information corresponding to the operating environment; The perception expert submodule is used to take the continuously collected multimodal data signals as input, and after processing, output semantic results for the planning expert submodule; the signal processing method adopts deep neural networks, including using models pre-trained on large-scale data sets to process images and sounds, or using programmable metasurfaces to detect the environment and using artificial neural networks trained through supervised learning to perceive and measure microwave signals; The planning expert submodule uses the knowledge provided by the memory submodule to analyze the natural language input from the perception expert submodule or the encoding expert submodule, and decomposes the natural language input into a series of executable subtask sequences; the executable subtask sequence to be executed is transmitted to the grounding expert submodule in the form of natural language; The coding expert submodule is used to form a triplet of the task sequence generated by the planning expert submodule, the function of the required action generated by the grounding expert module and the natural language device as input; write the code and run it automatically on the host; the coding expert submodule generates two outputs in natural language: the external output is used to communicate with human users through the chat tool; the internal output is inferred by natural language after discussion by multiple expert submodules and provided to the planning expert submodule and the grounding expert submodule; The cerebellum module includes: a programmable metasurface operating in different frequency bands, a field programmable gate array control module, a general software radio peripheral module, a transmitting antenna and a receiving antenna, a visual sensor module, a text input module, and a voice module; among which: The programmable metasurface is composed of multiple independently controllable control units, operating in multiple frequency bands; each programmable metasurface is controlled by a field programmable gate array control module; The general software radio peripheral module is used to collect microwave data in real time through the transceiver antenna; the general software radio peripheral module is connected to the host; the control algorithms are all run on the host; The visual perception module is used to collect image information of the environment; the text input module is used to collect text input information from users; and the voice module is used for users to interact with the system in real time through voice.

2. The autonomous electromagnetic intelligent agent system based on a programmable metasurface as claimed in claim 1, characterized in that: In the brain module, the memory bank in the memory submodule is used to store system information including a visual semantic map of the user's environment saved in the form of a knowledge graph.

3. The autonomous electromagnetic intelligent agent system based on a programmable metasurface as claimed in claim 1, characterized in that: The multimodal data continuously collected by the perception expert submodule includes sounds recorded by microphones, images recorded by cameras, text entered by keyboards, and microwave signals collected by software-defined radios.

4. The autonomous electromagnetic intelligent agent system based on a programmable metasurface as claimed in claim 1, characterized in that: Furthermore, the planning expert submodule is implemented based on the LLM that provides contextual demonstrations and is improved based on the human feedback stored in the memory bank by the memory submodule.

5. The autonomous electromagnetic intelligent agent system based on programmable metasurface according to claim 1, characterized in that: The structure of the control unit of the programmable metasurface includes a rectangular metal resonant patch, a phase bias line, a dielectric substrate and a metal reflector; the operating frequency bands include 2.4 GHz, 5.5 GHz and 9.7 GHz.

6. The autonomous electromagnetic intelligent agent system based on a programmable metasurface as claimed in claim 1, characterized in that: The visual perception module is composed of multiple ZED2 depth cameras; the text input module is composed of a keyboard connected to the host; and the voice module is composed of a microphone and a speaker.

7. The autonomous electromagnetic intelligent agent system based on programmable metasurface according to claim 1, characterized in that: The cerebellum module is represented by the function π as follows: (ID meta ,C)=π(l,other inputs;KG) Among them, ID meta is the index of the programmable hypersurface; C is the encoding mode; l is the natural language prompt based on the stored knowledge; otherinputs represents other inputs; KG represents the knowledge graph; the function π converts the natural language prompt l and other inputs based on the knowledge stored in the form of the knowledge graph into the index ID of the selected programmable hypersurface meta and spatial or spatiotemporal coding mode C.

8. The autonomous electromagnetic intelligent agent system based on programmable metasurface as claimed in claim 1, characterized in that: The working steps of the autonomous electromagnetic intelligent agent system include: 1) The brain module receives the sensor model data transmitted by the cerebellum module; 2) The perception expert submodule receives multi-modal data, analyzes the data through the corresponding deep learning network model, and converts the output results into semantic information in a specific format; 3) The planning expert submodule receives the semantic information output by the perception expert submodule, decomposes the task according to the knowledge of the memory submodule, and generates a series of executable subtask sequences; 4) The grounding expert submodule assigns corresponding executable equipment and corresponding skills to each subtask based on the knowledge of the memory submodule; 5) The coding expert submodule writes program codes that can be executed on the host according to the subtask sequence and the assigned equipment and skills; 5) The cerebellum module receives the program code sent by the brain module and executes it on the host; 7) The semantically programmable hypersurface executes the corresponding code to complete the corresponding task; 8) The cerebellum module outputs the task execution results to the user and also feeds back to the brain module; 9) The brain module updates the memory submodule.

Images