Air-ground integrated ris energy transmission system based on vision and gesture posture

Abstract

The application provides an air-ground integrated RIS energy transmission system based on vision and gesture posture, and belongs to the technical field of wireless energy transmission.The system comprises the following: a visual perception detection unit synchronously collects RGB and depth images of the same physical scene, and the physical scene comprises an operator and at least one energy receiving module; position information of the at least one energy receiving module in a three-dimensional space is obtained based on the collected images; a pointing intention analysis unit analyzes a ray representing the pointing intention of the operator based on the collected images; a target intention matching unit matches the three-dimensional space pointing ray with the position information, so as to determine a target object; a control calculation unit calculates phase control parameters required for energy focusing according to the position of the target object; and an RIS energy transmission unit adjusts the radiation characteristics of an electromagnetic beam according to the phase control parameters, so as to focus energy to the target object. The application captures gesture pointing and locks a moving target through vision, drives RIS to focus energy, and realizes precise energy transmission.

Description

Technical Field

[0001] This invention belongs to the field of wireless power transmission technology, and specifically relates to an air-ground integrated RIS power transmission system based on vision and gestures. Background Technology

[0002] With the widespread application of mobile devices such as unmanned aerial vehicles (UAVs) and autonomous mobile robots (AGVs), their battery life has become a key bottleneck restricting task execution efficiency. Traditional fixed or mechanical scanning wireless charging systems struggle to achieve efficient and flexible energy transfer in dynamic, multi-target scenarios.

[0003] Reconfigurable smart surfaces (RIS), as an emerging electromagnetic metamaterial technology, can programmatically control the amplitude and phase of electromagnetic waves in real time, thereby achieving flexible beam deflection and focusing, providing a new solution for long-distance, highly directional wireless power transmission. However, existing RIS power transmission systems generally lack an efficient, natural, and robust target selection and locking mechanism. When multiple potential receiving targets exist within the field of view, the system often struggles to autonomously and quickly decide which target to prioritize for power supply, or to seamlessly switch between targets. This significantly limits the application flexibility of RIS technology in complex dynamic scenarios.

[0004] Therefore, there is an urgent need in this field for a new energy transfer solution that can seamlessly integrate the advanced judgment and intentions of human operators into automatic control loops. Summary of the Invention

[0005] To address the aforementioned technical challenges, this invention proposes an integrated air-to-ground RIS energy transfer system based on vision and gestures. By using depth vision to perceive the environment and targets in real time, and innovatively introducing spatial pointing interaction based on human joint point recognition, it enables operators to intuitively and quickly select specific targets in multi-target scenarios and drive the RIS to perform efficient and stable near-field energy focusing and transfer.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: The vision- and gesture-based integrated air-to-ground RIS energy transfer system includes: A visual perception detection unit is used to simultaneously acquire RGB images and depth images of the same physical scene, wherein an operator and at least one energy receiving module exist simultaneously in the physical scene; and to identify and obtain the position information of the at least one energy receiving module in three-dimensional space based on the RGB images and depth images. The intention analysis unit is connected to the visual perception detection unit and analyzes the three-dimensional spatial pointing ray representing the operator's pointing intention based on the RGB image and the depth image. The intention target matching unit is connected to the visual perception detection unit and the pointing intention parsing unit respectively, and is used to match the three-dimensional spatial pointing ray with the position information, thereby determining a target object as the energy supply target from the at least one energy receiving module; The control computing unit is connected to the intent target matching unit. Based on the location information of the target object, it calculates the phase control parameters required for the reconfigurable smart surface to focus energy. The RIS energy transmission unit, connected to the control and computing unit, is used to adjust the radiation characteristics of its electromagnetic beam according to the phase control parameters, and to focus energy onto the target object.

[0007] The effects described in the invention are merely those of the embodiments, and not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects: This invention proposes an integrated air-to-ground RIS energy transfer system based on vision and gestures, belonging to the field of wireless energy transfer technology. It includes: a visual perception detection unit, used to simultaneously acquire RGB and depth images of the same physical scene, where an operator and at least one energy receiving module coexist; and to identify and obtain the position information of at least one energy receiving module in three-dimensional space based on the RGB and depth images; a pointing intent parsing unit, connected to the visual perception detection unit, which parses a three-dimensional spatial pointing ray representing the operator's pointing intent based on the RGB and depth images; an intent target matching unit, connected to both the visual perception detection unit and the pointing intent parsing unit, used to match the three-dimensional spatial pointing ray with the position information, thereby determining a target object from at least one energy receiving module as the energy supply target; a control calculation unit, connected to the intent target matching unit, which calculates the phase control parameters required for energy focusing on the reconfigurable smart surface based on the target object's position information; and a RIS energy transfer unit, connected to the control calculation unit, used to adjust the radiation characteristics of its electromagnetic beam according to the phase control parameters to focus energy onto the target object. This invention uses depth vision to perceive the environment and targets in real time, and innovatively introduces spatial pointing interaction based on human joint point recognition. This enables operators to intuitively and quickly select specific targets and drive RIS to perform efficient and stable near-field energy focusing transmission in multi-target scenarios.

[0008] This invention addresses the issue of targets appearing smaller from the perspective of a high-position camera by improving the detection accuracy and robustness of YOLOv5s for small and occluded targets through early fusion of depth information and the C3TR attention module.

[0009] This invention employs a multi-criteria fusion scoring system based on spatial distance and regional division, combined with a multi-frame voting and time-dwelling locking mechanism. This effectively filters out unintentional tremors of the operator's arm and noise from single-frame visual detection, ensuring the stability and continuity of the energy transmission process.

[0010] This invention utilizes the flexible beamforming capability of the reconfigurable smart surface RIS, combined with a precise near-field focusing algorithm, to efficiently and accurately focus electromagnetic energy onto a locked moving target, thereby improving energy transmission efficiency. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the connection of the air-ground integrated RIS energy transmission system based on vision and gesture posture proposed in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the air-ground integrated RIS energy transfer system based on vision and gesture posture proposed in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the visual perception and target detection process proposed in Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of the multi-head self-attention mechanism proposed in Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of the target selection and locking logic in Embodiment 1 of the present invention; Figure 6 This is a schematic diagram illustrating the principle of reconfigurable smart surface RIS phase control and near-field energy focusing proposed in Embodiment 1 of the present invention; Figure 7 This is a schematic diagram of the energy receiving module proposed in Embodiment 1 of the present invention; Figure 8 This is a circuit diagram for rectifying absorbed electromagnetic energy as proposed in Embodiment 1 of the present invention. Detailed Implementation

[0012] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific embodiments and in conjunction with the accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the invention. To simplify the disclosure of the invention, components and arrangements of specific examples are described below. Furthermore, reference numerals and / or letters may be repeated in different examples. This repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed. It should be noted that the components illustrated in the drawings are not necessarily drawn to scale. Descriptions of well-known components, processing techniques, and processes are omitted in this invention to avoid unnecessarily limiting the invention.

[0013] Example 1 Embodiment 1 of this invention proposes an air-ground integrated RIS energy transfer system based on vision and gestures. It uses depth vision to perceive the environment and targets in real time and innovatively introduces spatial pointing interaction based on human joint point recognition. This enables the operator to intuitively and quickly select a specific target and drive the RIS to perform efficient and stable near-field energy focusing transfer in multi-target scenarios. Figure 1 This is a schematic diagram of the air-to-ground integrated RIS energy transfer system based on vision and gestures proposed in Embodiment 1 of the present invention; the system includes: A visual perception detection unit is used to simultaneously acquire RGB images and depth images of the same physical scene, wherein an operator and at least one energy receiving module exist simultaneously in the physical scene; and to identify and obtain the position information of the at least one energy receiving module in three-dimensional space based on the RGB images and depth images. The intention analysis unit is connected to the visual perception detection unit and analyzes the three-dimensional spatial pointing ray representing the operator's pointing intention based on the RGB image and the depth image. The intention target matching unit is connected to the visual perception detection unit and the pointing intention parsing unit respectively, and is used to match the three-dimensional spatial pointing ray with the position information, thereby determining a target object as the energy supply target from the at least one energy receiving module; The control computing unit is connected to the intent target matching unit. Based on the location information of the target object, it calculates the phase control parameters required for the reconfigurable smart surface to focus energy. The RIS energy transmission unit, connected to the control and computing unit, is used to adjust the radiation characteristics of its electromagnetic beam according to the phase control parameters, and to focus energy onto the target object.

[0014] Figure 2 This is a schematic diagram of the air-ground integrated RIS energy transfer system based on vision and gesture posture proposed in Embodiment 1 of the present invention; The visual perception detection unit utilizes an Astra Pro Plus depth camera to simultaneously acquire RGB and depth images of the same physical scene, providing an information foundation for subsequent 3D localization and pose estimation. The physical scene contains both an operator and at least one energy receiving module. To simultaneously capture the operator's gestures and the distant energy receiving module, the camera typically needs to be mounted at a higher position to obtain a wider field of view. This setup results in targets such as drones and mobile vehicles appearing as small targets in the image.

[0015] The scope of protection of this invention is not limited to the camera types listed in Embodiment 1, and those skilled in the art can make reasonable selections according to the circumstances.

[0016] The visual perception detection unit also identifies and obtains the position information of at least one energy receiving module in three-dimensional space based on RGB and depth image recognition, specifically: After aligning the RGB image and the depth image, the two are stitched together along the channel dimension to form four-channel image data; Four-channel image data is input into an improved YOLOv5s network model for forward inference, and the output is two-dimensional bounding box information of each energy receiving module in the RGB image; the bounding box information includes the center pixel coordinates; Based on the center pixel coordinates, the corresponding depth value is obtained from the synchronously aligned depth image; using the pre-calibrated camera intrinsic parameter matrix, combined with the center pixel coordinates and the depth value, the three-dimensional coordinates of the energy receiving module in the camera coordinate system are calculated through perspective projection inverse transformation.

[0017] Traditional visual detection algorithms (such as the standard YOLOv5) are prone to missed or false detections when processing small targets, especially in scenarios with complex backgrounds, occlusion, or poor lighting, affecting the accuracy of subsequent localization and energy focusing. To address the issues of smaller target size and susceptibility to occlusion from the perspective of high-position cameras, this invention makes structural improvements to the YOLOv5 network by incorporating deep fusion and attention mechanisms.

[0018] The YOLOv5s network model introduces an SPPF module at the end of the backbone network and a C3TR module with a global attention mechanism in the fusion path of the feature pyramid network.

[0019] Figure 3This is a schematic diagram of the visual perception and target detection process proposed in Embodiment 1 of the present invention. The flowchart fully describes the entire processing flow from RGB-D dual-stream input → early four-channel fusion → multi-scale feature extraction through the improved CSPDarknet53 backbone network → feature fusion through the enhanced PANet with C3TR introduced → finally outputting the target bounding box, confidence score, and category through the multi-scale detection head. The Astra Pro Plus depth camera simultaneously acquires RGB images (640×480×3) and depth maps (640×480×1) at a rate of 30FPS. After alignment, the acquired images are stitched together in the channel dimension to form a four-channel input tensor [640, 480, 4]. After proportional scaling and padding preprocessing, it is adjusted to the standard input size of [640, 640, 4]. The fused four-channel tensor is fed into the improved backbone network CSPDarknet53 network for feature extraction. By introducing Mosaic data augmentation, an early fusion mechanism for deep information, a fast spatial pyramid pooling structure using SPPF, and a C3TR (C3-Transformer) module with a global attention mechanism in the Neck part, the detection accuracy for small targets, occluded targets, and complex scenes is improved while maintaining real-time performance. The three fused augmented feature maps are fed into three parallel detection heads, and the network ultimately outputs detection result tensors at three scales. For example: the output based on the 80×80 feature map has a size of 80×80×(num_anchor×(5+num_class)), primarily responsible for detecting small targets. The output based on the 40×40 feature map has a size of 40×40×(num_anchor×(5+num_class)), primarily responsible for detecting medium-sized targets. The output based on the 20×20 feature map has a size of 20×20×(num_anchor×(5+num_class)), primarily responsible for detecting large targets.

[0020] During the YOLOv5s network inference process, the four-channel input tensor first passes through a 6×6 convolutional layer to downsample the spatial resolution to [320, 320, 32], and then is fed into the CSPDarknet5 backbone network for hierarchical feature extraction. After processing by the first-level convolution, batch normalization, SiLU activation module, and C3 module, the feature map size is downsampled to [160, 160, 64]. It then passes through the second-level convolution, batch normalization, SiLU activation, and C3 module to obtain a shallow feature map with a size of [80, 80, 128]. After another convolution downsampling and C3 feature extraction, a mid-level feature map with a size of [40, 40, 256] is obtained. Finally, through convolution-batch normalization-SiLU activation and C3 module, a deep feature map with a size of [20, 20, 512] is generated. At the end of the backbone network, the SPPF module is introduced to enhance the receptive field of the [20, 20, 512] features at multiple scales.

[0021] Subsequently, the extracted multi-scale features are fed into the improved PANet structure for bidirectional feature fusion. In the top-down path, the deep feature map [20,20,512] is convolved and upsampled, and then concatenated with the feature map [40,40,256] in the channel dimension to form the feature map [40,40,512], which is then refined through the C3TR module. This module replaces the Bottleneck unit in the C3 structure with a Transformer encoder layer. The feature map [40,40,512] is split into two channels. One half is fed into the Transformer encoder layer, reshaping the feature map [40,40,256] into a sequence of [1600,256]. The global contextual dependencies of the feature map sequence are calculated through a multi-head self-attention mechanism. The other half retains the convolutional path to preserve local structural information. Then, the features of the two paths are fused along the channel dimension to output the enhanced features of [40,40,256], which significantly enhances the network's ability to detect occluded targets and fast-moving targets.

[0022] Figure 4 This diagram illustrates the multi-head self-attention mechanism proposed in Embodiment 1 of the present invention. It clearly compares the internal structural differences between the standard C3 module and the improved C3TR module. The input and output structures of the two modules are identical; the main difference lies in the replacement of the core feature processing unit, thereby giving the network a stronger feature learning capability.

[0023] The C3 module is the fundamental feature processing and extraction unit in the YOLOv5 backbone network (CSPDarknet) and neck network (PANet). Its core idea is Cross-Stage Partial connections (CSP), which aims to enhance gradient flow and reduce computation through a split-merge structure. Specifically, it includes: Input: Receives the feature map from the previous layer.

[0024] Conv: A convolutional layer that performs initial transformations on the input features.

[0025] Bottleneck: This is the core processing part of the module. It typically consists of three convolutional layers (1x1 dimensionality reduction, 3x3 convolution, 1x1 dimensionality increase), and may include residual connections. Bottleneck is designed for lightweight local feature extraction and channel number adjustment. It has a limited receptive field and mainly captures local spatial information.

[0026] Concat: Concatenates the feature stream processed by Bottleneck with the feature stream that has not been processed by Bottleneck (or has been simply processed by another path) in the channel dimension, thus fusing features of different processing depths.

[0027] Conv: The last convolutional layer, which fuses and reduces the dimensionality of the concatenated features, and outputs the refined feature map.

[0028] Output: The processed feature map.

[0029] The improved C3TR module structure is the core improvement of the standard C3 module in this invention. It replaces the locally operating Bottleneck unit with a Transformer Block that has global modeling capabilities, aiming to address the problem of insufficient feature representation capabilities in complex scenes (such as occlusion, small objects, and cluttered backgrounds). Specifically, it includes: Input: Receives the feature map from the previous layer.

[0030] Conv: A convolutional layer that performs preliminary transformations on the input features (same function as C3).

[0031] Transformer Block: This is the core innovative part of the module. Internally, it typically includes: Multi-Head Self-Attention (MHSA): This mechanism allows each "location" (which can be viewed as a feature vector) in the feature map to interact with all other locations in the map, calculating attention weights. This enables the network to establish global contextual dependencies; for example, even if an object is partially occluded, the network can infer its complete existence by focusing on its visible parts and surrounding contextual information. Feed-Forward Network (FFN): Performs a non-linear transformation on the output of the self-attention layers. Layer Normalization and Residual Connections: Used to stabilize training and accelerate convergence.

[0032] Concat: Concatenates the feature flow after global modeling by the Transformer Block with the feature flow from another path. This combines global context information with raw or simply processed local information.

[0033] Conv: Fusion of spliced ​​features rich in global and local information.

[0034] Output: The output is the feature map enhanced by global context.

[0035] The fused features are further convolved and upsampled, and then concatenated with the [80,80,128] feature map to form the [80,80,256] feature map. After processing by the C3 module, the shallow enhanced features of [80,80,128] are output. In the bottom-up path, the [80,80,128] feature is downsampled by convolution and fused with an intermediate feature of [40,40,128] in the top-down path. After processing by the C3 module, the intermediate enhanced features of [40,40,256] are obtained. This feature is further downsampled and fused to finally form the deep enhanced features of [20,20,512].

[0036] These three enhanced feature maps are fed into the detection head, with [80,80,128] used for small target detection, [40,40,256] for medium target detection, and [20,20,512] for large target detection. The final detection output dimension is num_anchor×(5+num_class), and the five parameters represent the center coordinates x and y of the detection box, the width w, the height h, and the confidence level, respectively.

[0037] The scope of protection of this invention is not limited to the values ​​listed in Example 1, and those skilled in the art can make reasonable selections based on the actual situation.

[0038] The visual perception detection unit is also used to adaptively divide the camera's field of view according to the number of energy receiving modules detected; specifically: when the number is 2, the region is divided by the perpendicular bisector of the line connecting the centroids of the two targets; in the implementation where the number of energy receiving modules N=2, the perpendicular bisector of the line connecting the centers of the two targets is used as a natural boundary. Figure 5 This is a schematic diagram of the target selection and locking logic in Embodiment 1 of the present invention. This partitioning method creates a logical 'exclusive region' for each target in physical space. When the number is greater than two, a Voronoi diagram is performed using the center of each target as the seed point. Specifically, the center position of each target is used as the seed point of the Voronoi diagram, and a Voronoi diagram covering the entire field of view is generated through calculation. This diagram divides the field of view into multiple non-overlapping convex polygonal regions, each containing one and only one target seed point. Under this design, the user's pointing action only needs to fall within any Voronoi polygonal region, and the system can automatically determine the unique target corresponding to that region as the user's selected target based on the nearest neighbor principle, thereby achieving the intuitive interactive effect of "pointing to the region is pointing to the coordinates".

[0039] The process executed by the pointing intent parsing unit includes: identifying the two-dimensional pixel coordinates of at least three joints of the operator's arm from the RGB image; converting the two-dimensional pixel coordinates into three-dimensional spatial coordinates by combining them with the depth image; and calculating and generating a three-dimensional pointing ray based on the three-dimensional spatial coordinates. The at least three joints include the operator's elbow, wrist, and fingertips.

[0040] A keypoint detection algorithm is used to capture the 2D pixel coordinates of three key points on the operator's arm: "elbow," "wrist," and "fingertip." Combined with depth camera data, reliable depth values ​​of the above three key points are obtained. Then, through camera intrinsic parameter backprojection, a 3D spatial pointing ray determined by the "elbow-wrist-finger" vector is constructed in three-dimensional space and smoothed in the temporal domain.

[0041] Assuming the three points are P_elbow, P_wrist, and P_finger, the system calculates vectors V1 = P_wrist - P_elbow and V2 = P_finger - P_wrist. These two vectors define a three-dimensional ray from the elbow, through the wrist, and towards the fingertip. Kalman filtering or moving average is applied to this ray to smooth out temporal jitter.

[0042] The intention target matching unit is connected to the visual perception detection unit and the pointing intention parsing unit respectively, and is used to match the three-dimensional spatial pointing ray with the position information, thereby determining a target object as the energy supply target from at least one energy receiving module; The system geometrically matches the 3D spatial pointing ray with the 3D spatial positions of each target. By calculating the shortest spatial distance from the ray to each target point and combining this with adaptively segmented image regions, the candidate target with the highest score is selected. A time-based multi-frame voting and dwell time threshold mechanism is introduced to ensure that the system only confirms "lock" when the decision to point to a target is continuous and stable, effectively avoiding misoperation caused by slight arm movements.

[0043] In a system that achieves near-field focusing using a reflective metasurface, a horn antenna is used as the feed source. The electromagnetic waves excited by the feed source are phase- and amplitude-modulated by the metasurface elements, resulting in constructive interference in the near-field region, and ultimately converging at the preset focal point, thereby improving the energy density and transmission efficiency of the target region. Figure 6 This is a schematic diagram illustrating the principle of reconfigurable smart surface RIS phase control and near-field energy focusing proposed in Embodiment 1 of the present invention; In the picture, and These are the length and width of the reconfigurable smart surface, respectively. and All represent energy focal points. In metasurface units The electric field reflected at the center position is expressed as: ; in, To describe the propagation of electromagnetic waves from the feed phase center to the metasurface unit The spatial propagation phase delay experienced during time, Indicates the imaginary part; For metasurface units The electric field reflected at the center position; For the first The magnitude of the reflection coefficient of each metasurface unit; For the first The reflection coefficient and phase factor of each metasurface unit; To be emitted from the feed source, incident on the first The electric field reflected at the center position; The wavenumber at the center frequency; The coordinates of the feed phase center; This is the center position of the metasurface unit.

[0044] In this application, the control computing unit calculates the phase control parameters required for energy focusing of the reconfigurable smart surface based on the position information of the target object, specifically: The first The coordinates of a target object in the RIS coordinate system are defined as follows: ,in, , The total number of target objects; Obtain the phase center coordinates of the feed. and the first in the RIS power transfer unit The center coordinates of each metasurface unit ; Calculate the first according to the following formula. The required reflection phase compensation value for each metasurface unit ; ; in, The free-space wavenumber corresponding to the system's operating frequency; ; The operating wavelength; Indicates the argument of the complex number parameter; Represents the imaginary unit; It is a configurable constant phase offset; By iterating through all metasurface elements of the RIS energy transfer unit and repeatedly performing phase calculations, the phase compensation values ​​for all metasurface elements are obtained. This forms the phase control parameters for controlling the RIS energy transfer unit.

[0045] Figure 7 This is a schematic diagram of the energy receiving module proposed in Embodiment 1 of the present invention; In this application, the RIS power transfer unit includes an array of multiple independently phase-tunable electromagnetic metasurface units, a high-power radio frequency source that provides radio frequency energy to the array, and a controller for adjusting the phase of each unit according to phase control parameters.

[0046] In this application, the energy receiving module includes: a metamaterial absorber with a copper-F4B-copper three-layer stacked structure for receiving electromagnetic energy; and a rectifier circuit electrically connected to the metamaterial absorber for converting the received radio frequency energy into direct current.

[0047] Figure 8 The circuit diagram for rectifying absorbed electromagnetic energy proposed in Embodiment 1 of the present invention is shown. To ensure the stability of the resonant frequency, the rectifier diode is placed on the bottom layer and four metallized vias are used to realize its electrical interconnection with the top layer absorption structure, thereby achieving efficient rectification and collection of captured electromagnetic energy, while avoiding the damage to the structural integrity caused by surface mount soldering.

[0048] The energy receiving module is installed on a drone or mobile vehicle and uses a highly efficient "metal-dielectric-metal" sandwich structure and rectifier circuit to convert the received high-frequency electromagnetic energy into DC power to charge the device.

[0049] Embodiment 1 of this invention proposes an air-ground integrated RIS energy transfer system based on vision and gestures. By visually capturing gestures and locking onto moving targets, the RIS is driven to focus energy, thus achieving precise energy transfer.

[0050] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that the elements inherent in a process, method, article, or apparatus that includes a list of elements are included. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Additionally, portions of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0051] While specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art can make other modifications or variations based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A vision- and gesture-based integrated air-to-ground RIS energy transfer system, characterized in that, include: A visual perception detection unit is used to simultaneously acquire RGB images and depth images of the same physical scene, wherein an operator and at least one energy receiving module are present in the physical scene at the same time. The system identifies and obtains the position information of at least one energy receiving module in three-dimensional space based on the RGB image and depth image. Specifically, the visual perception detection unit identifies and obtains the position information of at least one energy receiving module in three-dimensional space based on the RGB image and depth image, which involves: aligning the RGB image and depth image, then stitching them together in the channel dimension to form four-channel image data; inputting the four-channel image data into an improved YOLOv5s network model for forward inference, and outputting the two-dimensional bounding box information of each energy receiving module in the RGB image, the bounding box information including the center pixel coordinates; obtaining the corresponding depth value from the synchronously aligned depth image based on the center pixel coordinates; and using a pre-calibrated camera intrinsic parameter matrix, combined with the center pixel coordinates and the depth value, calculating the three-dimensional coordinates of the energy receiving module in the camera coordinate system through perspective projection inverse transformation. The visual perception detection unit is also used to adaptively divide the camera field of view according to the number of detected energy receiving modules; specifically: when the number is 2, the region is divided by the perpendicular bisector of the line connecting the centroids of the two targets; when the number is greater than 2, the Voronoi diagram is performed using the center of each target as the seed point. The intention analysis unit is connected to the visual perception detection unit and analyzes the three-dimensional spatial pointing ray representing the operator's pointing intention based on the RGB image and the depth image. The intention target matching unit is connected to the visual perception detection unit and the pointing intention parsing unit respectively, and is used to match the three-dimensional spatial pointing ray with the position information, thereby determining a target object as the energy supply target from the at least one energy receiving module; The intent target matching unit matches the three-dimensional spatial pointing rays with the position information, specifically: Calculate the spatial distance from the three-dimensional pointing ray to the location of each energy receiving module; combine the results of the adaptive region division to give a comprehensive score to the spatial distance; A time-series-based multi-frame voting mechanism and a dwell time threshold are used to verify the stability of the candidate target with the highest score and then determine it as the target object. The control computing unit, connected to the intent target matching unit, calculates the phase control parameters required for energy focusing on the reconfigurable smart surface based on the target object's location information. The RIS energy transmission unit, connected to the control and computing unit, is used to adjust the radiation characteristics of its electromagnetic beam according to the phase control parameters, and to focus energy onto the target object.

2. The system according to claim 1, characterized in that, The YOLOv5s network model introduces an SPPF module at the end of the backbone network and a C3TR module with a global attention mechanism in the fusion path of the feature pyramid network.

3. The system according to claim 1, characterized in that, The pointing intent parsing unit parses a three-dimensional spatial pointing ray representing the operator's pointing intent based on the RGB image and depth image, specifically: Identify the two-dimensional pixel coordinates of at least three joints of the operator's arm from an RGB image; By combining depth images, two-dimensional pixel coordinates are converted into three-dimensional spatial coordinates; Based on the three-dimensional spatial coordinates, a three-dimensional spatial pointing ray is calculated and generated.

4. The system according to claim 3, characterized in that, The at least three joint points include the operator's elbow, wrist, and fingertips.

5. The system according to claim 1, characterized in that, The control computing unit calculates the phase control parameters required for energy focusing on the reconfigurable smart surface based on the target object's location information. Specifically: The first The coordinates of a target object in the RIS coordinate system are defined as follows: ,in, , The total number of target objects; Obtain the phase center coordinates of the feed. and the first in the RIS power transfer unit The center coordinates of each metasurface unit ; Calculate the first according to the following formula. The required reflection phase compensation value for each metasurface unit ; ； in, The free-space wavenumber corresponding to the system's operating frequency; ; The operating wavelength; Indicates the argument of the complex number parameter; Represents the imaginary unit; It is a configurable constant phase offset; By iterating through all metasurface elements of the RIS energy transfer unit and repeatedly performing phase calculations, the phase compensation values ​​for all metasurface elements are obtained. This forms the phase control parameters for controlling the RIS energy transfer unit.

6. The system according to claim 1, characterized in that, The RIS power transfer unit includes an array of multiple independently phase-tunable electromagnetic metasurface units, a high-power radio frequency source that provides radio frequency energy to the array, and a controller for adjusting the phase of each unit according to the phase control parameters.

7. The system according to claim 1, characterized in that, The energy receiving module includes: A metamaterial absorber with a three-layer stacked structure of metal-dielectric-metal is used to receive electromagnetic energy; A rectifier circuit, electrically connected to the metamaterial absorber, is used to convert the received radio frequency energy into direct current.

Images