Multi-mode communication and perception channel twin AI data set generation method

Abstract

The invention discloses a multi-mode communication and perception channel twinborn AI data set generation method. The method comprises the following steps: establishing a high-precision three-dimensional indoor scene model and an outdoor scene model; configuring parameters of a signal receiving and transmitting end and the RIS, determining dielectric constants and magnetic conductivities of different object surfaces in a scene according to a signal center frequency, and determining reflection, refraction, scattering and diffraction effects of electromagnetic waves in different media based on a geometric optical theory and a geometric diffraction theory; calculating channel parameters of each ray, generating a channel matrix and a channel intensity thermodynamic diagram based on the channel parameters, and generating a complete data set in combination with scene information; and aligning the coordinates of the scene and the ray, loading and rendering ray data and a channel intensity thermodynamic diagram based on a moving path selected by a user, and obtaining a visual file of the multi-modal channel twin AI data set. According to the invention, refined materials and electromagnetic properties are given to objects in a scene, and efficient simulation and second speed visualization of electromagnetic wave propagation are realized.

Description

Technical Field

[0001] The present invention relates to the field of wireless AI, and in particular to a method for generating a twin AI data set for multimodal communication and perception channels. Background Art

[0002] In recent years, wireless AI—the deep integration of artificial intelligence and communications technology—has become a research and application hotspot in academia and industry worldwide. With the advent of the 6G era, computing technology has entered a new era of large models. Data is a critical foundation for promoting the deep integration of 6G and large models. High-quality and abundant data is the foundation for model training, validation, and optimization. Lack of this support will severely impact algorithm accuracy and system generalization capabilities.

[0003] However, collecting real-world communication data requires deploying a large number of sensors, building complex test environments, and performing repeated measurements and calibrations. This process is not only time-consuming and labor-intensive, but also comes with high costs. Therefore, academia and industry are actively working on the development and research of intelligent communication datasets. By simulating different electromagnetic field characteristics and channel behaviors in a virtual digital twin environment, they generate spatial electromagnetic data that is as close to reality as possible. Importantly, this approach enables researchers to flexibly adjust parameters such as base stations, user locations, and antenna configurations to accurately reproduce the characteristics of various channels, such as THz and millimeter waves. Ultimately, this simulated data will be widely used for model training and testing, supporting a variety of wireless AI applications.

[0004] Currently, mainstream intelligent communication datasets are mainly based on deterministic channel models to simulate different electromagnetic field characteristics and channel behaviors. Deterministic models are built based on electromagnetic wave propagation theory and actual environmental measurements. Given a specific scenario (such as indoor or outdoor environment), the location of terminal equipment (such as base stations, user equipment, RIS, etc.) and its antenna configuration (direction, polarization, etc.), and after assigning different materials to objects (reflected by dielectric constants and magnetic permeabilities), the deterministic model uses ray tracing technology based on computer graphics to simulate the reflection, refraction, and scattering of each ray path, thereby predicting characteristic information such as channel angle, delay, and polarization parameters, and achieving accurate modeling of channel characteristics. Furthermore, by visualizing high-precision electromagnetic spectrum data (such as ray diagrams, channel strength heat maps, etc.), it is used to analyze, train, and verify the performance of AI models in complex communication environments.

[0005] However, existing wireless AI datasets primarily rely on CPU-based ray tracing technology for channel simulation. Although numerous companies, enterprises, and research institutions have developed various ray tracing acceleration methods, limited by CPU computing performance, existing methods only support Level 1 (LoD1) scene reconstruction, resulting in insufficient model accuracy, low computational efficiency, and crude visualization. Furthermore, limited research has incorporated RIS scenarios into existing datasets, and the complexity of calibrating material and electromagnetic properties based on LoD1 scenarios makes it difficult to meet the requirements for high-precision channel modeling and applications.

[0006] Therefore, there is an urgent need to build a high-quality wireless AI dataset that integrates high-precision scene reconstruction, GPU-accelerated ray tracing technology, and multi-dimensional visualization methods. It can be compatible with multiple scene types, including static and dynamic scenes, indoor and outdoor scenes, large-scale complex scenes, etc., and quickly and efficiently generate electromagnetic data that is closer to the real world, supporting the development and verification of wireless communication systems and intelligent perception systems. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of the existing technology and propose a method for generating twin AI data sets of multimodal communication and perception channels.

[0008] The purpose of the present invention is to achieve the following technical solution: a method for generating a multimodal communication and perception channel twin AI data set, comprising the following steps:

[0009] S1. Establish high-precision three-dimensional indoor scene models and outdoor scene models;

[0010] S2. Configure the parameters of the signal transceiver and RIS, determine the dielectric constant and magnetic permeability of the surfaces of different objects in the scene based on the signal center frequency, and the reflection, refraction, scattering and diffraction effects of electromagnetic waves in different media based on the theory of geometric optics and geometric diffraction;

[0011] S3. Calculate the channel parameters of each ray according to the free space path loss formula, generate a channel matrix and channel intensity heat map based on the channel parameters, and generate a complete data set based on the scene information;

[0012] S4. Align the coordinates of the scene and the ray, load and render the ray data and channel intensity heat map based on the movement path selected by the user, and obtain a visualization file of the multimodal channel twin AI dataset.

[0013] Furthermore, in S1, the process of constructing the outdoor scene model includes:

[0014] Select any area in the Open Street Map and call the height, texture and other object data recorded by the satellite system to generate a rough 3D scene. Then segment the objects in the rough 3D scene, smooth the edges of the objects, and reconstruct the fuzzy textures to ensure that the building details are close to the real scene. Then add green belts, ground and roads, trees, street lights, and vehicles, and finally save the generated scene in XML file format.

[0015] Furthermore, in S1, the process of constructing the indoor scene model includes:

[0016] A global 3D scan of the indoor environment is performed using LiDAR to obtain LoD1-level point cloud data containing spatial coordinates and distance information. After denoising and filtering, the 3D structure of the environment is constructed, resulting in a LoD2-level global 3D map model of the indoor environment. Texture mapping and lighting optimization rendering are then performed to generate a LoD3-level model. Finally, the generated scene is saved in FBX file format. Furthermore, the configuration of the signal transceiver and RIS parameters in S2 includes:

[0017] Configure key parameters of the transceiver and RIS, including location coordinates, row and column array size, movement speed, antenna pattern, and polarization characteristics.

[0018] Furthermore, in step S2, the RIS supports both single-beam and multi-beam generation methods, including: precisely controlling the phase distribution of the RIS's reflector units based on a path difference focusing scheme and a gradient phase design method to achieve single-beam focusing in the target direction; and dynamically adjusting the RIS's reflection coefficient array using a stochastic gradient descent optimization algorithm to generate a multi-beam codebook that meets multi-user or multi-target requirements, providing efficient support for multi-point communications in complex scenarios.

[0019] Furthermore, the S2 also adopts a ray tracing method based on GPU acceleration, including: initializing some rays at the transmitting end according to computing resources, selecting key propagation directions to reduce redundant calculations; simulating the reflection, refraction, scattering and diffraction effects of rays in the scene, and extracting physical information such as the phase, power, propagation angle and delay of each ray; calling Optix ray tracing to check the simulation process for GPU parallel acceleration; cleaning and optimizing the generated path data, deleting abnormal paths and correcting deviations.

[0020] Furthermore, the channel data set generated in S3 includes the construction process of the MIMO channel matrix, including: synthesizing the channel impulse response to obtain the delay characteristics, generating the MIMO channel matrix to describe the response of the multi-antenna system, calculating the received signal strength and path loss, accurately drawing the radio map to display the key communication indicators at different locations, discretizing the path information into a CSV table format through equal-interval sampling, and saving the generated channel thermal intensity map in high-resolution JPG format for visual analysis.

[0021] Furthermore, the visualization processing of the large-scale multi-scene data set generated in S4 includes: importing scene data through the Babylon.js web platform to complete the alignment of the scene coordinate origin and the unit coordinate vector; importing ray coordinate data in CSV format, and generating a complete ray path based on the starting point and end point of each ray; loading the channel intensity heat map at the specified height of the scene, and scaling the heat map to a matching position; rendering the ray data and channel intensity heat map in real time in the three-dimensional scene through GPU acceleration; using key frame technology to set the time interval of each frame, loading and rendering the data frame by frame according to the above operations, and finally generating a coherent dynamic visualization animation, which is exported as a video file to intuitively display the channel propagation characteristics and dynamic changes in large-scale scenes.

[0022] On the other hand, the present invention also provides a system for generating a multimodal communication and perception channel twin AI dataset, including:

[0023] The scene twin module is used to build high-precision three-dimensional indoor and outdoor scene models;

[0024] The electromagnetic signal twin module is used to configure the parameters of the signal transceiver and RIS, determine the dielectric constant and magnetic permeability of different object surfaces in the scene based on the signal center frequency, and analyze the reflection, refraction, scattering and diffraction effects of electromagnetic waves in different media based on the theory of geometric optics and geometric diffraction.

[0025] The multimodal data generation module is used to calculate the channel parameters of each ray according to the free space path loss formula, generate the channel matrix and channel intensity heat map based on the channel parameters, and generate a complete data set based on the scene information;

[0026] The visualization module is used to align the coordinates of the scene and rays, load and render the ray data and channel intensity heat map based on the movement path selected by the user, and obtain a visualization file of the multimodal channel twin AI dataset.

[0027] On the other hand, the specification of the present invention also provides a device for generating a multimodal communication and perception channel twin AI dataset, comprising a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method for generating a multimodal communication and perception channel twin AI dataset.

[0028] Beneficial effects of the present invention: The present invention systematically and completely constructs a full-process high-speed wireless communication simulation. Compared with the existing wireless AI data set, the example of this application has completely realized the full process from indoor and outdoor LoD3 level high-precision scene modeling to electromagnetic property assignment for the first time. By accurately associating electromagnetic properties with object materials, the authenticity and usability of the simulation environment are greatly improved; combined with the Nvidia Optix library, GPU accelerated ray tracing and rendering optimization technology are called to achieve efficient simulation and second-speed visualization of electromagnetic wave propagation, support multi-angle dynamic video display, and intuitively present channel characteristics and propagation laws; through the integrated software front-end interface, users can configure scene parameters online, call multi-module functions in real time, generate high-quality channel data, and conveniently download result data and visualization files, significantly improving operational efficiency and application convenience. The channel dataset constructed by the solution of this application comprehensively covers all mechanisms in the actual propagation process of radio waves, supports large-scale MIMO far-field static and dynamic object channel modeling, supports sub-6GHz and millimeter wave bands, and has flexible adaptation capabilities such as multi-region, multi-antenna, multi-band, and multi-link. This solution is widely applicable to physical layer tasks such as channel estimation and beamforming, as well as perception layer tasks such as pose estimation and three-dimensional reconstruction. It can effectively meet the application needs of multiple fields in complex scenarios and provide strong technical support for the research and development of wireless communication and perception systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flowchart of a method for generating a twin AI dataset for multimodal communication and perception channels provided in an embodiment of the present application;

[0030] Figure 2 This is a flowchart of a multimodal channel twin AI dataset for information communication and perception according to an embodiment of the present application;

[0031] Figure 3 This is a schematic diagram of the operation of selecting an area in OSM provided in an embodiment of the present application;

[0032] Figure 4 A schematic diagram of Google 3D capturing a rough outdoor scene provided in an embodiment of the present application;

[0033] Figure 5 A schematic diagram of outdoor scene reconstruction provided in an embodiment of the present application;

[0034] Figure 6 Schematic diagram of separation and reorganization of outdoor scene materials provided in an embodiment of the present application;

[0035] Figure 7 A schematic diagram of a visual video of an indoor scene provided in an embodiment of the present application;

[0036] Figure 8 A schematic diagram of a visual video of an outdoor scene provided in an embodiment of the present application;

[0037] Figure 9 A schematic diagram of a multimodal communication and perception channel twin AI data set generation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the technical problems, technical solutions and advantages to be solved by this application clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0039] It should be understood that references throughout this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic associated with the embodiment is included in at least one embodiment of the present application. Therefore, the appearances of "in one embodiment" or "in an embodiment" throughout this specification do not necessarily refer to the same embodiment. Furthermore, these particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0040] In the embodiments of the present application, the term "and / or" describes the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0041] In the embodiments of the present application, the term "plurality" refers to two or more than two, and other quantifiers are similar.

[0042] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0043] Reference Figure 1 and Figure 2 This embodiment relates to a method for generating a multimodal communication and perception channel twin AI dataset, which specifically includes the following steps:

[0044] S1: The LoD1 scene is gradually optimized to generate the LoD3 scene, and different objects in the LoD3 scene are given refined material and electromagnetic properties to fully restore the geometric and physical characteristics of the real scene;

[0045] Among them, scene generation is divided into outdoor and indoor scene generation.

[0046] Outdoor scene generation includes:

[0047] S101. Install Blender and Sketchup, install the Blosm plugin in Blender, apply for a Google 3DTiles license, and then insert the successfully applied key into the corresponding location in the Blender tool library.

[0048] S102, using Google 3D Tiles to call the GIS data of the area recorded in the satellite system, such as height and texture, and generate a rough texture LoD1 scene in Blender;

[0049] In this embodiment, all objects in the scene are connected as a whole to obtain the main page shortcut key N. Open the side toolbar, select Blosm, and click select to enter the area selection page. Select the geographic layer in the upper right corner and change it to "Arcgis Satellite". Click the tool "Show selection rectangle" on the left, select a certain range of the city scene, and click "Copy". Return to the Blender interface, click "paste" in the Blosm interface, paste the area selection information, select "Google 3D Tiles" data source, select "buildings with details" in "Level of details", click "import", and wait for the map to be imported. Figure 3 and Figure 4 After the map is imported, click "File", "Export", "gLTF 2.0" in the top bar and select the appropriate export path. After the map is fully imported, we will obtain the LoD1 level scene for subsequent refinement.

[0050] S103. Next, optimize the LoD1 scene to LoD2 level, import the GLTF file into Sketchup for strict architectural reconstruction, smooth the edges of objects, and reconstruct the blurred textures to ensure that the details of building elevations, doors and windows are as close to the real scene as possible.

[0051] In this embodiment, considering the difficulty of modeling and simulation calculation, the processing of non-core parts is simplified, such as Figure 5As shown. In this way, high-precision scene reconstruction can be achieved without offline measurement, thereby greatly reducing the workload of manual modeling. According to the rough 3D model, use drawing tools such as push / pull, and according to experimental requirements, reconstruct some buildings, doors and windows, roads, and bushes in the scene one-to-one, and use material tools to give different materials to the model surface, laying the foundation for subsequent simulation calculations. After the processing is completed, export the model again to the OBJ file format, click "File", "Export", "3D Model" in sequence, and select the path to be saved. In order to ensure smooth operation in the future, each building unit needs to be exported separately, such as road surface, single building, sand, bushes, etc.;

[0052] S104. Import the model created in Sketchup into Blender, separate and merge the materials, and finally obtain a LoD3 level scene. The scene is then converted into XML format for subsequent channel modeling operations.

[0053] In this embodiment, each building unit is separated according to the material type. For each model unit exported from Sketchup, press the "TAB" key to enter the editing mode, press "P" to use the classification tool, select "Separate by material", and separate the model by material. Return to the object mode "Shift" + "A", create a new empty object, select the pure axis, and re-incorporate the separated units into the newly created empty object. This embodiment includes two situations: indoor and outdoor, and the two situations are independent of each other. Figure 6 As shown. Separate the materials of all building units in turn and then merge them. You can further merge them according to your needs, such as merging all buildings into one building complex.

[0054] Indoor scene generation includes:

[0055] S105: The laser radar transmits and receives a high-frequency laser beam, records the time difference between the emission and reception of the laser beam, calculates the distance traveled by the laser beam, and forms a rough point cloud data of the LoD1 level;

[0056] Among them, each data point contains the spatial coordinates and distance information of the point;

[0057] S106. The lidar also captures the intensity of the reflected signal to characterize the reflective properties of the object, such as the color or material of the surface;

[0058] S107, denoising and filtering the collected point cloud data to improve LoD1 level accuracy;

[0059] S108, converting the point cloud data into points in a three-dimensional coordinate system, gradually constructing the three-dimensional structure of the environment, reflecting the shape and spatial position of the object. This step generates a Level 2 model with a more refined geometric structure;

[0060] S109, using a depth camera to capture the texture of the object for mapping, and generating a LoD3 level refined model;

[0061] S110. The generated scene will be exported as an FBX format file.

[0062] S2. Set scene parameters in Python, optimize ray direction and number of rays, and achieve efficient simulation of electromagnetic wave spatial effects by combining GPU-accelerated ray tracing technology, supporting accurate channel modeling in complex environments;

[0063] S201. Accurately configure key parameters of the transceiver and RIS, including location coordinates, array size, mobile speed, antenna pattern, and polarization characteristics;

[0064] In this embodiment, the RIS supports both single-beam and multi-beam generation methods, including conventional path difference-based focusing schemes and gradient phase design methods to precisely control the phase distribution of the RIS's reflector elements to achieve single-beam focusing in the target direction. Furthermore, we apply the SGD optimization algorithm to dynamically adjust the RIS's reflection coefficient array to generate a multi-beam codebook that meets the needs of multiple users or multiple targets.

[0065] S202, defining the frequency and bandwidth parameters of the signal, and assigning the dielectric constant and magnetic permeability of each object in the scene according to the center frequency of the signal, and calculating the reflection, diffraction, and diffraction coefficients of the relevant scatterers;

[0066] S203, initializing some rays on the transmitting end sphere according to the Fibonacci sequence, and preferentially retaining rays facing the direction of densely populated buildings where users are located, to improve ray tracing efficiency and accuracy;

[0067] S204, setting a maximum limit on the number of times a single path contacts a scatterer. When the number of ray contacts during the simulation process exceeds the threshold, the simulation of the ray is stopped.

[0068] S205, setting a power determination threshold when the ray contacts the scatterer, and stopping the simulation of the ray when the power of the ray contacts the scatterer is lower than the threshold;

[0069] S206, using the Sionna ray tracing open source tool and calling Mitsuba rendering software, simulate the spatial effects of electromagnetic waves accelerated by GPU;

[0070] In this embodiment, the spatial effects of electromagnetic waves include penetration through obstacles, reflection from obstacle surfaces, and diffraction effects at the edges of obstacles. Transmission and reflection are modeled using geometric optics theory, while diffraction is modeled using UTD. An exhaustive and greedy algorithm is used to obtain the phase and position information of each reflection point, and arithmetic sampling is used to store the physical information of each ray, including phase, power, propagation angle, and delay.

[0071] It should be noted that the precise channel modeling supports sub-6GHz and millimeter wave frequency bands, comprehensively covering the channel modeling requirements in different application scenarios, and providing high-precision data support for multimodal channel simulation.

[0072] S3. Generate multimodal data that meets the needs of various downstream applications, including scene images, channel matrices, electromagnetic wave arrival and reception angles, signal gain, delay, etc., providing data support for communication and perception tasks in complex environments;

[0073] S301. Based on the path information, synthesize the channel impulse response, which is formulated as follows:

[0074]

[0075] Where N is the total number of paths, a i is the amplitude gain of the ith path, φ i is the phase of the ith path, τ i is the delay of the ith path, δ(t-τ i ) is the delay distribution function of the i-th path. Therefore, we can obtain the average arrival delay:

[0076]

[0077] And RMS delay spread:

[0078]

[0079] in,

[0080]

[0081] S302: Generate a MIMO channel matrix H(f) based on the channel impulse response. The formula is:

[0082]

[0083] Where H(f) is the MIMO channel matrix at frequency f, a r (θ r,i ) is the direction vector of the receiving antenna, representing AoA, is the direction vector of the transmitting antenna, representing AoD, a iis the amplitude gain of the ith path, τ i is the delay of the ith path, θ r,i ,θ t,i are the arrival and departure angles of the ith path, is the conjugate transpose of the transmit antenna direction vector.

[0084] Calculate received signal strength:

[0085]

[0086] Among them G t and G r is the gain of the transmitter and receiver, h t and h r is the height of each antenna, and d is the horizontal distance between them. The path loss between the two antennas can be calculated as:

[0087] PL(dB)=40log d-(10log G t +10log G r +20log h t +20log h r )

[0088] S303. Accurately calculate the radio map for all transmitters in the scenario. For each transmitter, key metrics such as path gain, received signal strength (RSS), or signal-to-interference-plus-noise ratio (SINR) are normalized and mapped to each coordinate point on the plane. In other words, for a specific transmitter, any point on the plane is mapped to the channel gain, RSS, or SINR value observed by the receiver in a specific direction at that point, and saved as a strength heatmap.

[0089] S304: Sample the path between every two reflection points at equal intervals and save the sampling point information as a CSV format file, that is, discretize and store the path between the reflection points.

[0090] S305 , saving each channel heat intensity map in the highest resolution JPG format, and providing a scaling factor for the image, so as to scale the image to the correct position during visualization.

[0091] S4. Based on the selected user movement path, tens of thousands of ray propagation trajectories and hundreds of channel intensity heat maps are quickly visualized. Multi-angle dynamic video display is also supported to intuitively present the dynamic propagation characteristics of the electromagnetic channel. Doppler shift is generated during user movement and must also be taken into account. Doppler shift can be expressed as follows:

[0092]

[0093] Where λ is the wavelength, vi ,k i Represent the velocity vector and the outgoing direction vector at the i reflection points respectively.

[0094] S401. In Babylon.js, adjust the scene coordinate origin and unit coordinate vector to align precisely with the XML file;

[0095] This step is based on Babylon.js and uses WebGL technology to send graphics directly to the GPU for processing through hardware acceleration, achieving efficient large-scale ray trajectories and channel data rendering;

[0096] S402, ray trajectories are read from a CSV table, and a complete ray path is generated based on the starting point and end point of each ray, and then rendered into a 3D scene in real time;

[0097] S403: The channel data is visualized by loading a channel intensity heat map, and the heat map is scaled and adjusted according to the scene requirements, and accurately placed at the matching geographic coordinate position to ensure consistency between the data and the scene;

[0098] S404: Select the user's movement path, set the time sampling point interval of the movement, and read the ray trajectory and channel intensity heat map corresponding to the time point;

[0099] S405, achieving frame-by-frame rendering of ray paths and channel intensity changes through key frame technology, and ultimately generating a coherent dynamic visualization effect;

[0100] S406. The user zooms in and out on the software interface to change the video effect at different angles and exports the complete video. The effect is as follows: Figure 7 and Figure 8 shown.

[0101] The embodiment of the present application provides a multimodal communication and perception channel twin AI data set generation system, the software interface is as follows Figure 9 Shown, including:

[0102] The scene twin module includes an indoor submodule and an outdoor submodule, which are used to build high-precision three-dimensional indoor scene models and outdoor scene models respectively;

[0103] The electromagnetic signal twin module is used to configure the parameters of the signal transceiver and RIS, determine the dielectric constant and magnetic permeability of different object surfaces in the scene based on the signal center frequency, and analyze the reflection, refraction, scattering and diffraction effects of electromagnetic waves in different media based on the theory of geometric optics and geometric diffraction.

[0104] The multimodal data generation module is used to calculate the channel parameters of each ray according to the free space path loss formula, generate the channel matrix and channel intensity heat map based on the channel parameters, and generate a complete data set based on the scene information;

[0105] The visualization module aligns scene and ray coordinates, loads and renders ray data and channel intensity heatmaps based on the user-selected movement path, and generates visualizations of the multimodal channel twin AI dataset. Developed based on Babylon.js, the visualization module leverages WebGL technology to send graphics directly to the GPU for processing via hardware acceleration, enabling efficient large-scale rendering of ray trajectories and channel data.

[0106] The system realizes the control of Figure 1 Full-process automated management of all modules shown; a fully functional and user-friendly web display platform is designed to visualize the workflow and intermediate results of each module; users are supported to configure parameters online and call core functional modules according to actual needs, such as selecting scene areas for LoD3 reconstruction, configuring ray tracing parameters, loading channel data and generating heat maps, and outputting dynamic videos; and the function of exporting result data in multiple formats is provided, including scene data, ray tracing paths, channel intensity heat maps, and the final generated video files.

[0107] In summary, compared with the existing wireless AI dataset, the embodiment of the present application generates a high-precision scene model of the LoD3 level for indoor and outdoor, giving refined material and electromagnetic properties to different objects in the scene, greatly improving the realism and usability of the simulation environment; combined with GPU-accelerated ray tracing and rendering optimization technology, it realizes efficient simulation and instant visualization of electromagnetic wave propagation, supports multi-angle dynamic video display, and intuitively presents channel characteristics and propagation laws; through the integrated software front-end interface, users can configure scene parameters online, call multi-module functions in real time, generate high-quality channel data, and conveniently download result data and visualization files, significantly improving operational efficiency and application convenience. The channel dataset constructed by the scheme of the present application comprehensively covers all mechanisms in the actual propagation process of radio waves, supports large-scale MIMO far-field static and dynamic object channel modeling, supports sub-6GHz and millimeter wave bands, and has flexible adaptation capabilities such as multi-region, multi-antenna, multi-band, and multi-link. This solution is widely applicable to physical layer tasks such as channel estimation and beamforming, as well as perception layer tasks such as pose estimation and three-dimensional reconstruction. It can effectively meet the application needs of multiple fields in complex scenarios and provide strong technical support for the research and development of wireless communication and perception systems.

[0108] Corresponding to the aforementioned embodiment of a method for generating a twin AI dataset of multimodal communication and perception channels, the present invention also provides an embodiment of a device for generating a twin AI dataset of multimodal communication and perception channels.

[0109] See also Figure 9 , an embodiment of the present invention provides a multimodal communication and perception channel twin AI dataset generation device, including a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it is used to implement a multimodal communication and perception channel twin AI dataset generation method in the above embodiment.

[0110] An embodiment of a device for generating a multimodal communication and perception channel twin AI data set provided by the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory for execution. From the hardware level, if Figure 9 As shown, this is a hardware structure diagram of any device with data processing capability where a multimodal communication and perception channel twin AI data set generation device provided by the present invention is located, except Figure 9 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0111] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0112] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present invention. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0113] An embodiment of the present invention also provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, it implements a method for generating a multimodal communication and perception channel twin AI data set in the above embodiment.

[0114] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device of any device with data processing capabilities, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0115] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the method for generating a multimodal communication and perception channel twin AI data set.

[0116] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered merely as exemplary, and the true scope and spirit of the present application are indicated by the claims.

[0117] It should be understood that the above general description and the detailed description that follows are exemplary and explanatory only and do not limit the present application. The present application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope of the present application. The scope of the present application is limited only by the appended claims.

Claims

1. A method for generating a twin AI dataset of multimodal communication and perception channels, characterized in that: The following steps are involved: S1. Establish high-precision three-dimensional indoor scene models and outdoor scene models; S2. Configure the parameters of the signal transceiver and RIS, determine the dielectric constant and magnetic permeability of the surfaces of different objects in the scene based on the signal center frequency, and the reflection, refraction, scattering and diffraction effects of electromagnetic waves in different media based on the theory of geometric optics and geometric diffraction; S3. Calculate the channel parameters of each ray according to the free space path loss formula, generate a channel matrix and channel intensity heat map based on the channel parameters, and generate a complete data set based on the scene information; S4. Align the coordinates of the scene and the ray, load and render the ray data and channel intensity heat map based on the movement path selected by the user, and obtain a visualization file of the multimodal channel twin AI dataset.

2. A method for generating a multimodal communication and perception channel twin AI dataset according to claim 1, characterized in that: In S1, the construction process of the outdoor scene model includes: Select any area in the Open Street Map and call the height, texture, and other object data recorded by the satellite system to generate a rough 3D scene at the LoD1 level. Then, segment the objects in the rough 3D scene, smooth their edges, and reconstruct the blurred textures to ensure that the building details are close to the real scene, resulting in a LoD2 level scene. Then, add green belts, ground and roads, trees, street lights, and vehicles, and finally save the generated LoD3 level scene in XML file format.

3. The method for generating a multimodal communication and perception channel twin AI dataset according to claim 1, characterized in that: In S1, the process of constructing the indoor scene model includes: Use LiDAR to perform a global 3D scan of the indoor environment to obtain LoD1-level point cloud data containing spatial coordinates and distance information. After denoising and filtering, the 3D structure of the environment is constructed to obtain a LoD2-level indoor global 3D map model. Texture mapping and lighting optimization rendering are performed to generate a LoD3-level model. Finally, the generated scene is saved in the FBX file format.

4. The method for generating a multimodal communication and perception channel twin AI dataset according to claim 1, characterized in that: The configuration of the parameters of the signal transceiver and RIS in S2 includes: Configure key parameters of the transceiver and RIS, including location coordinates, row and column array size, movement speed, antenna pattern, and polarization characteristics.

5. The method for generating a multimodal communication and perception channel twin AI dataset according to claim 4 is characterized in that: In step S2, the RIS supports both single-beam and multi-beam generation methods, including: precisely controlling the phase distribution of the RIS's reflector units based on a path difference focusing scheme and a gradient phase design method to achieve single-beam focusing in the target direction; and dynamically adjusting the RIS's reflection coefficient array using a stochastic gradient descent optimization algorithm to generate a multi-beam codebook that meets the needs of multiple users or multiple targets, providing efficient support for multi-point communications in complex scenarios.

6. The method for generating a multimodal communication and perception channel twin AI dataset according to claim 1, characterized in that: S2 also uses a GPU-accelerated ray tracing method, including: initializing some rays at the transmitter based on computing resources, selecting key propagation directions to reduce redundant calculations; obtaining the phase and position information of each reflection point of the simulated ray in the scene, and storing the phase, power, propagation angle and delay information of each ray through arithmetic sampling; The Optix ray tracing core is called to perform GPU parallel acceleration on the simulation process; the generated path data is cleaned and optimized, abnormal paths are deleted, and deviations are corrected.

7. The method for generating a multimodal communication and perception channel twin AI dataset according to claim 4, characterized in that: The channel dataset generated in S3 includes the construction process of the MIMO channel matrix, including: synthesizing the channel impulse response to obtain the delay characteristics, generating the MIMO channel matrix to describe the response of the multi-antenna system, calculating the received signal strength and path loss, accurately drawing the radio map to display the key communication indicators at different locations, discretizing the path information into a CSV table format through equal-interval sampling, and saving the generated channel thermal intensity map in high-resolution JPG format.

8. The method for generating a multimodal communication and perception channel twin AI dataset according to claim 1, characterized in that: The S4 specifically includes: importing scene data through the Babylon.js web platform to align the scene coordinate origin with the unit coordinate vector; importing ray coordinate data in CSV format, and generating a complete ray path based on the starting and ending points of each ray; loading a channel intensity heat map at a specified height in the scene, and scaling the heat map to a matching position; rendering ray data and channel intensity heat maps in real time in a three-dimensional scene through GPU acceleration; using keyframe technology to set the time interval for each frame, loading and rendering data frame by frame according to the above operations, and finally generating a coherent dynamic visualization animation, which is exported as a video file to intuitively display the channel propagation characteristics and dynamic changes in large-scale scenes.

9. A multimodal communication and perception channel twin AI dataset generation system, characterized by: include: The scene twin module is used to build high-precision three-dimensional indoor and outdoor scene models; The electromagnetic signal twin module is used to configure the parameters of the signal transceiver and RIS, determine the dielectric constant and magnetic permeability of different object surfaces in the scene based on the signal center frequency, and analyze the reflection, refraction, scattering and diffraction effects of electromagnetic waves in different media based on the theory of geometric optics and geometric diffraction. The multimodal data generation module is used to calculate the channel parameters of each ray according to the free space path loss formula, generate the channel matrix and channel intensity heat map based on the channel parameters, and generate a complete data set based on the scene information; The visualization module is used to align the coordinates of the scene and rays, load and render the ray data and channel intensity heat map based on the movement path selected by the user, and obtain a visualization file of the multimodal channel twin AI dataset.

10. A device for generating a multimodal communication and perception channel twin AI data set, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the processor executes the executable code, it implements a method for generating a multimodal communication and perception channel twin AI data set according to any one of claims 1-8.

Images