Hybrid teaching method, system and electronic device

Abstract

The present disclosure discloses a hybrid teaching method, system and electronic device, which collects point cloud data of a teaching environment and pre-processes the data, generates a point cloud scene model by using a decoupling sparse prior guided diffusion compression model, improves display quality by using an adversarial diffusion compression, deploys a teaching management system based on a micro-service architecture to realize multi-user real-time interaction, collects interaction behavior data and performs deep learning analysis to generate an evaluation report. The present disclosure greatly improves the construction efficiency and image quality of the virtual teaching environment through point cloud processing and image optimization methods, and realizes the distributed deployment and real-time interaction capability of the teaching platform, providing technical support for hybrid teaching.

Description

Technical Field

[0001] This disclosure relates to the fields of artificial intelligence, virtual reality and education technology, and in particular to a blended learning method, system and electronic device. Background Technology

[0002] With the development of VR / AR (Virtual Reality / Augmented Reality) technology, virtual teaching environments have been gradually applied in the education field.

[0003] Traditional VR / AR teaching systems lack efficient processing capabilities for point cloud data, resulting in insufficient accuracy in virtual scene reconstruction; and in the real-time rendering process, it is difficult to balance image quality and system performance.

[0004] Furthermore, most existing virtual teaching systems adopt a traditional client-server architecture, which makes it difficult to support large-scale concurrent access and real-time interaction. When multiple users enter the virtual teaching environment at the same time, the system often experiences response delays or inconsistent states, seriously affecting the teaching effect.

[0005] Therefore, there is a need for a blended learning approach that can efficiently process point cloud data, improve image quality, and support large-scale real-time interaction. Summary of the Invention

[0006] To achieve one of the aforementioned objectives, this disclosure provides a hybrid teaching method, comprising: acquiring point cloud data of a teaching environment; performing noise removal and density equalization processing on the point cloud data to obtain preprocessed point cloud data; inputting the preprocessed point cloud data into a decoupled sparse prior guided diffusion compression model; extracting local and global features of the point cloud through a spatial feature encoder to generate a point cloud scene model; performing multi-scale discrimination and generation processing on the point cloud scene model; calculating content loss, structural loss, and perceptual loss to optimize image quality and obtain an optimized virtual teaching scene; configuring a microservice architecture and deploying a teaching management system; implementing real-time multi-user interaction through WebRTC and WebSocket to construct a hybrid teaching platform; collecting user interaction behavior data and learning progress data in the virtual teaching scene; analyzing the interaction behavior data and learning progress data based on a deep learning algorithm to generate a learning evaluation report.

[0007] Furthermore, point cloud data of the teaching environment is acquired, and noise removal and density equalization are performed on the point cloud data to obtain preprocessed point cloud data, including:

[0008] Point cloud data of the teaching environment is acquired, and raw point cloud data is generated by multi-site scanning and automatic registration.

[0009] The original point cloud data is subjected to noise identification, and abnormal points are filtered based on a density threshold to obtain filtered point cloud data.

[0010] The filtered point cloud data is divided into grids, and the point cloud density in each grid is calculated to obtain preprocessed point cloud data.

[0011] Furthermore, the preprocessed point cloud data is input into a decoupled sparse prior guided diffusion compression model, and local and global features of the point cloud are extracted through a spatial feature encoder to generate a point cloud scene model, including:

[0012] The preprocessed point cloud data is processed by a multilayer perceptron to extract local features of the point cloud.

[0013] Based on the local features, a graph convolutional network is used to process the points cloud and extract global features.

[0014] The local and global features are input into the feature decoupling module to extract geometric and semantic features and generate a point cloud scene model.

[0015] Furthermore, the local features and the global features are input into the feature decoupling module to extract geometric and semantic features, generating a point cloud scene model, including:

[0016] The local and global features are input into the condition generator, and the feature generation process is adjusted based on the diffusion time step to obtain the time feature vector;

[0017] Variational inference processing is performed on the time feature vector to obtain a sparse representation of the point cloud data;

[0018] Feature reconstruction is performed based on the sparse representation to generate a point cloud scene model.

[0019] Furthermore, the point cloud scene model undergoes multi-scale discrimination and generation processing, and content loss, structural loss, and perceptual loss are calculated to optimize image quality, resulting in an optimized virtual teaching scene, including:

[0020] The point cloud scene model is input into a multi-scale discriminator network to extract and evaluate image features at different scales, thereby obtaining image quality assessment results.

[0021] Based on the image quality assessment results, the image is progressively generated and thinned to obtain a high-resolution image;

[0022] The content loss, structural loss, and perceptual loss of the high-resolution image are calculated to obtain the optimized virtual teaching scene.

[0023] Furthermore, a microservice architecture is configured and a teaching management system is deployed. Multi-user real-time interaction is achieved through WebRTC and WebSocket, building a hybrid teaching platform, including:

[0024] Deploy user authentication and resource management services, configure user authentication protocols and resource access control policies, and build the basic service layer;

[0025] Based on the aforementioned basic service layer, configure real-time communication services, establish a WebRTC audio and video transmission channel, configure the transmission control protocol, and realize real-time data transmission.

[0026] The WebSocket protocol is used for instant messaging communication to enable real-time interaction among multiple users and to build a hybrid teaching platform.

[0027] Construct a scene tree manager to maintain the hierarchical relationship of objects in the optimized virtual teaching scene and establish a scene management framework;

[0028] Based on the aforementioned scene management framework, a state synchronizer is implemented to perform state synchronization processing on multi-user operations, ensuring scene state consistency.

[0029] The scene state is rendered in real time to build a hybrid teaching platform.

[0030] Furthermore, user interaction behavior data and learning progress data in the virtual teaching scenario are collected. Based on deep learning algorithms, this interaction behavior data and learning progress data are analyzed to generate a learning evaluation report, including:

[0031] Record user interaction data in the virtual teaching scenario and generate a sequence of operation behaviors;

[0032] Collect users' course completion status and exercise scores to obtain learning progress data;

[0033] The operation sequence and the learning progress data are preprocessed and standardized to extract key learning features;

[0034] Input the key learning features into the deep learning model to generate a learning evaluation report.

[0035] Furthermore, the key learning features are input into the deep learning model to generate a learning evaluation report, including:

[0036] Based on the key learning features, a learner profile model is constructed, and a learning feature vector is generated.

[0037] The deep learning model is used to analyze the learning feature vectors to assess skill mastery, knowledge comprehension, and learning engagement, resulting in multi-dimensional evaluation results.

[0038] A learning assessment report is generated based on the multi-dimensional assessment results.

[0039] To achieve the above-mentioned objectives, this disclosure provides a hybrid teaching device, including: a point cloud data preprocessing module, used to acquire point cloud data of the teaching environment, and to perform noise removal and density equalization processing on the point cloud data to obtain preprocessed point cloud data;

[0040] The scene modeling module is used to input the preprocessed point cloud data into the decoupled sparse prior guided diffusion compression model, and extract the local and global features of the point cloud through the spatial feature encoder to generate a point cloud scene model.

[0041] The image optimization module is used to perform multi-scale discrimination and generation processing on the point cloud scene model, calculate content loss, structural loss and perceptual loss to optimize image quality, and obtain an optimized virtual teaching scene.

[0042] The teaching platform module is used to configure the microservice architecture and deploy the teaching management system. It enables real-time multi-user interaction through WebRTC and WebSocket to build a hybrid teaching platform.

[0043] The data analysis module is used to collect user interaction behavior data and learning progress data in the virtual teaching scenario, analyze the interaction behavior data and learning progress data based on deep learning algorithms, and generate a learning evaluation report.

[0044] To achieve the above-mentioned objectives, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the hybrid teaching method described above.

[0045] Compared to existing technologies, the technical advantages of this disclosure are as follows: The hybrid teaching method, system, and electronic device disclosed herein significantly improve the efficiency and quality of virtual scene reconstruction by decoupling the sparse prior guided diffusion compression model of point clouds and extracting local and global features of point clouds through a spatial feature encoder; the image super-resolution adversarial diffusion compression method for VR / AR scenes improves the display effect while ensuring performance through multi-scale discrimination and perceptual loss optimization; and the distributed teaching platform based on microservice architecture supports large-scale concurrent access and real-time interaction through WebRTC and WebSocket technologies, providing technical support for hybrid teaching. Attached Figure Description

[0046] Figure 1 A flowchart illustrating a blended teaching method provided in one embodiment of this disclosure;

[0047] Figures 2 to 7They are respectively Figure 1 A schematic diagram illustrating the specific implementation process of one of the steps;

[0048] Figure 8 This is a schematic diagram of the modules of the blended learning system provided in this disclosure. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0050] like Figure 1 As shown, the blended learning method provided in this disclosure includes the following steps:

[0051] S1. Acquire point cloud data of the teaching environment, and perform noise removal and density equalization processing on the point cloud data to obtain preprocessed point cloud data;

[0052] S2. Input the preprocessed point cloud data into the decoupled sparse prior guided diffusion compression model, and extract the local and global features of the point cloud through the spatial feature encoder to generate a point cloud scene model;

[0053] S3. Perform multi-scale discrimination and generation processing on the point cloud scene model, calculate content loss, structural loss and perceptual loss to optimize image quality, and obtain the optimized virtual teaching scene;

[0054] S4. Configure a microservice architecture and deploy the teaching management system, and realize real-time multi-user interaction through WebRTC and WebSocket to build a hybrid teaching platform;

[0055] S5. Collect user interaction behavior data and learning progress data in the virtual teaching scenario, analyze the interaction behavior data and learning progress data based on deep learning algorithms, and generate a learning evaluation report.

[0056] The hybrid teaching method disclosed herein significantly improves the efficiency and quality of virtual scene reconstruction by decoupling a sparse prior-guided diffusion compression model from point clouds and extracting local and global features from point clouds using a spatial feature encoder. For VR / AR scenarios, an adversarial diffusion compression method for image super-resolution improves display quality while maintaining performance through multi-scale discrimination and perceptual loss optimization. A distributed teaching platform based on a microservice architecture supports large-scale concurrent access and real-time interaction through WebRTC and WebSocket technologies, providing technical support for hybrid teaching.

[0057] Combination Figure 2 As shown, step S1 includes:

[0058] S11. Acquire point cloud data of the teaching environment, and generate raw point cloud data by multi-site scanning and automatic registration processing; This disclosure acquires high-precision three-dimensional data of the real teaching environment; Step S11 uses advanced laser scanning equipment to perform multi-site scanning of the teaching environment; Multiple scanning stations are planned in the teaching environment, and the arrangement of the stations takes into account spatial coverage and visual overlap to ensure that all areas can be observed by at least two stations; In actual operation, for a standard classroom, it is usually necessary to set up 4-6 scanning stations, distributed in the four corners and the middle of the classroom, to form an all-round scanning network.

[0059] Ideally, each scanning station uses a high-precision time-of-flight (TOF) or phase-difference laser scanner to acquire point clouds. When the scanner is working, the laser beam is emitted from different angles and the reflected signals are received. The precise three-dimensional coordinates of the points on the object surface are calculated by measuring the laser propagation time or phase difference. Modern laser scanners typically have a sampling accuracy in the millions, an angular resolution of up to 0.1° in the horizontal and vertical directions, and a ranging accuracy of up to the millimeter level, enabling them to accurately capture the geometric details of the teaching environment. At the same time, some high-end scanners are also equipped with color cameras, which can simultaneously acquire color information to add texture details to the point cloud data.

[0060] After multi-site scanning is completed, automatic registration is performed to integrate local point cloud data from different sites into a unified coordinate system. The registration process employs a two-stage strategy: coarse registration and fine registration. In the coarse registration stage, feature point matching algorithms are used to identify corresponding point pairs between different point clouds. For example, local feature descriptors such as FPFH (Fast Point Feature Histograms) or SHOT (Signature of Histograms of Orientations) are used to extract point cloud features, and then the RANSAC (Random Sample Consensus) algorithm is used for feature matching to quickly estimate the initial transformation matrix. In the fine registration stage, the Iterative Closest Point (ICP) algorithm is used for precise alignment. The ICP algorithm iteratively optimizes the transformation matrix to minimize the Euclidean distance between corresponding point pairs, achieving precise alignment of the point clouds. To improve the robustness of registration, this disclosure also employs a point-to-surface distance metric and a weighting strategy, assigning different weights to planar regions and feature-rich regions, effectively reducing registration errors.

[0061] Furthermore, all point cloud data are merged to generate complete raw point cloud data containing the entire teaching environment. This data typically contains tens of millions of 3D points, each containing position coordinates (X,Y,Z) and possible color information (R,G,B), fully recording the geometry and visual appearance of the teaching environment, providing a rich raw data source for subsequent processing.

[0062] S12. Noise identification is performed on the original point cloud data, and outliers are filtered based on density thresholds to obtain filtered point cloud data. The original point cloud data contains rich environmental information, as well as various noises and outliers; these noises come from various sources, including equipment measurement errors, ambient light interference, and changes in reflective surface characteristics; the presence of noise will affect the accuracy of subsequent processing, so noise identification and filtering are required; step S12 uses a density analysis-based method to identify and remove outliers in the point cloud, thereby improving data quality.

[0063] Specifically, noise identification is performed on the raw point cloud data. The identification process is based on observation; points on the surface of a real object typically have relatively stable local density, while noise often manifests as isolated points or small clusters with significantly abnormal density. Accordingly, local density analysis methods are used to identify potential noise. In this specific implementation, for each point P in the point cloud, its local neighborhood is defined as a spherical region centered on P with a radius of r (the value of r is adaptively determined based on the overall density of the point cloud, typically 2-3 times the average point spacing). The number of points N contained within the neighborhood is calculated, and N is used as the local density estimate at point P. By statistically analyzing the local density of all points, a density distribution model can be established to identify points with abnormal density.

[0064] Furthermore, outlier filtering is performed based on a density threshold. This disclosure can employ an adaptive threshold strategy instead of a simple global fixed threshold. Statistical analysis is performed on the local density values ​​of all points to calculate the mean μ and standard deviation σ of the density distribution. A filtering threshold T = μ - k·σ is set, where k is an adjustable parameter (usually 2-3) to control the strictness of the filtering. Points with local densities below the threshold T are marked as noise and removed from the dataset. The adaptive threshold method of this disclosure can automatically adjust the filtering criteria according to the overall characteristics of the point cloud, adapting to point cloud data under different scenarios and acquisition conditions.

[0065] To further improve the accuracy of noise recognition, morphological feature analysis can be combined. Points on the surface of real objects usually form continuous local planes or curved surfaces, exhibiting a certain geometric continuity. Noise, on the other hand, often breaks this continuity. Principal component analysis (PCA) is used to calculate the local surface characteristics of the neighborhood of each point, including the surface normal vector and principal curvature. Combining density features with geometric features can more accurately distinguish between real data points and noise, reducing the false positive rate.

[0066] After the above noise identification and filtering processes, filtered point cloud data is obtained. The noise in these data is significantly reduced, and the distribution of points is more consistent with the characteristics of the surface of real objects, laying the foundation for subsequent density equalization and feature extraction. Compared with traditional fixed radius or fixed threshold filtering methods, the adaptive strategy adopted in step S12 is more adaptable to changes in point cloud density in complex scenes, maintaining the accuracy and effectiveness of filtering.

[0067] Step S12 calculates the neighborhood density of each point in the original point cloud data and identifies outliers that are below a preset density threshold. The outliers are filtered using a statistical outlier filtering algorithm to generate filtered point cloud data. This effectively removes noise caused by equipment errors or environmental interference, improving the accuracy of subsequent processing.

[0068] S13. Divide the filtered point cloud data into grids, calculate the point cloud density in each grid, and obtain the preprocessed point cloud data.

[0069] Due to the working characteristics of the self-scanning device and the material differences of the object surface, the quality of the point cloud data is initially improved after noise filtering; however, the problem of uneven density distribution still exists. This manifests as excessively high point density in close-range areas, excessively low point density in distant areas, and density anomalies in complex geometric structures. Uneven density will affect the consistency and accuracy of subsequent feature extraction. In order to further improve the quality of point cloud data and solve the problem of uneven density distribution, this disclosure implements density equalization processing to make the point cloud distribution more uniform and reasonable.

[0070] Specifically, the filtered point cloud data is divided into regular grids; the entire point cloud space is divided into cubic grids (voxels) of equal size, each grid having the same dimensions and shape; the choice of grid size is crucial, as too large a grid will lead to loss of detail, while too small a grid will increase computational overhead and may produce empty grids; in practical applications, the grid size is usually determined based on the overall scale of the point cloud and the desired resolution. For point clouds in a classroom environment, the grid side length is usually set to 5-10 cm to maintain computational efficiency while meeting accuracy requirements.

[0071] After completing the mesh generation, the point cloud density within each mesh is calculated. The density can be simply defined as the number of points within the mesh, or a more complex definition can be used, such as the inverse average of the distances between points. All meshes are traversed, the number of points in each mesh is counted and recorded in the density map, forming a density distribution map of the entire point cloud space. This density distribution information will be used for subsequent equalization processing.

[0072] Adaptive density adjustment is performed based on density maps. For grids with excessively high density (grids with significantly more points than the average), random downsampling or voxel centering is used to filter points. Random downsampling simply selects a fixed number of points randomly from the grid, which is simple to implement and computationally efficient. Voxel centering calculates the average position of all points in the grid and uses this average point to replace all points in the original grid, which better preserves the distribution characteristics of the points. This disclosure prioritizes the use of voxel centering for downsampling to maintain the accuracy of geometric details. For grids with excessively low density (grids with significantly fewer points than the average and not zero), local interpolation is used to supplement the point cloud. The interpolation process analyzes the distribution characteristics of points around the grid and generates new points in low-density areas. In specific implementations, methods such as moving least squares (MLS) surface reconstruction or radial basis function (RBF) interpolation can be used to infer the surface equation based on known points around the grid, and then uniformly sample on the surface to generate new points that conform to the surface characteristics. The interpolation method based on surface reconstruction in this disclosure can maintain the geometric continuity of the point cloud and avoid generating pseudo-points that do not conform to the actual object surface.

[0073] For completely blank grids (i.e., grids that do not contain any points), their physical meaning needs to be determined. Empty grids may represent actual blank areas (such as transparent objects, mirrors, or gaps between objects), or they may be due to data loss caused by scan occlusion. By analyzing the location of the grid and the distribution characteristics of surrounding points, we can identify real blank areas and data loss areas. For data loss areas, we can try to fill them in using more advanced interpolation or inference methods, such as using the symmetry or repetition patterns of the point cloud. For real blank areas, we can preserve their blank state to avoid introducing erroneous data.

[0074] Through the above meshing, density calculation, and adaptive adjustment processes, preprocessed point cloud data is obtained. These data have a more uniform density distribution, which not only preserves the geometric details of the original point cloud but also eliminates the negative impact of uneven density, providing high-quality input data for subsequent feature extraction and scene modeling. Compared with traditional global uniform downsampling or simple threshold filtering methods, the density equalization process in this step is more intelligent and accurate, and can better maintain the geometric integrity and detail of the point cloud.

[0075] Step S13 divides the filtered point cloud data into uniform grids, calculates the point cloud density in each grid, and performs adaptive sampling based on the density distribution to ensure the uniformity of the point cloud distribution. For areas with excessively high density, downsampling is performed; for areas with excessively low density, interpolation is performed to supplement the density, and finally, preprocessed point cloud data with uniform density is obtained.

[0076] The high-quality preprocessed point cloud data obtained through step S1 has the characteristics of low noise, uniform distribution and complete structure, which provides a solid foundation for subsequent feature extraction and scene reconstruction. Compared with traditional point cloud preprocessing methods, step S1 introduces multi-site registration and adaptive density equalization technology, which significantly improves the quality and usability of point cloud data and creates favorable conditions for the accurate construction of virtual teaching environments.

[0077] Combination Figure 3 As shown, step S2 includes:

[0078] Step S21. Perform multilayer perceptron processing on the preprocessed point cloud data to extract local features of the point cloud; use the multilayer perceptron network to extract local features of the point cloud; local features are high-dimensional vector representations that describe the geometric and semantic characteristics of points and their neighborhoods, and are the basis for understanding the structure of point clouds; Step S21 adopts a deep learning method to automatically learn and extract these features through the multilayer perceptron network, avoiding the limitations of manually designing feature descriptors in traditional methods.

[0079] For each point in the point cloud, a local neighborhood is defined. The selection of the local neighborhood has a significant impact on the quality of feature extraction. The K-Nearest Neighbors (KNN) method is used to determine the neighborhood of each point, that is, to find the K points with the closest Euclidean distance for each point as its neighborhood. The choice of K value needs to balance the richness of local information and computational efficiency, and is usually between 16 and 32. For point clouds in a classroom environment, K=24 provides a good balance. Compared with fixed radius search, the KNN method can better adapt to changes in point cloud density and ensure that each point has sufficient neighborhood information.

[0080] After determining the neighborhood, input features are constructed for each point and its neighborhood. To maintain the translation invariance of feature extraction, the absolute coordinates of the points are not used directly; instead, relative coordinates are used. Specifically, for point p and its K neighbors {q1, q2, ..., q...} k}, calculate the relative coordinates {q1-p, q2-p, ..., q} k `-p` is used as the initial feature; in addition to spatial coordinates, other available features, such as point color, reflection intensity, and normal vector, are also incorporated to enrich the expressive power of the features. These original features are concatenated into a high-dimensional vector, which serves as the input to the multilayer perceptron.

[0081] A Multilayer Perceptron (MLP) network is a feedforward neural network composed of multiple fully connected layers, capable of learning nonlinear mappings of input data. The MLP network disclosed herein employs a hierarchical design, containing multiple hidden layers, each followed by a batch normalization layer and a nonlinear activation function (such as ReLU). An example network structure is MLP(64-128-256-512), representing four hidden layers with output dimensions of 64, 128, 256, and 512, respectively. This progressively expanding dimensionality design enables the network to learn more complex feature representations layer by layer, from low-level geometric features to high-level semantic features.

[0082] The optimization of MLP networks adopts an end-to-end learning approach, using loss functions from point cloud segmentation or classification tasks for supervised training. The training process uses large-scale point cloud datasets, such as ModelNet40 or ShapeNet, and updates network parameters through backpropagation to enable them to extract discriminative features of point clouds. To enhance the model's generalization ability, data augmentation techniques, such as random rotation, scaling, and jitter, are also introduced during training to make the model more robust to point cloud transformations.

[0083] A trained MLP network can map each point and its neighborhood into a high-dimensional feature vector. These vectors encode the local geometric structure information of the point. Specifically, different dimensions of the vectors may respond to different geometric structures such as edges, corners, and planes, forming a complete description of the local morphology. Compared with traditional hand-designed features, the features learned by MLPs have stronger expressive power and discriminative ability, and can better capture the subtle structural changes of point clouds.

[0084] To enhance the positional sensitivity of features, positional encoding techniques are introduced. Traditional MLPs tend to lose spatial arrangement information when processing neighborhood points, affecting the discriminative power of features. To address this issue, relative positional encoding is added during feature extraction, using sine and cosine functions to map spatial coordinates to a higher-dimensional space, enhancing the model's ability to perceive spatial structure. This encoding method is inspired by the Transformer architecture and significantly improves the ability of features to express local structures.

[0085] Through the multilayer perceptron processing described above, rich local features were extracted from the point cloud data. These features formed a high-dimensional representation describing the local geometric structure of the point cloud, laying the foundation for subsequent global feature extraction and scene understanding.

[0086] Step S21 inputs the preprocessed point cloud data into a multilayer perceptron network composed of multiple fully connected layers; the network extracts features from each point and its neighborhood, capturing the local geometric structure information of the point; in order to improve the expressive power of the features, the network adopts a nonlinear activation function and batch normalization processing, and finally outputs a high-dimensional local feature representation of the point cloud.

[0087] Step S22. Based on the local features, perform graph convolutional network processing to extract the global features of the point cloud. Although local features describe the microstructure of the point cloud, they lack an understanding of the overall shape and are difficult to capture the long-range dependencies between points. In order to obtain the global feature representation of the point cloud, step S22 uses a graph convolutional network (GCN) to process the local features and realize the transformation from local information to global context. Graph convolutional networks are a type of deep learning model that is specifically designed to process non-Euclidean structure data and are suitable for irregularly arranged three-dimensional point sets such as point clouds.

[0088] A graph structure G = (V, E) is constructed based on point clouds, where the vertex set V corresponds to each point in the point cloud, and the edge set E represents the connection relationship between points. The construction of the graph is crucial for subsequent processing, and multiple strategies are used to determine the connection method of the edges: spatial proximity graphs connect K nearest neighbors (usually K=16 or 20) based on the Euclidean distance between points; feature similarity graphs connect points with similar features based on the cosine similarity of local features; dynamic graphs dynamically update the connections according to the current features during the forward propagation of the network; in practical applications, a hybrid strategy combining spatial distance and feature similarity is preferred, which considers both geometric proximity and feature-level connections to form a more expressive graph structure.

[0089] After constructing the graph, graph convolution is performed on the local features of each point. Unlike traditional convolution, graph convolution propagates information on the graph through a message passing mechanism. Specifically, a spatial graph convolution framework can be used, with the core formula being:

[0090] hl+1i=σ(Wl·AGG({hlj|j∈ N(i)∪{i}})),

[0091] Where hli represents the feature of the i-th point in the l-th layer, N(i) represents the neighborhood of the i-th point, AGG is the aggregation function, Wl is the learnable weight matrix, and σ is the non-linear activation function; the aggregation function can be a simple mean or maximum operation, or a more complex attention weighted sum or gating mechanism.

[0092] This disclosure employs an attention-based aggregation function to dynamically assign importance weights to different neighboring points based on feature similarity, thereby improving the selectivity and effectiveness of feature propagation.

[0093] Graph convolutional networks employ a multi-layered architecture, typically containing 3-4 graph convolutional layers, each followed by batch normalization and ReLU activation functions. The feature dimension gradually increases between layers (e.g., 64-128-256-512), enabling the network to learn increasingly abstract feature representations. Furthermore, residual connections are introduced to directly pass features from shallower layers to deeper layers, mitigating the vanishing gradient problem in deep networks and improving training stability and model expressive power.

[0094] To enhance the network's ability to capture multi-scale features, a parallel multi-receptive-field graph convolution module is adopted. This module simultaneously performs graph convolution with neighborhoods of different sizes (e.g., K=4, 8, 16), and then fuses features of different scales to form a multi-scale understanding of the point cloud structure. This design enables the network to focus on both local details and structural features over a larger range, improving the comprehensiveness and robustness of the features.

[0095] Based on the graph convolutional layer, graph pooling is introduced to gradually reduce the resolution of the point cloud and extract higher-level features. Graph pooling uses a hierarchical clustering method to group similar points and represent them with center points to form a multi-resolution point cloud representation. This hierarchical processing is similar to the pooling operation in traditional convolutional networks, which can expand the receptive field, reduce computational complexity, and preserve global structural information.

[0096] After multi-layer graph convolution and pooling, the features are integrated into a fixed-dimensional global descriptor through global pooling (such as global max pooling or global average pooling). This global feature vector encodes the overall morphology and semantic information of the point cloud, including high-level attributes such as object category, size, and pose. Compared with local features, global features focus more on the overall structure and semantic content, providing a macroscopic perspective for subsequent scene understanding and interaction.

[0097] By using graph convolutional networks, the transformation from local features to global features was achieved, establishing the connections between different parts of the point cloud and forming a holistic understanding of the point cloud structure. The global feature extraction method overcomes the challenges of the disorder and irregular distribution of point clouds, providing a solid foundation for subsequent feature decoupling and scene modeling.

[0098] Step S22 constructs a point cloud graph structure based on the local features extracted in the previous step S21 and inputs it into a graph convolutional network. The graph convolutional network aggregates the feature information of neighboring points through a message passing mechanism, capturing the long-range dependencies and global structural information of the point cloud. After multiple layers of graph convolution and pooling operations, the global feature representation of the point cloud is obtained.

[0099] Step S23. Input the local features and the global features into the feature decoupling module to extract geometric features and semantic features, and generate a point cloud scene model.

[0100] After acquiring local and global features, the feature decoupling stage begins. The goal is to separate the hybrid features into geometric and semantic features, enabling a more refined understanding and representation of point cloud data. Geometric features describe the spatial location, normal vectors, and other physical attributes of points, while semantic features represent higher-level concepts such as object categories and functional uses. Decoupled representation not only improves the interpretability of features but also provides greater flexibility for subsequent scene editing and interaction.

[0101] Feature decoupling combines local and global features into a hybrid feature representation. For each point in the point cloud, the local feature vector is connected to the global feature vector to form a comprehensive representation that includes local details and global context. Local-global feature fusion allows each point to retain its unique local characteristics while also containing an understanding of the overall structure, providing a rich source of information for subsequent decoupling processing.

[0102] The feature decoupling module adopts a self-supervised learning framework, which does not require explicit labeled data and learns decoupling representations through feature reconstruction tasks. The core of this module is a variational autoencoder (VAE) structure, which consists of an encoder and a decoder. The encoder maps the mixed features to two independent latent spaces, corresponding to geometric features and semantic features, respectively. The decoder attempts to reconstruct the original mixed features from the two types of features. By minimizing the reconstruction error, the mixed features are decomposed into two complementary and information-rich parts.

[0103] To ensure the independence of the two types of features, orthogonal constraints and adversarial training mechanisms are introduced. Orthogonal constraints force geometric and semantic features to be orthogonal to each other in the latent space, reducing information redundancy. Adversarial training, through the discriminator network, ensures that one type of feature cannot predict the other type of feature, further enhancing the decoupling of features. These constraint mechanisms work together to ensure that truly decoupled feature representations are learned.

[0104] Geometric feature extraction focuses on the spatial structure and morphological characteristics of points. Through a dedicated branch in the encoder, feature vectors describing the geometric attributes of points, such as position, normal vector, and curvature, are learned and extracted. These features mainly reflect the shape information of the point cloud and are independent of specific object categories or functions. To enhance the shape perception capability of geometric features, an auxiliary task based on local surface reconstruction is introduced, enabling geometric features to accurately predict the local surface equations of points and improve the accuracy of geometric structure representation.

[0105] Semantic feature extraction focuses on the category and functional attributes of the object to which the point belongs; through another dedicated encoder branch, feature vectors containing high-level semantic information are learned to be extracted; these features mainly reflect the type, purpose and functional attributes of the object to which the point is located, and are relatively independent of the specific geometric shape; to enhance the discriminativeness of semantic features, an auxiliary task based on object classification is added to guide semantic features to effectively distinguish objects of different categories and improve semantic understanding ability.

[0106] Through the processing of the feature decoupling module, geometric and semantic feature representations of each point are obtained. These two types of features together constitute a comprehensive understanding of the point cloud. Compared with traditional hybrid features, decoupling features provide finer control granularity, allowing separate manipulation of the shape and semantic attributes of the point cloud, providing a flexible foundation for subsequent scene editing and interaction.

[0107] This disclosure provides a preferred embodiment, in conjunction with... Figure 4 As shown, step S23 specifically includes:

[0108] Step S231. Input the local and global features into the condition generator, adjust the feature generation process based on the diffusion time step, and obtain the time feature vector.

[0109] To achieve more accurate feature decoupling and scene modeling, a conditional generator based on a diffusion model is used to process the fused local and global features. The diffusion model is an important advancement in the field of generative models in recent years. Its core idea is to model the data generation process as a stepwise denoising process and achieve high-quality generation by learning the mapping from noise to signal. Step S231 introduces the diffusion model into point cloud feature processing, uses a conditional diffusion generator, and adjusts the feature generation process using time step information to improve the quality and controllability of feature representation.

[0110] The input to the conditional generator consists of three parts: local features, global features, and time step encoding. Local and global features are combined into hybrid features through a feature fusion layer (such as concatenation or attention weighting). The time step is mapped to a temporal embedding vector, serving as conditional information to guide the generation process. The temporal embedding uses a sinusoidal positional encoding method to convert the scalar time step t into a high-dimensional vector representation.

[0111] e(t)i=sin(t / 100002i / d),e(t){i+1}=cos(t / 100002i / d), where d is the embedding dimension and i is the dimension index.

[0112] This encoding method enables the model to distinguish generation tasks at different time steps, assigning a unique feature representation to each step.

[0113] The architecture of the conditional generator can be based on the U-Net structure, which includes three parts: encoder, intermediate processing and decoder. The encoder maps the mixed features to the latent space through multi-layer downsampling convolution or self-attention modules. The intermediate processing module combines temporal embedding information to adjust the feature representation. The decoder restores the feature dimension through upsampling convolution and generates the final output. Residual connections and skip connections are interspersed throughout the architecture to ensure efficient information transfer and stable gradient flow.

[0114] The core of the diffusion process is time-conditional control. During the training phase, the forward diffusion process is simulated by adding Gaussian noise of different degrees to the features, forming a series of gradually degenerating feature states, corresponding to different time steps t∈[0,T]. The model learns the backward diffusion process, that is, given the time step t and the corresponding noise features, it predicts the denoising direction or the original features. The time-step-based conditional generation enables the model to learn the gradual changes in the feature space and capture information at different scales and levels of abstraction.

[0115] To enhance the generator's expressive power, a self-attention mechanism is integrated into the U-Net architecture. The self-attention module calculates the correlations within feature maps, enabling direct information exchange between distant feature points and overcoming the limitations of local receptive fields. This step is suitable for point cloud data processing, capturing long-range dependencies between points and generating more coherent feature representations. The calculation formula for the attention mechanism is:

[0116] Attention(Q,K,V) = softmax(QKT / √d)·V, where Q, K, and V are the query, key, and value matrices, respectively, obtained by linear transformation of the feature vectors, and d is the feature dimension.

[0117] The training process employs the variational lower bound (ELBO) objective function to optimize model parameters so that they can accurately predict noise or original features. To improve training stability and generation quality, techniques such as prediction variance learning, low-rank adaptation, and mixed-precision training are also introduced. These optimization strategies work together to enable the conditional generator to learn the structure of the point cloud feature space and generate high-quality feature representations at different time steps.

[0118] The trained conditional generator can generate temporal feature vectors based on given local features, global features, and time steps. These vectors encode the performance of features at different time scales and contain multi-level structural information, from subtle local details to large-scale global structures. Compared with traditional single feature representations, temporal feature vectors provide richer feature descriptions and provide a multi-perspective source of information for subsequent variational inference and feature reconstruction.

[0119] By using a condition generator, local and global features are transformed into feature vectors containing temporal information. This transformation enriches the expressive power of the features and improves the accuracy and quality of subsequent decoupling and reconstruction. This disclosure presents a feature processing method based on a diffusion model, which opens up new avenues for efficient representation and accurate modeling of point cloud data.

[0120] Step S231 uses a conditional generator based on the diffusion time step, taking local and global features as input. This generator can capture the geometric structure changes of point cloud data at different time scales. By introducing a time embedding vector, the feature generation process can be better controlled, the convergence performance of the model can be improved, and finally, a time feature vector is generated.

[0121] Step S232. Perform variational inference processing on the time feature vector to obtain a sparse representation of the point cloud data.

[0122] Step S232 is to learn a low-dimensional sparse representation of point cloud data. Sparse representation is an efficient data encoding method that uses a small number of non-zero elements to represent the original signal, which reduces storage and computation overhead while preserving the key information of the data. Step S232 adopts a variational inference framework to map high-dimensional temporal feature vectors to a low-dimensional latent space and ensures the sparsity of the representation through regularization constraints, thereby achieving efficient compression and accurate reconstruction of point cloud data.

[0123] The encoder network architecture adopts a multilayer perceptron or Transformer structure, containing multiple fully connected layers or self-attention layers, which progressively map high-dimensional feature vectors to low-dimensional distribution parameters. Each layer is followed by batch normalization and non-linear activation functions (such as LeakyReLU) to enhance the network's expressive power and training stability. The last layer of the network is divided into two parallel branches, which output the mean μ and the log-variance logσ², respectively. The latent vector is generated by the reparameterization technique z=μ+ε·exp(0.5·logσ²), where ε is the standard normal distribution sample.

[0124] To improve coding efficiency, this disclosure also introduces adaptive quantization technology. Traditional VAEs directly output continuous latent vectors, resulting in high storage overhead. Quantized VAEs (VQ-VAEs), on the other hand, discretize the latent vectors into finite codebook vectors, significantly reducing storage requirements. This disclosure employs an improved vector quantization method, dividing the latent space into multiple subspaces, each of which is quantized independently to form a product quantization structure. While maintaining a high compression ratio, it preserves the accuracy and diversity of the representation, providing a more efficient storage and transmission solution for point cloud data.

[0125] Preferably, in addition to sparsity and quantization, this disclosure also focuses on the structured characteristics of latent representations; traditional VAEs assume that different dimensions of latent vectors are independent of each other, ignoring the structural relationships between features; to capture these structures, graph-regularized VAEs are introduced, using graph Laplacian operators to constrain the structure of the latent space, ensuring that semantically similar features remain close in the latent space; this improves the semantic consistency of the encoding, making the decoding and reconstruction process more reliable.

[0126] Variational inference transforms high-dimensional temporal feature vectors into low-dimensional sparse representations. This representation significantly reduces data storage requirements (typically achieving compression ratios of 10:1 or higher) while retaining the key information needed to reconstruct the point cloud. Compared to traditional direct compression methods, sparse representations based on variational inference consider not only the statistical properties of the data but also geometric and semantic information, enabling them to more accurately capture the essential structure of the point cloud and provide high-quality intermediate representations for subsequent feature reconstruction.

[0127] Step S232 processes the temporal feature vector through a variational inference network to learn the latent distribution of the point cloud data. The variational inference network adopts an encoder-decoder structure to map high-dimensional features to a low-dimensional latent space and introduces sparsity constraints to make the latent representation sparsity. This sparse representation method not only preserves the key geometric information, but also greatly reduces the data storage requirements.

[0128] Step S233. Perform feature reconstruction based on the sparse representation to generate a point cloud scene model.

[0129] While sparse representation is efficient and information-rich, it needs to be converted back to the original point cloud form before it can be used for visualization and interaction. Step S233 performs feature reconstruction based on the sparse representation, converting the low-dimensional encoding into a complete point cloud scene model. The reconstruction process is divided into two stages: the first stage uses a decoder network to restore the sparse representation into geometric and semantic features; the second stage reconstructs the 3D point cloud based on these features to form a scene model for visualization and interaction. This reconstruction method based on decoupled features not only ensures the reconstruction quality but also provides fine control over the scene content.

[0130] The decoder network is the second part of the variational autoencoder framework, responsible for mapping sparse vectors in the latent space back to the high-dimensional feature space. The network architecture is similar to that of the encoder but in the opposite direction, gradually expanding the feature dimension through multi-layer transformations. To maintain the independence of the geometric and semantic information encoded in the sparse representation, the decoder adopts a dual-branch design to process and reconstruct geometric and semantic features separately. The two branches share the bottom network layers, but are computed independently in the high-level parts, ensuring that different types of information can be recovered separately and maintaining the decoupled nature of the features.

[0131] The geometric feature reconstruction branch focuses on recovering the spatial location, normal vectors, curvature, and other geometric attributes of points. The output dimension of this branch matches the number of geometric features of the points, typically allocating 3-9 channels (position coordinates and possible derived values ​​such as normal vectors) to each point. To improve the accuracy of geometric reconstruction, a multi-resolution reconstruction strategy is adopted, first reconstructing the coarse outline of the point cloud, and then gradually refining the local details, forming a hierarchical reconstruction process similar to coarse-precision control. This strategy not only improves reconstruction efficiency but also enhances the ability to express subtle geometric features.

[0132] The semantic feature reconstruction branch is responsible for recovering high-level semantic information such as the category and functional attributes of the points. The output of this branch is usually the probability distribution of the semantic category or the semantic embedding vector, which encodes the type and functional characteristics of the object to which the point belongs. To enhance the discriminativeness of semantic reconstruction, an auxiliary classifier is introduced to map the reconstructed semantic features to a predefined category space and guide the recovery process of semantic features through classification loss. This supervised auxiliary task significantly improves the accuracy and consistency of semantic reconstruction.

[0133] After feature reconstruction is completed, the point cloud generation stage begins, converting the reconstructed geometric and semantic features into a visualized 3D point cloud model. Point cloud generation employs a conditional generation method based on geometric and semantic features, using a learnable mapping function to convert the representation of the feature space into a set of 3D coordinate points. A multilayer perceptron network is used, taking the combination of geometric and semantic features as input and outputting the 3D coordinates of each point and possible additional attributes (such as color, reflectivity, etc.). To ensure the quality of the generated point cloud, various constraints are introduced, such as surface smoothness, point distribution uniformity, and structural integrity. Multi-objective optimization ensures that the reconstruction results conform to the laws of the physical world and visual expectations.

[0134] A key challenge in point cloud generation is maintaining the clarity of geometric details; traditional methods often result in blurred or lost details due to information loss. To address this issue, this disclosure introduces an adaptive detail enhancement mechanism, which automatically identifies complex regions based on the reconstructed geometric features and increases the sampling density or applies detail enhancement filters in these regions to strengthen the representation of features such as edges and corners. This adaptive processing method effectively improves the visual quality and geometric accuracy of the reconstructed point cloud.

[0135] Another important consideration is semantic consistency; in complex scenes, the semantic boundaries between different objects should be clear, and the semantic attributes of the same object should be consistent; by modeling the semantic relationships between points through Conditional Random Fields (CRF) or Graph Convolutional Networks (GCN), the spatial smoothness and consistency optimization of semantic labels can be achieved; this post-processing significantly reduces the noise and incoherence of semantic segmentation, providing a more coherent understanding of the scene.

[0136] A complete point cloud scene model not only includes static geometric and semantic information but also needs to support dynamic interaction and editing. This disclosure implements a parameterized representation of the scene based on reconstructed decoupled features, allowing users to adjust the scene content by modifying feature parameters. For example, users can modify geometric features to change the shape of objects or adjust semantic features to change the type of objects without affecting other characteristics. This feature-based parameterized editing greatly enhances the flexibility and usability of point cloud scenes, providing a solid foundation for subsequent teaching interactions.

[0137] Step S233 performs feature reconstruction based on sparse representation to generate a complete point cloud scene model. The reconstruction process adopts a decoupled attention mechanism to separate geometric features (such as the spatial location and normal vector of points) and semantic features (such as object category and functional attributes). This decoupled representation not only improves the interpretability of features, but also provides a more flexible operating space for subsequent scene reconstruction and interactive editing.

[0138] Combination Figure 5 As shown, step S3 includes:

[0139] S31. Input the point cloud scene model into the multi-scale discriminator network to extract and evaluate image features at different scales, and obtain image quality evaluation results;

[0140] The detailed structure and training of a multi-scale discriminator network can be achieved using the following methods:

[0141] The network structure adopts a hierarchical design, processing input images in parallel at three different resolution levels (256×256, 128×128, and 64×64). Each resolution branch contains 5 convolutional blocks, each consisting of a convolutional layer, an instance normalization layer, and a LeakyReLU activation function. The number of channels in the convolutional layers are {64, 128, 256, 512, 1}, the kernel size is 4×4, and the stride is 2. To enhance the perception of texture details, a self-attention module is introduced after the 2nd and 4th convolutional blocks, and the attention module uses 8 attention heads.

[0142] The network training data uses two types of image data: (1) 20,000 high-resolution scene images selected from the public dataset Places365; (2) 5,000 virtual teaching scene images generated using point cloud reconstruction of this public dataset; the training images are generated into multi-resolution versions through bilinear interpolation for different branches of the multi-scale discriminator.

[0143] The training process employs an adversarial training strategy, with the discriminator and generator being optimized alternately. The discriminator aims to distinguish between real high-resolution images and generated super-resolution images, while the generator aims to generate high-quality images that can deceive the discriminator. The training uses the RMSprop optimizer with a learning rate of 0.0001 and a batch size of 16. To improve training stability, gradient penalty techniques are used to constrain the Lipschitz continuity of the discriminator, with the penalty weight set to 10.

[0144] The discriminator uses the WGAN-GP loss function, which includes the discrimination scores of real samples, the discrimination scores of generated samples, and a gradient penalty term. For multi-scale features, the final loss is a weighted sum of the losses of the three resolution branches, with weights of {0.4, 0.3, 0.3}, respectively, with higher resolutions receiving greater weights.

[0145] Network performance was evaluated using discrimination accuracy and perceptual quality scores. Experiments showed that the multi-scale discriminator can effectively distinguish between real and generated images with a discrimination accuracy of over 85%, while providing effective gradient signals to guide generator optimization and significantly improving the visual realism of generated images.

[0146] When inputting the point cloud scene model into the multi-scale discriminator network, a hierarchical discriminator architecture is adopted. This discriminator operates simultaneously at multiple resolution levels, forming a pyramid structure that can comprehensively evaluate image quality from macroscopic structure to microscopic details. In practice, the input image is downsampled at multiple levels to generate image sequences at different resolutions, and then an independent discriminator sub-network is designed for each resolution level. Each sub-network consists of multiple layers of convolution, batch normalization, and nonlinear activation functions, extracting feature representations at that resolution level layer by layer. Notably, an attention mechanism can also be incorporated into the discriminator, enabling the network to automatically focus on visually more perceptually sensitive areas in the image, such as edges, textures, and high-frequency details. Through the multi-scale discrimination strategy, the global consistency and local detail quality of the image can be evaluated simultaneously, generating comprehensive and accurate quality assessment results.

[0147] Step S31 uses a multi-scale discriminator network to evaluate the quality of the image generated by the point cloud scene model rendering. The discriminator adopts a hierarchical structure design, which can simultaneously focus on the global structure and local details of the image. The discriminator extracts and distinguishes image features at different scales, and obtains the final quality evaluation score through weighted fusion.

[0148] S32. Based on the image quality assessment results, the image is progressively generated and thinned to obtain a high-resolution image.

[0149] Based on the quality evaluation results of the multi-scale discriminator network, progressive image generation and thinning are performed. A diffusion-based generation framework is adopted to model image generation as a stepwise denoising process from noise to a clear image. A U-Net generator network is constructed, including symmetrical encoder and decoder paths, as well as an intermediate feature transformation module. The generation process starts from low resolution, first outlining the general contour and main structure of the image, and then gradually increasing the resolution and adding detailed information. Ideally, an adaptive noise scheduling algorithm can dynamically adjust the denoising intensity and iteration steps according to the complexity of different image regions. More computing resources and more refined processing are allocated to regions with rich details, while more efficient processing methods are used for flat regions. In addition, the generator also integrates a residual attention module, which preserves high-frequency details of the original image through skip connections, while the attention mechanism enables the network to more accurately recover important structural features, ultimately generating a high-resolution image with rich details and clear structure.

[0150] Step S32, based on the image quality assessment results, designs a generator network based on a diffusion model to optimize the image. This generator employs a progressive generation strategy, generating a coarse outline of the image and gradually refining and optimizing the image details. During the generation process, an adaptive noise scheduling mechanism is introduced, dynamically adjusting the diffusion step size according to the image complexity of different regions. This mechanism ensures both generation quality and computational efficiency. Simultaneously, the generator network also integrates a residual attention module, which better preserves the texture details of the original image.

[0151] S33. Calculate the content loss, structural loss, and perceptual loss of the high-resolution image to obtain the optimized virtual teaching scene.

[0152] Quality optimization is achieved by calculating multiple perceptual losses. Step S33 employs three complementary loss functions: content loss, structural loss, and perceptual loss. Content loss is calculated based on pixel-level differences, typically using L1 or L2 norms to ensure consistency in basic content between the generated image and the reference image. Structural loss is based on the Structural Similarity Index (SSIM), focusing on evaluating the brightness, contrast, and structural characteristics of the image, which better aligns with the perceptual characteristics of the human visual system. Perceptual loss extracts high-level semantic features through pre-trained deep neural networks (such as VGG networks), calculating differences in the feature space to capture more abstract visual quality factors. Through a dynamic weight adjustment mechanism, the contributions of these three losses are automatically balanced according to the training process and image content. In the early stages of training, emphasis is placed on structural consistency; in the middle stages, attention is paid to content details; and in the later stages, perceptual quality is optimized. This multi-objective optimization strategy can generate visually superior virtual teaching scenarios while maintaining high computational efficiency, providing users with an immersive learning experience.

[0153] Step S33 optimizes the high-resolution image based on the multi-objective perception loss function. The loss function contains three key components: content loss ensures the semantic consistency between the generated image and the original image; structural loss ensures the accurate reconstruction of the geometric structure of the image; and perceptual loss uses a pre-trained feature extraction network to evaluate the visual realism of the generated image. Through the dynamic balance of these three loss terms, the computational cost can be controlled within an acceptable range while ensuring image quality, ultimately generating an optimized virtual teaching scene.

[0154] Combination Figure 6 As shown, step S4 includes:

[0155] S41. Deploy user authentication service and resource management service, configure user authentication protocol and resource access control policy, and build the basic service layer;

[0156] In deploying the user authentication service and resource management service, a microservice architecture was adopted, decoupling each functional module into an independent service unit. The user authentication service is responsible for handling security-related functions such as identity verification, authorization, and session management, supporting multiple authentication methods including account passwords, third-party OAuth authorization, and biometric recognition. This service implements JWT (JSON Web Token) technology, reducing server storage burden and improving response speed through stateless session design. The resource management service focuses on the storage, indexing, and access control of teaching content, adopting a layered design to separate metadata management from actual content storage, supporting large file chunked uploads and breakpoint resume functionality. It should be noted that a strict RBAC (Role-Based Access Control) policy is configured to assign differentiated resource access permissions to different user roles (such as teachers, students, and administrators), ensuring data security while providing flexible permission management. Through the deployment of these two basic services, a secure, reliable, and fully functional basic service layer is built, providing solid support for upper-layer applications.

[0157] Step S41 deploys user authentication service and resource management service based on microservice architecture; the user authentication service is responsible for user authentication and permission management, and supports multiple authentication methods, such as account password, single sign-on and third-party authorization; the resource management service is responsible for processing course resources and teaching content, and realizes resource uploading, storage, retrieval and version control; by configuring user authentication protocol and resource access control policy, a secure and reliable basic service layer is built.

[0158] S42. Based on the aforementioned basic service layer, configure real-time communication services, establish a WebRTC audio and video transmission channel, configure the transmission control protocol, and realize real-time data transmission;

[0159] Based on the basic service layer, a real-time communication service is configured to enable audio and video interaction between multiple users. The core of this service utilizes WebRTC technology, supporting peer-to-peer audio and video transmission between browsers without requiring additional plugins. A complete signaling server handles connection establishment processes such as session setup, ICE candidate collection, and media negotiation. To address connectivity issues in complex network environments, STUN and TURN server clusters are deployed, and NAT traversal technology ensures connection stability for users in different network environments. In terms of transmission control, an adaptive bitrate algorithm is implemented, dynamically adjusting the encoding parameters of audio and video streams based on real-time network conditions. Under bandwidth constraints, audio quality is prioritized to ensure the continuity of teaching and communication. Furthermore, the audio processing workflow is optimized according to the characteristics of teaching scenarios, integrating echo cancellation, noise suppression, and automatic gain control functions to provide a clear voice experience. Through professional audio and video transmission technology, low-latency, high-quality real-time data transmission is achieved, providing a smooth interactive experience for blended learning.

[0160] Step S42. Configure real-time communication services based on the basic service layer; establish a point-to-point audio and video transmission channel using WebRTC technology to support real-time voice and video communication between teachers and students; establish and maintain connections through a signaling server, and solve NAT traversal problems through a STUN / TURN server to ensure stable communication in different network environments; at the same time, configure an adaptive transmission control protocol to dynamically adjust audio and video encoding parameters according to network conditions to optimize transmission efficiency and quality.

[0161] S43. Employ the WebSocket protocol for instant messaging communication to enable real-time interaction among multiple users and build a hybrid teaching platform.

[0162] Unlike traditional HTTP polling, instant messaging using the WebSocket protocol establishes persistent connections, supporting bidirectional real-time communication between servers and clients, significantly reducing communication latency and server load. Based on this protocol, various instant messaging types are implemented, including text messages, system notifications, status synchronization, and file transfers. To handle large-scale concurrent connections, a cluster architecture and load balancing strategy are adopted, and a message queue middleware is introduced to buffer instantaneous peak traffic. Ideally, a publish-subscribe message distribution mechanism allows users to subscribe to messages on specific topics, reducing unnecessary message transmission. Regarding message reliability, message acknowledgment and retransmission mechanisms are implemented to ensure message delivery success rates even under unstable network conditions. Through the efficient application of the WebSocket protocol, low-latency real-time interaction between multiple users is achieved, meeting the needs of instant feedback and collaborative learning in teaching scenarios, ultimately constructing a fully functional and high-performance hybrid teaching platform.

[0163] Step S43 uses the WebSocket protocol to implement real-time messaging communication, supporting text messages, file sharing, and status synchronization between teachers and students. Compared with traditional HTTP polling, WebSocket provides a lower latency and more efficient bidirectional communication mechanism, which is particularly suitable for multi-user real-time interaction scenarios. Concurrent messages are handled through message queues and publish-subscribe patterns to ensure reliable message delivery and sequential consistency, ultimately building a fully functional hybrid teaching platform.

[0164] S44. Construct a scene tree manager to maintain the hierarchical relationship of objects in the optimized virtual teaching scene and establish a scene management framework; construct a unified scene management framework, the core of which includes three components: scene tree manager, state synchronizer, and real-time rendering engine. The scene tree manager organizes all objects in the scene based on a tree data structure, with each scene object as a node in the tree, expressing spatial inclusion and logical subordination relationships through parent-child relationships; the scene tree not only stores the geometric attributes (position, rotation, and scaling) of objects, but also maintains attribute information such as material, interactive behavior, and animation state; it implements an attribute inheritance mechanism, allowing child nodes to inherit specific attributes from their parent nodes, simplifying batch attribute modification operations; in addition, the scene tree supports event bubbling and capturing mechanisms, enabling interactive events to propagate orderly within the tree structure.

[0165] It achieves five-dimensional visualization functions: Building visualization: Based on point cloud models, it constructs building information models containing geometric information and semantic annotations, supporting dynamic loading and hiding / showing of different floors and areas; Equipment visualization: It integrates a 3D model library of teaching equipment to realize real-time status display and interactive control of equipment; Object visualization: It performs detailed modeling of teaching objects, supporting material mapping and physical property simulation; Operation visualization: It records and visualizes the user's operation sequence, supporting operation playback and demonstration teaching; Process visualization: It transforms the teaching process into a visual operation guide, providing step-by-step guidance and real-time feedback.

[0166] Based on a scenario management framework, a state synchronizer is implemented to solve the scenario consistency problem in a multi-user environment. The state synchronizer uses an operation transformation algorithm to abstract user operations into instructions, which are then distributed to all participants through a reliable message transmission mechanism. A conflict detection and resolution mechanism based on a logical clock handles the state inconsistency problem caused by concurrent operations. Each operation is assigned a unique timestamp and operation ID, and conflicts are resolved according to preset rules (such as time priority or specific user priority) to ensure that all clients eventually reach a consistent state. To improve synchronization efficiency, an incremental synchronization strategy is implemented, transmitting only the changed scenario parts to reduce network transmission volume. In addition, latency compensation technology is supported, providing a smooth user experience through local prediction and post-correction in high-latency network environments.

[0167] The system enables dynamic scene management capabilities, including: 1. An event-driven scene update mechanism: a complete event system is defined, including event types such as object creation, attribute modification, and interaction triggering, supporting custom event handlers; 2. An incremental synchronization algorithm: a binary difference algorithm is used to calculate scene state changes, transmitting only the changed parts to reduce bandwidth requirements; 3. A scene change history record system: all scene modification operations are recorded, supporting state rollback and operation undo / redo; 4. Supplementary explanation of the multi-scale discriminator network.

[0168] Building a scene tree manager is a crucial step in achieving 3D scene management. The manager uses a tree-like data structure to organize all objects in the scene, with each scene object acting as a node in the tree. Spatial inclusion and logical subordination are expressed through parent-child relationships. This hierarchical structure enables efficient execution of selection, grouping, and batch operations. The scene tree not only stores the geometric attributes of objects (such as position, rotation, and scaling) but also maintains attribute information such as materials, interactive behaviors, and animation states. Ideally, it implements an attribute inheritance mechanism, allowing child nodes to inherit specific attributes from their parent nodes, greatly simplifying batch attribute modification operations. Furthermore, the scene tree supports event bubbling and capturing mechanisms, enabling interactive events to propagate orderly within the tree structure, providing a unified processing framework for complex user interactions. Through this structured scene management approach, a clear and efficient scene management framework is established, laying the foundation for subsequent multi-user collaborative interaction.

[0169] Step S44 constructs a scene tree manager, which uses a tree structure to organize and manage various objects in the virtual teaching scene; each scene object is represented as a node in the tree, and the parent-child relationship between nodes reflects the hierarchical and containment relationships of the objects; the scene tree manager provides basic operations such as adding, deleting, modifying, and querying objects, supports object attribute inheritance and event bubbling mechanism, and provides a unified framework for scene management.

[0170] S45. A state synchronizer is implemented based on the aforementioned scene management framework to perform state synchronization processing on multi-user operations, ensuring scene state consistency; solving the core problem of scene consistency in a multi-user environment; the state synchronizer adopts an operation conversion algorithm to abstract user operations into operation instructions, which are distributed to all participants through a reliable message transmission mechanism; a conflict detection and resolution mechanism based on a logical clock can handle the state inconsistency problem that may be caused by concurrent operations; in the specific implementation, each operation is assigned a unique timestamp and operation ID, and conflicts are resolved according to preset rules (such as time priority or specific user priority) to ensure that all clients eventually reach a consistent state; to improve synchronization efficiency, an incremental synchronization strategy is adopted, transmitting only the changed scene parts, significantly reducing network transmission volume; in addition, latency compensation technology is supported, providing a smooth user experience through local prediction and post-correction in high-latency network environments; through the state synchronization mechanism, the state consistency of multiple users operating the virtual teaching scene simultaneously is ensured, providing a reliable guarantee for collaborative learning.

[0171] Step S45 implements a state synchronizer based on the scene management framework, which is responsible for maintaining the consistency of scene state in a multi-user environment. The state synchronizer adopts an operation-based synchronization strategy, serializes user operations into command objects, and distributes them to all participants through a reliable message transmission mechanism. In order to handle the conflicts that may be caused by concurrent operations, a timestamp-based conflict detection and resolution mechanism is implemented to ensure that all users eventually see the same scene state.

[0172] S46. Perform real-time rendering processing on the scene state to construct a hybrid teaching platform.

[0173] Step S46, based on the point cloud reconstruction results and image optimization results output from the previous steps, performs real-time rendering processing on the scene state; the rendering engine supports a variety of rendering technologies, such as physically based rendering, global illumination, and real-time shadows, which can generate high-quality visual effects; at the same time, an adaptive rendering strategy is implemented for the performance characteristics of different terminal devices, optimizing rendering performance while ensuring visual quality, and finally building a high-quality, high-performance hybrid teaching platform.

[0174] Step S46 utilizes WebGL / WebGPU rendering technology, supporting modern graphics rendering pipelines and advanced visual effects. The rendering engine implements physically based rendering (PBR), generating highly realistic visual effects by accurately simulating the physical interaction between light and materials. To optimize performance, various rendering optimization techniques are employed, including frustum culling, level of detail (LOD) control, and instantiated rendering. Specifically for VR / AR environments, adaptive rendering resolution technology is implemented, dynamically adjusting rendering quality based on device performance and visual requirements to provide the best visual experience while ensuring smooth frame rates. Notably, this disclosure also supports advanced lighting effects, such as global illumination, soft shadows, and ambient occlusion, further enhancing the visual realism of scenes. On different terminal devices, the rendering engine can automatically adapt to hardware capabilities, providing high-quality rendering on high-performance PCs and optimizing for a lightweight mode on mobile devices, ensuring a smooth interactive experience across various devices. Through real-time rendering technology, a hybrid teaching platform with excellent visual effects and a smooth interactive experience is constructed, providing users with an immersive learning environment.

[0175] Combination Figure 7 As shown, step S5 includes:

[0176] S51. Record user interaction data in the virtual teaching scenario to generate an operation behavior sequence; record user interaction data in the virtual teaching scenario; adopt a non-intrusive event capture mechanism to record various interactive behaviors throughout the learning process without the user's awareness; the captured data includes multi-dimensional information such as the user's movement trajectory, changes in gaze focus, hand operation actions, selection of interactive objects, and interaction duration; the raw interaction data is accurately timestamped and organized into a time-series data structure to form a complete operation behavior sequence; to ensure the comprehensiveness and accuracy of data collection, a multi-level event listening architecture is adopted to record everything from low-level hardware input (such as gamepad buttons, head display posture) to high-level semantic events (such as completing specific operation tasks); further implement an intelligent sampling strategy to appropriately downsample high-frequency repetitive operations while ensuring data integrity, which reduces storage pressure and retains the essential characteristics of behavioral patterns; through a refined interaction data acquisition mechanism, a digital behavioral profile reflecting the user's real learning behavior can be constructed, providing a rich and reliable raw data source for subsequent analysis.

[0177] Step S51 records all kinds of user interaction behaviors in the virtual teaching scenario, including movement trajectory, interaction objects, operation type and duration, etc.; the raw interaction data is organized into a time series to form a complete operation behavior record; a lightweight data collection mechanism is adopted to minimize the impact on the user operation experience, while ensuring the integrity and accuracy of the data.

[0178] S52. Collect user course completion and practice scores to obtain learning progress data. This data is used to evaluate learning effectiveness and establish a structured learning progress tracking framework, comprehensively recording various quantitative indicators of the user's learning process. Regarding course completion, it records the completion status of course modules, tracking in detail the learning time, number of repetitions, and time distribution characteristics of each knowledge point. For practice and assessment, it collects richer data indicators, including answer accuracy, problem-solving time, error type distribution, reference material usage, and number of attempts to correct errors. Furthermore, it implements knowledge point correlation analysis, establishing a mapping relationship between practice questions and specific knowledge points, thereby accurately assessing the mastery of each knowledge point. In addition, it records user learning rhythm characteristics, such as the distribution of learning time periods, the duration of a single learning session, and learning interval patterns—time-dimensional data that is valuable for understanding user learning habits and efficiency. Through a comprehensive learning progress data collection mechanism, a multi-dimensional quantitative representation of the user's learning status is established, laying a solid foundation for subsequent personalized learning analysis.

[0179] Step S52 collects information on the user's course learning, including indicators such as course completion progress, completion of practice questions, test scores, and learning time; it reflects the user's learning progress and effectiveness, supports the collection of data from various types of learning activities, and can comprehensively capture the user's learning journey.

[0180] S53. The operation behavior sequence and the learning progress data are preprocessed and standardized to extract key learning features for data analysis; data cleaning is performed, including removing invalid records, correcting outliers, and filling in missing data; anomaly detection uses a hybrid method based on statistics and machine learning to accurately identify data points that do not conform to the user's normal behavior pattern, such as records with abnormally short or excessively long operation times; the standardization process converts raw data of different types and scales into a unified representation, using a context-aware standardization algorithm that considers individual user differences and task difficulty factors to make the standardization results more comparable; further, feature engineering is performed to extract high-level learning features from the raw data; features include time efficiency indicators (such as task completion speed, response time distribution), accuracy indicators (such as error rate, correction efficiency), learning pattern indicators (such as knowledge point access order, review frequency), and interaction depth indicators (such as operation complexity, degree of exploration behavior), etc. Temporal feature extraction is achieved by capturing the dynamic features of user learning behavior over time through sliding window analysis and sequence pattern mining. Through the above data processing operations, the raw data is transformed into a structured learning feature set. The features accurately reflect the user's learning status and ability level, providing high-quality input data for deep learning models.

[0181] Step S53 involves cleaning and standardizing the collected raw data. Data cleaning includes removing invalid data, filling in missing values, and detecting outliers. Standardization transforms different types of data into a unified representation for easier subsequent analysis. Based on this, key learning features, such as learning speed, error rate, repetition patterns, and interaction depth, are extracted using feature engineering techniques to provide structured input for deep analysis.

[0182] Step S54. Input the key learning features into the deep learning model to generate a learning evaluation report. A multimodal deep learning architecture is adopted, capable of simultaneously processing structured feature data and temporal behavioral data, thereby comprehensively understanding the user's learning status. In model design, the advantages of feedforward neural networks, recurrent neural networks, and attention mechanisms are combined to form a composite network structure adapted to educational evaluation scenarios. The feedforward network processes static features, such as learning performance and ability indicators; the recurrent network analyzes temporal data, such as operational behavior sequences and changes in learning progress; the attention mechanism identifies key behaviors and important moments, extracting the most discriminative information. Model training employs transfer learning and multi-task learning strategies, pre-training the base model on a large-scale general learning behavior dataset. Fine-tuning the model on data from specific teaching scenarios ensures both its generalization ability and the assessment needs of those scenarios. Feature importance analysis and decision path visualization make the assessment results not only accurate but also transparent, allowing teachers and students to understand the rationale behind the assessment. The model output includes multi-dimensional assessment results, providing both descriptive analysis of the current learning status and predictive judgments of future learning trends, offering a scientific basis for learning planning. This highly personalized deep learning analysis process generates comprehensive, accurate, and practically valuable learning assessment reports, providing data-driven guidance for teaching optimization and learning improvement.

[0183] In a preferred embodiment of this disclosure, step S54 specifically includes:

[0184] S541. Construct a learner profile model based on the key learning features and generate learning feature vectors.

[0185] A multi-layered profile building framework is designed, starting from the original learned features and extracting abstract learner representations through layer-by-layer transformations of deep neural networks. In terms of network structure, an encoder-decoder architecture is adopted. The encoder consists of multiple fully connected layers and non-linear activation functions, extracting complex relationships between features layer by layer. The decoder performs self-supervised learning by reconstructing the original features, enhancing the model's understanding of the essence of the features. To capture the interactions between features, a self-attention mechanism is introduced, enabling the model to identify important correlations between features, such as the relationship between learning time and accuracy, and the balance between operation speed and accuracy. A contrastive learning framework is implemented by constructing... Positive and negative sample pairs guide the model to learn key features that distinguish different learning styles and ability levels. The output of the profiling model is a high-dimensional learning feature vector, which encodes learners' cognitive characteristics, learning preferences, ability levels, and behavioral patterns in the semantic space. Through visualization technology, these abstract features can be mapped to two-dimensional or three-dimensional space to form an intuitive learner profile map, helping teachers quickly understand individual differences among students. This disclosure presents a deep learning-based learner profile construction method that breaks through the limitations of traditional assessment models, can capture more subtle and personalized learning features, and provides a solid foundation for subsequent multi-dimensional assessment.

[0186] Step S541 constructs a learner profile model based on the extracted key learning features. This model employs a deep neural network structure, which can capture the complex relationships and implicit patterns between learning features. By inputting learning features into the model, a high-dimensional feature vector describing the learner's characteristics is generated, encoding information such as the learner's cognitive style, learning habits, and ability level.

[0187] S542. Utilize the deep learning model to analyze the learning feature vectors for skill mastery, knowledge comprehension, and learning engagement, and obtain multi-dimensional evaluation results.

[0188] The evaluation model architecture employs a refined analysis across three core dimensions: skill mastery, knowledge comprehension, and learning engagement. For skill mastery assessment, the model focuses on the accuracy, fluency, and consistency of operations. By comparing user operation sequences with standard operating procedures, it calculates operation similarity and proficiency indices. A sequence alignment algorithm handles the variability in operation order, identifying equivalent but not entirely identical operation paths and providing a fairer skill assessment. For knowledge comprehension assessment, the model, based on a knowledge graph structure, analyzes user performance on different knowledge points and the ability to transfer knowledge between them. Bayesian knowledge tracing technology estimates the mastery probability of each knowledge point and identifies conceptual confusion and comprehension barriers. Furthermore, the model can distinguish between rote memorization and deep understanding, assessing the degree of knowledge internalization by analyzing user performance in modified questions and application scenarios. For learning engagement assessment, quantitative indicators such as learning duration and interaction frequency are considered, and behavioral patterns are analyzed to identify focused states and inattentive behaviors. We introduce an engagement assessment method based on attention distribution, and identify the behavioral characteristics of deep engagement and superficial coping through eye tracking and interaction pattern analysis. Through comprehensive analysis of the three dimensions, we generate comprehensive and detailed multi-dimensional assessment results, providing learners with accurate ability diagnosis and learning status feedback.

[0189] Step S542 utilizes a deep learning model to perform multi-dimensional analysis of the learning feature vectors. Regarding skill mastery assessment, the model evaluates learners' practical skill levels based on indicators such as operational accuracy and completion time. For knowledge comprehension assessment, the model assesses learners' understanding of knowledge points by analyzing practice answers and test results. In terms of learning engagement assessment, the model evaluates learners' participation and focus by analyzing data such as learning duration and interaction frequency. These analytical results collectively constitute a comprehensive multi-dimensional assessment.

[0190] Step S543. Generate a learning assessment report based on the multi-dimensional assessment results. Natural language generation technology is used to transform the quantitative analysis results into structured text descriptions and visual charts, forming a complete assessment report that is easy to understand and has guiding value. The report generation employs a combination of template-based and deep generation methods, constructing a document structure based on a preset report framework, including core parts such as overall evaluation, strengths analysis, weaknesses identification, and improvement suggestions. A conditional text generation model is used to generate natural and fluent descriptive text based on specific assessment data. Context-aware language generation is implemented, enabling adjustments to the expression and suggestions based on the learner's specific situation, avoiding overly general or inapplicable evaluations. In terms of visualization design, the report offers various data display formats, including ability radar charts, knowledge point mastery heatmaps, learning progress timelines, and peer comparison analysis. These visualization components intuitively demonstrate learners' performance and progress across different dimensions. The report also includes personalized learning resource recommendations and improvement strategies. These suggestions are based on assessment results and learning pattern analysis, and are customized to the specific needs and learning styles of users. Special emphasis is placed on the practicality and operability of the report, ensuring that each suggestion has clear action guidelines and expected results. Through a comprehensive report combining quantitative analysis and qualitative evaluation, the report provides learners with scientifically accurate and practically effective learning feedback, helping them better understand their own situation and develop targeted learning plans.

[0191] Step S543 generates a personalized learning assessment report based on the multi-dimensional assessment results. The report includes the learner's overall performance evaluation, strengths and weaknesses analysis, learning suggestions and improvement directions. Natural language generation technology is used to transform the quantitative analysis results into easy-to-understand text descriptions, and key data and trends are displayed with visualization charts. The assessment report can be used for learners to reflect on themselves, and can also help teachers understand students' situations, adjust teaching strategies, and form a closed-loop teaching improvement mechanism.

[0192] like Figure 8 As shown, the hybrid teaching system provided in this disclosure includes: a point cloud data preprocessing module 100, a scene modeling module 200, an image optimization module 300, a teaching platform module 400, and a data analysis module 500.

[0193] The system includes the following modules: a point cloud data preprocessing module 100, which acquires point cloud data of the teaching environment, performs noise removal and density equalization on the point cloud data to obtain preprocessed point cloud data; a scene modeling module 200, which inputs the preprocessed point cloud data into a decoupled sparse prior guided diffusion compression model, extracts local and global features of the point cloud through a spatial feature encoder, and generates a point cloud scene model; an image optimization module 300, which performs multi-scale discrimination and generation processing on the point cloud scene model, calculates content loss, structural loss, and perceptual loss to optimize image quality, and obtains an optimized virtual teaching scene; a teaching platform module 400, which configures a microservice architecture and deploys a teaching management system, realizes real-time multi-user interaction through WebRTC and WebSocket, and builds a hybrid teaching platform; and a data analysis module 500, which collects user interaction behavior data and learning progress data in the virtual teaching scene, analyzes the interaction behavior data and learning progress data based on deep learning algorithms, and generates a learning evaluation report.

[0194] In summary, the hybrid teaching method and system provided in this disclosure significantly improve the efficiency and quality of virtual scene reconstruction by decoupling the sparse prior guided diffusion compression model from point clouds and extracting local and global features from the point clouds using a spatial feature encoder. The image super-resolution adversarial diffusion compression method for VR / AR scenes improves display effects while maintaining performance through multi-scale discrimination and perceptual loss optimization. The distributed teaching platform based on a microservice architecture supports large-scale concurrent access and real-time interaction through WebRTC and WebSocket technologies, providing technical support for hybrid teaching.

[0195] In practical applications, the hybrid teaching device provided in this disclosure can be used in conjunction with standard computer hardware, such as servers, network devices, VR / AR headsets, and mobile terminals. The various modules within the device can be implemented in software, dedicated hardware, or a combination of both.

[0196] Based on the same inventive concept, corresponding to any of the above-described embodiments, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the hybrid teaching method described in any of the above embodiments.

[0197] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of a hybrid teaching method described in the above-described method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0198] The above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and are not intended to limit it. The scope of protection of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A blended learning method, characterized in that, The method comprises the following steps: Obtain point cloud data of a teaching environment, remove noise and balance density of the point cloud data, and obtain preprocessed point cloud data; Perform multi-layer perception processing on the preprocessed point cloud data to extract local features of the point cloud; Perform graph convolution network processing based on the local features to extract global features of the point cloud; Input the local features and the global features into a conditional generator, adjust the feature generation process based on diffusion time steps, and obtain a time feature vector; Perform variational inference processing on the time feature vector to obtain a sparse representation of the point cloud data; Reconstruct features based on the sparse representation to generate a point cloud scene model; Perform multi-scale discrimination and generation processing on the point cloud scene model, calculate content loss, structure loss, and perception loss to optimize image quality, and obtain an optimized virtual teaching scene; Configure a micro-service architecture and deploy a teaching management system, realize multi-user real-time interaction through WebRTC and WebSocket, and build a hybrid teaching platform; Collect interaction behavior data and learning progress data of users in the virtual teaching scene, analyze the interaction behavior data and the learning progress data based on a deep learning algorithm, and generate a learning evaluation report.

2. The blended method of instruction of claim 1, wherein, Obtain point cloud data of a teaching environment, remove noise and balance density of the point cloud data, and obtain preprocessed point cloud data, comprising: Obtain point cloud data of a teaching environment, use multi-station scanning and automatic registration processing to generate original point cloud data; Identify noise points of the original point cloud data, filter abnormal points based on a density threshold, and obtain filtered point cloud data; Divide the filtered point cloud data into grids, calculate the point cloud density in each grid, and obtain preprocessed point cloud data.

3. The blended method of teaching of claim 1, wherein, Perform multi-scale discrimination and generation processing on the point cloud scene model, calculate content loss, structure loss, and perception loss to optimize image quality, and obtain an optimized virtual teaching scene, comprising: Input the point cloud scene model into a multi-scale discriminator network to extract and evaluate image features at different scales, and obtain image quality evaluation results; Based on the image quality evaluation results, perform progressive generation and refinement processing on the image to obtain a high-resolution image; Calculate the content loss, structure loss, and perception loss of the high-resolution image to obtain an optimized virtual teaching scene.

4. The blended method of instruction of claim 1, wherein, Configure a micro-service architecture and deploy a teaching management system, realize multi-user real-time interaction through WebRTC and WebSocket, and build a hybrid teaching platform, comprising: Deploy user authentication services and resource management services, configure user identity verification protocols and resource access control strategies, and build a basic service layer; Based on the basic service layer, configure real-time communication services, establish a WebRTC audio and video transmission channel, configure a transmission control protocol, and realize real-time data transmission; Use the WebSocket protocol for instant messaging to realize multi-user real-time interaction and build a hybrid teaching platform; Build a scene tree manager to maintain the hierarchical relationship of objects in the optimized virtual teaching scene and establish a scene management framework; A state synchronizer is implemented based on the scene management framework to perform state synchronization processing on multi-user operations and ensure consistency of the scene state; The scene state is rendered in real time to build a hybrid teaching platform.

5. The blended method of instruction of claim 1, wherein, Interaction behavior data and learning progress data of the user in the virtual teaching scene are collected, and the interaction behavior data and learning progress data are analyzed based on a deep learning algorithm to generate a learning evaluation report, including: Interaction operation data of the user in the virtual teaching scene is recorded to generate an operation behavior sequence; Course completion and practice scores of the user are collected to obtain learning progress data; The operation behavior sequence and the learning progress data are preprocessed and standardized to extract key learning features; The key learning features are input to a deep learning model to generate a learning evaluation report.

6. The blended method of instruction of claim 5, wherein, The key learning features are input to a deep learning model to generate a learning evaluation report, including: A learner portrait model is constructed based on the key learning features to generate a learning feature vector; The deep learning model is used to analyze the learning feature vector in terms of skill mastery, knowledge understanding, and learning input to obtain multi-dimensional evaluation results; A learning evaluation report is generated based on the multi-dimensional evaluation results.

7. A hybrid teaching system characterized by, It includes: A point cloud data preprocessing module is used to obtain point cloud data of a teaching environment, remove noise from the point cloud data, and perform density equalization processing to obtain preprocessed point cloud data; A scene modeling module is used to perform multi-layer perception processing on the preprocessed point cloud data to extract local features of the point cloud, perform graph convolution network processing based on the local features to extract global features of the point cloud, input the local features and the global features to a conditional generator, adjust the feature generation process based on diffusion time steps to obtain a time feature vector, and perform variational inference processing on the time feature vector to obtain a sparse representation of the point cloud data; Based on the sparse representation, feature reconstruction is performed to generate a point cloud scene model; An image optimization module is used to perform multi-scale discrimination and generation processing on the point cloud scene model, calculate content loss, structure loss, and perception loss to optimize image quality, and obtain an optimized virtual teaching scene; A teaching platform module is used to configure a microservice architecture and deploy a teaching management system, realize multi-user real-time interaction through WebRTC and WebSocket, and build a hybrid teaching platform; 8. An electronic device, comprising: A data analysis module is used to collect interaction behavior data and learning progress data of the user in the virtual teaching scene, analyze the interaction behavior data and learning progress data based on a deep learning algorithm, and generate a learning evaluation report. It includes a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the hybrid teaching method of any one of claims 1 to 6.

Images