Stator guider assembling process monitoring and guiding method based on action sequence

Abstract

The invention discloses a stator guider assembly process detecting and guiding method based on an action sequence, which belongs to the technical field of assembly assistance and comprises an action recognition module, a data acquisition module, a comparison judgment module and a process guiding module. The action recognition module adopts an ST-GCN algorithm to perform action recognition; the gesture data acquisition module adopts a Leap Motion somatosensory controller to acquire assembly gesture 3D skeleton point data; the comparison and judgment module is used for forming an action set by the data obtained by ST-GCN classification within a period of time, performing cross comparison on the action set and a preset process action to obtain which step the current assembly is in, and judging whether the current assembly is correct or not; and the process guiding module is used for pushing the next assembly process action through AR (Augmented Reality). Through the assembly process monitoring and guiding method, the assembly action can be accurately identified, and the correctness of the assembly steps can be judged in real time, so that the assembly efficiency and accuracy are improved, and the manual training cost is reduced.

Description

Technical Field

[0001] This invention relates to the field of assembly assistance technology, specifically to a method for monitoring and guiding the assembly process of a stator guide based on action sequences. Background Technology

[0002] Traditional assembly processes rely on workers' experience and memory, which is prone to errors, especially in the assembly of complex products such as aerospace equipment, leading to low efficiency and inconsistent quality. While existing technologies attempt to assist assembly through sensors or visual inspection, they suffer from the following problems: Insufficient motion recognition accuracy: Existing algorithms struggle to accurately analyze the spatiotemporal characteristics of hand movements, resulting in misjudgments of assembly steps; Poor real-time performance: Some systems rely on fixed perspectives or offline processing, failing to achieve real-time monitoring and guidance of assembly actions; Unintuitive interaction: Traditional guidance methods require workers to frequently consult design documents or assembly manuals, constantly shifting their attention and increasing cognitive load.

[0003] The assembly process of aircraft engines is complex, with numerous parts to be assembled. Assembly guidance mainly relies on workers' experience, which places high demands on workers and increases training costs. At the same time, the increased training time also reduces assembly efficiency and work progress.

[0004] Therefore, in order to improve the assembly efficiency and accuracy of aero-engine stator guides, there is an urgent need for an assembly process monitoring and guidance method to achieve accurate identification of assembly actions, step verification and dynamic guidance, thereby improving assembly efficiency and accuracy and reducing labor costs. Summary of the Invention

[0005] To overcome the above-mentioned shortcomings, this invention provides a method for monitoring and guiding the assembly process of a stator guide based on action sequences. By integrating the ST-GCN action recognition algorithm, Leap Motion high-precision gesture tracking, and AR real-time guidance technology, it achieves accurate identification, step verification, and dynamic guidance of the assembly actions of the stator guide, thereby improving assembly efficiency and accuracy and reducing labor costs.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for monitoring and guiding the assembly process of a stator guide based on action sequences includes the following steps: Step 1) Building a hand data acquisition module to achieve real-time acquisition of 3D hand skeletal point data: First, install the Leap Motion motion sensor, which is the key hardware foundation for the entire acquisition process and can accurately sense hand movements. Second, fully consider the characteristics of the human hand skeleton and rigorously define the hand skeleton data model to ensure data accuracy. Finally, for various movements during the assembly process, meticulously identify hand gestures at a certain frequency, integrate the identified information, and thus form a rich and detailed hand skeletal point dataset, providing strong support for subsequent related research.

[0007] Step 2) Use the ST-GCN algorithm to perform spatiotemporal feature analysis on the skeletal point data and identify the assembly action category: First, the skeletal point data from the Leap Motion motion sensor is preprocessed to ensure that it matches the input format of the ST-GCN algorithm; second, the ST-GCN algorithm is used to process and identify the data to determine the current gesture; finally, the identification results are grouped according to a certain time interval to form an action set, which is called the assembly action.

[0008] Step 3) Match the identification results with the predefined process action library to determine the correctness of the current step: First, compare the assembly action obtained by the ST-GCN algorithm with the predefined actions. Finally, determine the correctness of the current action. If the action is incorrect, an alarm will be issued. If it is correct, the next assembly action will be pushed. Step 4) Push the next assembly action guidance or error prompt through the AR device: Display the recognition result through AR. If the assembly action is incorrect, issue an error warning and display it to the assembly personnel. If the assembly action is correct, push the next assembly action for the current part to the assembly personnel.

[0009] Step 1) specifically refers to: Step 1.1) Install the Leap Motion motion-sensing interaction device at the assembly station, such as... Figure 2 As shown, its hardware configuration includes dual infrared cameras (200Hz frame rate), four infrared LED light sources and a filter layer to ensure accurate capture of hand movements in complex lighting environments, providing accurate data for subsequent data processing; Step 1.2) Define the hand skeletal data model: Based on the anatomical features of the human hand, construct a 3D model containing 21 skeletal points (including 4 phalanges each from the thumb to the little finger and the wrist joint), such as... Figure 3 As shown, ensure accurate identification of assembly workers' assembly gestures; Step 1.3) Collect the coordinates, velocity and rotation angle data of the hand bones in real time at a frequency of 200Hz, extract the three-dimensional position of the hand, and establish a three-dimensional model of the hand; use a grayscale camera to reduce the amount of computational data and improve the algorithm speed; then transmit the collected bone point data to the ST-GCN algorithm module through wired / wireless communication. Step 2) specifically refers to: Step 2.1) The ST-GCN algorithm takes the joint coordinate vectors on the graph nodes as input. Therefore, the video frames of the skeletal point data obtained by LeapMotion need to be put into the OpenPose processor for pose estimation to obtain the features of each skeletal key point of the human body. The OpenPose algorithm performs post-processing on the estimated pose, including pose adjustment, pose smoothing and other operations, and finally converts the continuous frame skeletal point data into a spatiotemporal graph structure. Step 2.2) Data Processing: A learnable weight matrix (of the same size as the adjacency matrix) is introduced and multiplied bitwise with the adjacency matrix to assign greater weights to important edges (nodes) and suppress the weights of unimportant edges (nodes). Then, the weighted adjacency matrix and the input are fed into the GCN for computation. Simultaneously, a residual structure is introduced to calculate Res, which is added bitwise with the GCN output to aggregate spatial dimension information. Finally, the TCN network is used to aggregate temporal dimension information. Step 2.3) Action Classification and Aggregation: Based on the calculation results output by the ST-GCN algorithm, identify the action category of a single frame in real time, and cluster a continuous segment of frame data to define it as an assembly process action.

[0010] Step 3) specifically refers to: Step 3.1) Compare an assembly process action with the pre-set assembly action of the current step. By comparing the spatial position coordinates, posture and gesture of the current action, it is determined whether it meets the expected assembly action. Step 3.2) If the comparison result shows that the action meets the expected requirements, then push the next assembly action to the assembly worker and output the result to the AR device for display; if the comparison result shows that the action does not meet the expected requirements, then output the action error and output the result to the AR device for display. Step 4) is specifically as follows: Step 4.1) After the assembly worker puts on the AR device, the comparison result is received. If the cross-comparison result shows that the action is correct, the next assembly action is pushed to the AR device and displayed to the assembly worker; if the cross-comparison result shows that the action is incorrect, the action error is displayed and an alert is issued to prompt the assembly worker to reassemble or make adjustments.

[0011] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention utilizes the ST-GCN algorithm, which can accurately identify assembly actions and improve assembly accuracy. The algorithm meticulously compares the collected actual assembly actions with predefined standard process actions. Once a difference is found, errors in the assembly process can be detected in a timely manner. This effectively avoids product quality defects caused by human negligence, greatly improves assembly accuracy, ensures product quality, reduces additional costs such as subsequent rework, and makes the production process smoother and more efficient.

[0012] (2) By leveraging AR technology, this invention enables real-time push notifications for the next assembly process step, thereby improving assembly efficiency. Through augmented reality equipment, workers can intuitively obtain operational guidance without having to expend effort memorizing tedious assembly steps. This guidance method allows workers to complete each assembly step quickly and accurately, reducing time wasted due to operational confusion or misunderstanding of the assembly sequence, significantly improving assembly efficiency, accelerating production progress, and avoiding waste of production time costs. Attached Figure Description

[0013] Figure 1 This is a flowchart of an embodiment of the present invention.

[0014] Figure 2 This is a schematic diagram of the LeapMotion motion-sensing interactive device, where 1 is a high frame rate binocular camera system, 2 is a filter layer, and 3 is an infrared LED.

[0015] Figure 3 Define the skeletal points of the hand in the diagram. Detailed Implementation

[0016] The present invention will now be described in detail with reference to embodiments and accompanying drawings. The present invention can be implemented in many different forms and should not be considered limited to the embodiments described herein. Rather, these embodiments are provided so that the present invention will be thoroughly and completely disclosed and will fully express the scope of the invention to those skilled in the art.

[0017] like Figure 1 As shown, a method for monitoring and guiding the assembly process of a stator guide based on action sequences includes the following steps: Step 1) Building a hand data acquisition module to achieve real-time acquisition of hand 3D skeletal point data: First, install the Leap Motion motion sensor, which is the key hardware foundation for the entire acquisition process and can accurately sense hand movements. Second, fully consider the characteristics of the human hand skeleton and rigorously define the hand skeleton data model to ensure data accuracy. Finally, for various movements during the assembly process, meticulously identify hand gestures at a certain frequency, integrate the identified information, and thus form a rich and detailed hand skeleton point dataset, providing strong support for subsequent related research; specifically: Step 1.1) Install the Leap Motion motion sensor at the assembly station; the Leap Motion motion sensor uses optical gesture tracking technology, has two high frame rate grayscale infrared cameras with wide-angle lenses and four infrared LEDs. Its optimal working environment is a lighting environment that produces clear, high-contrast object outlines, such as... Figure 1 As shown, the top filter layer only allows infrared light waves to enter and exit, performing preliminary processing on the data collected by the camera and simplifying the complexity of subsequent calculations; a binocular camera is used to extract the three-dimensional position of the hand through the principle of binocular stereo vision imaging, and a three-dimensional model of the hand is established; a grayscale camera is used to reduce the amount of computational data and improve the algorithm speed.

[0018] Step 1.2) Define the hand skeleton data model: Based on the human hand gesture characteristics during the assembly of the aero-engine stator guide, collect assembly gesture information and construct a 3D model containing 21 skeletal points (including four phalanges each from the thumb to the little finger and the wrist joint), as shown in the figure. Figure 2 To ensure accurate identification of assembly workers' assembly gestures; Step 1.3) Collect the coordinates, velocity and rotation angle data of the hand bones in real time at a frequency of 200Hz, extract the three-dimensional position of the hand, and establish a three-dimensional model of the hand; use a grayscale camera to reduce the amount of computational data and improve the algorithm speed, and then transmit the collected bone point data to the ST-GCN algorithm module through wired / wireless communication. Step 2) Utilizing the ST-GCN algorithm to perform spatiotemporal feature analysis on the skeletal point data and identify the assembly action category: First, the skeletal point data from the Leap Motion motion sensor is preprocessed to ensure it matches the input format of the ST-GCN algorithm; second, the ST-GCN algorithm is used to process and identify the data, determining the current gesture; finally, the identification results are grouped according to a certain time interval to form an action set, referred to as the assembly action; specifically: Step 2.1) Data Preprocessing: The ST-GCN algorithm takes joint coordinate vectors on graph nodes as input. Therefore, the skeletal point data video frames acquired by Leap Motion need to be fed into the OpenPose processor for pose estimation to obtain the features of each skeletal keypoint of the human body. The process is as follows: First, preprocessing is performed, including image scaling, color space conversion, edge detection, etc. Then, human detection is performed using a deep learning model, followed by pose estimation. The OpenPose algorithm estimates the pose of each human movement by processing the keypoints. Finally, the OpenPose algorithm performs post-processing on the estimated pose, including pose adjustment, pose smoothing, etc. Ultimately, the continuous frame skeletal point data is converted into a spatiotemporal graph structure, where nodes represent skeletal points and edges represent spatiotemporal relationships. The output format is as follows: (N,C,T,V,M) (1) Where: N represents the number of videos, C represents joint features, T represents the number of keyframes, V represents the number of joints, and M represents the number of people in a frame. Step 2.2) Data Processing: First, the input matrix from Step 2.1) is normalized to normalize the positional features of a joint across different frames. Then, a learnable weight matrix (of the same size as the adjacency matrix) is introduced and multiplied bitwise with the adjacency matrix. This assigns greater weights to important edges (nodes) in the adjacency matrix and suppresses the weights of unimportant edges (nodes). Finally, the weighted adjacency matrix and the input are fed into the GCN for computation. Simultaneously, a residual structure is introduced to calculate Res, which is then added bitwise with the GCN output to aggregate spatial dimension information. After completing the above operations, the TCN network is used to aggregate temporal dimension information. Step 2.3) Action Classification and Aggregation: Based on the calculation results output by the ST-GCN algorithm, identify the action category of a single frame in real time, and cluster a continuous segment of frame data to define it as an assembly process action.

[0019] Step 3) Match the identification results with the predefined process action library to determine the correctness of the current step: First, compare the assembly actions obtained by the ST-GCN algorithm with the predefined actions. Finally, determine the correctness of the current action. If the action is incorrect, an alarm is issued; if it is correct, the next assembly action is pushed. Specifically: Step 3.1) Compare an assembly process action with the pre-set assembly action of the current step. By comparing the spatial position coordinates, posture and gesture of the current action, it is determined whether it meets the expected assembly action. Step 3.2) If the comparison result shows that the action meets the expected requirements, then push the next assembly action to the assembly worker and output the result to the AR device for display; if the comparison result shows that the action does not meet the expected requirements, then output the action error and output the result to the AR device for display. Step 4) Push the next assembly action guidance or error prompt via AR device: Display the recognition result via AR. If the assembly action is incorrect, issue an error warning and display it to the assembler. If the assembly action is correct, push the next assembly action for the current part to the assembler; specifically: Step 4.1) After the assembly worker puts on the AR device, the comparison result is received. If the cross-comparison result shows that the action is correct, the next assembly action is pushed to the AR device and displayed to the assembly worker; if the cross-comparison result shows that the action is incorrect, the action error is displayed and an alert is issued to prompt the assembly worker to reassemble or make adjustments.

[0020] Reference Figure 3 The diagram shows the definition of skeletal points in the hand, dividing the entire hand into 25 skeletal points: 1 is the tip of the thumb, 2 is the distal end of the thumb, 3 is the proximal end of the thumb, 4 is the metacarpal bone of the thumb, 5 is the wrist, 6 is the tip of the index finger, 7 is the distal end of the index finger, 8 is the proximal end of the index finger, 9 is the index finger joint, 10 is the index finger metacarpal bone, 11 is the tip of the middle finger, 12 is the distal end of the middle finger, 13 is the proximal end of the middle finger, 14 is the middle finger joint, 15 is the middle finger metacarpal bone, 16 is the tip of the ring finger, 17 is the distal end of the ring finger, 18 is the proximal end of the ring finger, 19 is the ring finger joint, 20 is the ring finger metacarpal bone, 21 is the tip of the little finger, 22 is the distal end of the little finger, 23 is the proximal end of the little finger, 24 is the little finger joint, and 25 is the little finger metacarpal bone.

[0021] In summary, this invention proposes a method for monitoring and guiding the assembly process of a stator guide based on action sequences. First, it proposes to use a Leap Motion motion sensor to capture assembly gestures. Second, it identifies the types of assembly gestures based on the ST-GCN algorithm. Then, it forms a set of assembly actions based on the known gestures. Finally, it compares the identified assembly actions with the pre-set actions and pushes the results to an AR device, which then delivers them to the assembly worker, achieving efficient and accurate assembly.

[0022] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0023] It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for monitoring and guiding the assembly process of a stator guide based on action sequences, characterized in that, Includes the following steps: Step 1) Build a hand data acquisition module to achieve real-time acquisition of hand 3D skeleton point data: First, install the LeapMotion motion sensor; second, define the hand skeleton data model to ensure the accuracy of the data; finally, for various actions during the assembly process, carefully identify hand gestures and integrate the identified information. Step 2) Use the ST-GCN algorithm to perform spatiotemporal feature analysis on the skeletal point data and identify the assembly action category: First, preprocess the skeletal point data from the Leap Motion somatosensory interactive device to ensure that it matches the input format of the ST-GCN algorithm; Secondly, the ST-GCN algorithm is used to process and recognize the data to determine the current gesture; finally, the recognition results are grouped according to a certain time interval to form a set of actions, which is called assembly action. Step 3) Match the identification results with the predefined process action library to determine the correctness of the current step: First, compare the assembly action obtained by the ST-GCN algorithm with the predefined actions. Finally, determine the correctness of the current action. If the action is incorrect, an alarm will be issued. If it is correct, the next assembly action will be pushed. Step 4) Push the next assembly action guide or error prompt through AR device: Display the recognition result through AR. If the assembly action is incorrect, issue an error warning and display it to the assembly personnel. If the assembly action is correct, push the next assembly action for the current part to the assembly personnel.

2. The method according to claim 1, characterized in that, Step 1) Specifically: Step 1.1) Install the Leap Motion motion sensor at the assembly station. Its hardware configuration includes dual infrared cameras with a frame rate of 200Hz, four infrared LED light sources and a filter layer to ensure accurate capture of hand movements in complex lighting environments and provide accurate data for subsequent data processing. Step 1.2) Define the hand skeleton data model: Based on the anatomical features of the human hand, construct a 3D model containing 21 skeletal points, including 4 phalanges from the thumb to the little finger and the wrist joint, to ensure accurate recognition of assembly workers' assembly gestures; Step 1.3) Collect the coordinates, velocity and rotation angle data of the hand bones in real time at a frequency of 200Hz, extract the three-dimensional position of the hand, and establish a three-dimensional model of the hand; use a grayscale camera to reduce the amount of computational data and improve the algorithm speed; then transmit the collected bone point data to the ST-GCN algorithm module through wired / wireless communication.

3. The method according to claim 1, characterized in that, Step 2) specifically involves: Step 2.1) Data preprocessing: The ST-GCN algorithm takes the joint coordinate vectors on the graph nodes as input. Therefore, the bone point data obtained by leap-motion needs to be processed through multiple spatiotemporal convolution operations to generate higher-level feature maps on the graph, and the continuous frame bone point data is converted into a spatiotemporal graph structure, where nodes represent bone points and edges represent spatiotemporal correlations. Step 2.2) Data Processing: A learnable weight matrix of the same size as the adjacency matrix is ​​introduced and multiplied bitwise by the adjacency matrix. This weight matrix is ​​called "Learnable edge importance weight" and is used to assign greater weights to important edge nodes in the adjacency matrix and suppress the weights of unimportant edge nodes. Then, the weighted adjacency matrix and the input are fed into the GCN for computation. At the same time, a residual structure is introduced to calculate Res and add it bitwise by the output of the GCN to realize the aggregation of spatial dimension information. Finally, the TCN network is used to realize the aggregation of temporal dimension information. Step 2.3) Action Classification and Aggregation: Based on the calculation results output by the ST-GCN algorithm, identify the action category of a single frame in real time, and cluster a continuous segment of frame data to define it as an assembly process action.

4. The method according to claim 1, characterized in that, The preprocessing of the skeletal point data of the Leap Motion motion sensor is as follows: To address issues such as hand occlusion and sensor recognition errors that may result in missing data or coordinate jumps in skeletal points in some frames, a weighted average interpolation method based on adjacent valid frames is used to estimate missing points. For outliers that significantly deviate from the normal range, corrections or removals are made based on the trends of preceding and following frames to ensure data continuity. Subsequently, the skeletal point data acquired by LeapMotion is fed into the OpenPose processor for pose estimation to obtain the features of each skeletal keypoint of the human body. The OpenPose algorithm performs post-processing on the estimated pose, including pose adjustment and pose smoothing operations. Finally, the continuous frame skeletal point data is converted into a spatiotemporal graph structure, where nodes represent skeletal points and edges represent spatiotemporal correlations.

5. The method according to claim 1, characterized in that, The specific steps for processing and identifying data using the ST-GCN algorithm are as follows: First, a "Learnable edge importance weight" matrix is ​​introduced. This learnable weight matrix has the same dimension as the adjacency matrix. By multiplying the elements bitwise, it can autonomously learn the importance of each connection during training: for key edges that determine the core joint connections of the gesture category, the matrix will assign a larger weight to strengthen their influence on feature extraction; for unimportant edges, noise, or minor connections, their interference will be suppressed with a smaller weight. This design allows GCN to no longer be limited by preset node relationships, but to dynamically adjust its focus according to data characteristics, thereby improving the targeting of spatial features. The weighted adjacency matrix and the input features are then fed into the GCN for computation. The core function of the GCN is to capture the spatial relationships in the graph structure by aggregating the features of the nodes themselves and the features of the neighboring nodes. At the same time, the residual structure (Res) introduced alleviates the feature degradation problem in deep networks: the residual term is obtained by performing a simple linear transformation on the input features. After being added bit by bit to the output of the GCN, it can not only retain the basic information in the original input, but also superimpose the high-order spatial features learned by the GCN, making the aggregation of spatial dimension information more stable, avoiding the loss of effective features due to the stacking of multiple layers of networks, and accelerating gradient propagation during model training. Finally, the TCN temporal convolutional network is used to process the dynamic features in the time dimension. Through causal convolution and dilated convolution, the correlation information in the time dimension can be effectively extracted, the spatial aggregation features of different time steps are fused, and the complete dynamic process of the gesture from start to finish is captured, realizing information aggregation in the time dimension.

6. The method according to claim 1, characterized in that, Action classification and aggregation are as follows: For the single-frame action recognition results output by ST-GCN, real-time classification is achieved through lightweight processing and sliding window majority voting to reduce instantaneous noise interference. For continuous frame sequences, action label mutation and skeletal motion speed detection boundaries are combined, and the start and end of the action are determined by a threshold. Then, through dynamic time warping or hierarchical clustering, semantically related sub-action sequences within the boundary are aggregated into complete process actions. After duration verification and logical order verification, action units that conform to process specifications are finally formed, realizing the transformation from single-frame recognition to temporal semantic aggregation.

7. The method according to claim 1, characterized in that, Step 3) specifically involves: Step 3.1) Compare an assembly process action with the pre-set assembly action of the current step. By comparing the spatial position coordinates, posture and gesture of the current action, it is determined whether it meets the expected assembly action. Step 3.2) If the comparison result shows that the expected action requirements are met, then push the next assembly action to the assembly worker and output the result to the AR device for display; if the comparison result shows that the action does not meet the expected action, then output the action error and output the result to the AR device for display.

8. The method according to claim 1, characterized in that, The assembly actions obtained by the ST-GCN algorithm are compared with predefined actions in the following ways: The accuracy and consistency of the recognition are verified by measuring the degree of matching between the two in terms of spatiotemporal features. The assembly actions output by ST-GCN contain a feature sequence aggregated in spatiotemporal terms, covering the spatial skeleton association and temporal dynamic changes of the actions. The predefined actions are standard action templates set based on process specifications, which include the preset spatial structure of skeleton points, the temporal logic of sub-actions, and key feature thresholds. First, the core features of the ST-GCN output actions are extracted and quantitatively compared with the corresponding features of the predefined actions. Spatially, the structural matching degree is judged by calculating the skeleton point coordinate deviation or the cosine similarity of feature vectors. Temporally, the temporal consistency is evaluated by calculating the sequence alignment error through dynamic time warping. At the same time, it is verified whether the logical order of sub-actions conforms to the predefined process. If all indicators are within the preset thresholds, it means that the recognized action matches the predefined action and can be judged as a compliant operation; if the deviation exceeds the threshold, it is marked as an abnormal action.

9. The method according to claim 1, characterized in that, Step 4) specifically involves: Step 4.1) After the assembly worker puts on the AR device, the comparison result is received. If the cross-comparison result shows that the action is correct, the next assembly action is pushed to the AR device and displayed to the assembly worker. If the cross-comparison results indicate an error, an error message will be displayed, and an alert will be issued to prompt the assembly worker to reassemble or make adjustments.

10. The method according to claim 1, characterized in that, The AR devices used are specifically: Using Microsoft HoloLens, a laser beam scanning (LBS) + diffractive waveguide method is employed. The laser scanning direction is controlled by a MEMS microelectromechanical system, and the light is guided by a grating structure to achieve total internal reflection, thus realizing holographic image projection.

Images