Gesture-assisted aero-engine similar connecting piece recognition method for augmented reality-assisted assembly

Abstract

A gesture-assisted aero-engine similar connecting piece recognition method for augmented reality-assisted assembly comprises the steps of firstly constructing an assembly feature gesture recognition algorithm for aero-engine similar connecting pieces, and then constructing a target detection aero-engine similar connecting piece type recognition network algorithm based on deep learning. Then, an augmented reality process decision network database facing similar connector recognition is built, and finally, an augmented reality similar connector detection guiding system based on visual gesture auxiliary recognition is built. According to the method, augmented reality auxiliary gesture guide classification of various similar connecting pieces of the aero-engine is realized, and the method has the advantages of accurate part recognition and classification, high camera mobility, high process guide visibility, reduction of worker training cost and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of aeroengines, and particularly relates to a gesture-assisted recognition method for similar connecting parts of aeroengines for augmented reality-assisted assembly. Background Art

[0002] Traditional similar connecting parts of aeroengines rely on manual recognition and classification, which have extremely high technical requirements for people and face many challenges. There are numerous and complex similar connecting parts of aeroengines, and they have high visual similarity, making it easy to have misassembly and missing assembly phenomena. Moreover, the manual measurement has a large amount of repetitive work, which is likely to cause a reduction in efficiency. At present, most of the device recognition relying on cameras is visual measurement from a fixed perspective, such as A visual identification method with position recovering and contour comparison for highly similar non-planar aviation angle pieces[J](He Q, Yang J, Li H, et al. A visual identification method with position recovering and contour comparison for highly similar non-planar aviation angle pieces[J]. Advanced Engineering Informatics, 2024, 62(PD): 102901-102901.), with poor mobility. Although the efficiency has been improved to some extent, the benefit is not obvious enough.

[0003] During the assembly process of aeroengines, the readability of paper documents is poor. The assembly guidance relies on the experience of workers, which has high requirements for personnel and high training costs. It requires workers to spend more time and energy learning the correct process flow, reducing the assembly efficiency.

[0004] Therefore, in order to improve the on-site auxiliary assembly efficiency of aeroengines, there is an urgent need for a gesture-assisted recognition method for similar connecting parts of aeroengines for augmented reality-assisted assembly, which can achieve advantages such as accurate part recognition and classification, high mobility of the camera, high visibility of the process guidance, and reduction of the worker training cost. There is no corresponding literature published yet. Summary of the Invention

[0005] To overcome the above-mentioned drawbacks of the prior art, the purpose of the present invention is to provide a gesture-assisted recognition method for similar connecting parts of aero-engines for augmented reality-assisted assembly, which realizes augmented reality-assisted gesture-guided classification of various similar connecting parts of aero-engines, and has the advantages of accurate part recognition and classification, high camera mobility, high visibility of process guidance, and reduction of worker training costs.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0007] A gesture-assisted recognition method for similar connecting parts of aero-engines for augmented reality-assisted assembly, comprising the following steps:

[0008] Step 1), constructing an assembly feature gesture recognition algorithm for similar connecting parts of aero-engines: First, customizing the holding gesture actions of different similar connecting parts of aero-engines based on the human skeleton and assembly behavior habits; Second, collecting and establishing a first-person gesture skeleton point data set for different part holding actions; Third, training a static part holding gesture classifier according to different gesture data; Finally, using a static gesture recognition method to identify the correctness of gestures for specific actions of the assembly behavior;

[0009] Step 2), constructing a target detection network algorithm for identifying the types of similar connecting parts of aero-engines based on deep learning: First, collecting a first-person handheld part data set within a specific range based on a specific gesture; Second, training a target detection and recognition network for similar connecting parts of aero-engines based on Yolo_v9-swin; Finally, realizing the recognition and detection of similar connecting parts of aero-engines at a specific visual distance under a specific holding gesture;

[0010] Step 3), building an augmented reality process decision network database for similar connecting part recognition: First, establishing a part information storage model using the mesh distributed idea according to the assembly process flow requirements of similar connecting parts of aero-engines; Second, augmenting the process information of the similar connecting part storage model, including digital models, text push of assembly process information, animation guidance of assembly process information, assembly-related gesture information, and part feature dimension information; Finally, establishing a database to store relevant information;

[0011] Step 4), establish an augmented reality similar connector detection guidance system based on visual gesture-assisted recognition: First, based on the custom gestures in Step 1), guide the user to hold the similar connector with the defined specific gestures to the corresponding position range; Second, call the static gesture recognition method in Step 1) to confirm whether the current gesture is correct; Third, perform object detection based on the neural network trained in Step 2) to obtain the types of similar connectors, and combine with the process information database in Step 3) to retrieve the current part object detection results to verify the category of the current part; Finally, based on the assembly information in Step 3), guide and prompt the user for the subsequent assembly operations of the current part.

[0012] The specific content of the said Step 1) is as follows:

[0013] Step 1.1) For the human behavior gesture characteristics in the assembly process of similar connectors of aero-engines, collect the assembly behavior gesture information, conduct in-depth analysis of the data, and extract and define the characteristic gestures in the assembly holding stage; Based on the skeleton standard method, define the gesture skeleton point matrix and vector characteristics in the assembly process, and construct the gesture feature matrix and vector model;

[0014] Step 1.2) Define the gesture skeleton points based on Hololens2, collect the custom gesture part holding skeleton point vector information in the first person, and normalize the gesture skeleton point vector. Take ThumbMetacarpals-Wrist as the reference vector for unitization and normalize the global vector, and establish a first-person 3D gesture skeleton point data set;

[0015] Step 1.3) Build a support vector machine classifier, perform non-linear mapping in the hyperspace using the Gaussian kernel function to achieve one-to-many classification, and obtain a static gesture recognition classifier; Use the support vector machine classifier to train on the custom data set to complete the optimization adjustment of the parameters and obtain the training model;

[0016] Step 1.4) During the assembly process, collect the current gesture data in real time, transmit it to the pre-trained classifier through wired / wireless communication for prediction and classification, obtain the current gesture type and return it to Hololens2.

[0017] The specific content of the said Step 2) is as follows:

[0018] Step 2.1) Based on Hololens2, build a first-person visual acquisition experimental scenario, and take the part held by the human hand with the custom gesture in Step 1) at the specified position as the acquisition object for image acquisition;

[0019] Step 2.2) Count the types of similar connectors involved, use the LabelImg tool to construct a dataset of different types of parts, with the annotation information including the part name and its bounding box in the 2D image, and use the trained model to automatically annotate and manually modify to increase the volume of the dataset, finally obtaining a large training dataset sample;

[0020] Step 2.3) Design a combination of the Yolo_v9 and swin-transformer algorithms, integrate the RepNCSPELAN4 feature extraction module and the SwinTransformerBlock module to extract the depth information features of the image. The self-attention mechanism enhances the model's global perception ability of target features by calculating the correlation between pixels in different regions of the image, dynamically allocating weight ratios, highlighting key features, and suppressing redundant information; Yolo_v9 realizes the accurate positioning and type recognition of similar connectors in the image by regressing and predicting the coordinate information and class probability of the target bounding box;

[0021] Step 2.4) Based on the image data collected in real time by Hololens2, transmit the image data to the server side in real time through the TCP communication protocol, use the pre-trained Yolo_v9-swin model to identify and locate the targets in the image, and transmit the recognition results including the target type, coordinate information, and visualization data back to the Hololens2 side through TCP.

[0022] The specific content of step 3) is as follows:

[0023] Step 3.1) Plan the association relationship of the similar connectors involved during the assembly process, and use a network architecture to sort out the assembly sequence and assembly correlation between each part;

[0024] Step 3.2) Digitalize the process requirements of the corresponding similar connectors, establish a visual process model, and perform associated modeling on the visual information, dimension information, custom associated gesture information, assembly position information, assembly method animation information, and assembly result information related to the parts;

[0025] Step 3.3) Establish a database based on MySQL, design an assembly process information model for the installation of small connectors in the PowerDesigner software, establish a multi-level part information database according to the part type hierarchy, save the digitalized process information in step 3.2), and realize information upload, retrieval, and download.

[0026] The specific content of step 4) is as follows:

[0027] Step 4.1) Set up a on-site gesture recognition system based on the Hololens augmented reality glasses. The user wears the Hololens to ensure that the current gestures of the person can be captured in the first person. Establish a human hand model in the unity software, and according to Step 2), customize the gesture model and use the augmented reality visualization visual guidance method to display the target gesture, and give prompts to guide the worker to make the corresponding holding gesture for the part to the corresponding position range;

[0028] Step 4.2) Invoke the static gesture recognition method in Step 1) to identify whether the current gesture is the expected defined gesture;

[0029] Step 4.3) Invoke the Yolov9_swin algorithm in Step 2) to identify the current part information and combine it with the process database in Step 3) to obtain the specific type of the current part and display it on the Hololens2 glasses screen;

[0030] Step 4.4) Invoke the process information database in Step 3) to obtain the current assembly link, prompt and guide the worker about the assembly position of the current part, and realize visual animation-guided assembly.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0032] (1) The present invention proposes an auxiliary recognition and assembly guidance technology for aero-engine connectors based on augmented reality technology, constructs a database containing target connector images, 3D models and specification parameters, and uses the camera of the augmented reality device to collect the image information of the connector to be recognized, combines subsequent image recognition algorithms with the database for matching, dynamically generates installation guidance information, realizes the rapid and accurate recognition of the connector, and can guide the installation of similar connectors on-site;

[0033] (2) The present invention uses gesture recognition based on the human skeleton to assist part recognition technology. By customizing the pinching gesture and using augmented reality technology to guide the gesture behavior for recognition, it determines that the pose of the assembler's hand is correct, ensuring the accuracy of subsequent visual detection of similar connectors;

[0034] (3) The present invention creatively adopts the Yolo_v9-swin deep learning neural network technology, combines the Yolo_v9 object detection algorithm with the swin_transformer feature extraction network, realizes the rapid positioning and region extraction of similar connectors through Yolo_v9, and uses swin_transformer to perform multi-level feature fusion on the local details and overall structure of the connector, realizes the accurate recognition and classification of connectors with different models, specifications and similar appearances, and significantly improves the recognition efficiency and accuracy of similar connectors. Description of the Drawings

[0035] Figure 1This is the flowchart of the embodiment of the present invention.

[0036] Figure 2 This is the definition diagram of gesture bone points in the embodiment of the present invention.

[0037] Figure 3 This is the Yolo_v9-swin connector visual recognition and classification neural network in the embodiment of the present invention.

[0038] Figure 4 This is the schematic diagram of visual gesture-assisted recognition of similar connectors in the embodiment of the present invention. Detailed implementation manners

[0039] The present invention will be described in detail below in conjunction with the embodiments and the drawings. The present invention can be implemented in many different forms and should not be considered limited to the embodiments described herein. On the contrary, these embodiments are provided so that the disclosure of the present invention is thorough and complete, and will fully convey the scope of the present invention to those skilled in the art.

[0040] As Figure 1 shown, a gesture-assisted recognition method for similar connectors of an aero-engine for augmented reality-assisted assembly includes the following steps:

[0041] Step 1), construct an assembly feature gesture recognition algorithm for similar connectors of an aero-engine: First, customize the holding gesture actions of different similar connectors of an aero-engine based on the human skeleton and assembly behavior habits; Second, collect and establish a first-person gesture bone point data set for different part holding actions; Third, train a static part holding gesture classifier according to different gesture data; Finally, use a static gesture recognition method to identify the correctness of gestures for specific actions of the assembly behavior; specifically:

[0042] Step 1.1) For the human behavior gesture features in the assembly process of similar connectors of an aero-engine, collect the assembly behavior gesture information, deeply analyze the data, and extract and define the feature gestures in the assembly holding stage; Based on the standard skeleton method, define the gesture bone point matrix and vector features in the assembly process, and construct the gesture feature matrix and vector model.

[0043] Step 1.2) Define the gesture bone points based on Hololens2, collect the first-person custom gesture part holding bone point vector information, and normalize the gesture bone point vector. Use ThumbMetacarpals-Wrist as the reference vector for unitization and normalize the global vector, as Figure 2 shown, and establish a first-person 3D gesture bone point data set.

[0044] Step 1.3) Build a support vector machine classifier. Flatten the 25×3 feature matrix into a 75-dimensional vector, and perform non-linear mapping using the Gaussian kernel function in the hyperspace to achieve one-versus-all classification, obtaining a static gesture recognition classifier. Its mathematical principle is shown in Formula (1) and Formula (2).

[0045]

[0046] Among them, Formula (1) is the objective function, and Formula (2) is the Gaussian kernel function; α i is the Lagrange multiplier, κ

[0047] is the kernel Gaussian function, b is the parameter to be solved, x, x i are the feature vectors of the input gesture samples, y i is the gesture category label, and σ is the standard deviation of the Gaussian distribution;

[0048] Use the support vector machine classifier to train on the custom dataset to complete the optimization adjustment of the parameters and obtain the training model;

[0049] Step 1.4) During the assembly process, collect the current gesture data in real time, transmit it to the pre-trained classifier through wired / wireless communication for prediction and classification, obtain the current gesture type, and return it to Hololens2;

[0050] Step 2), construct a deep learning-based target detection network algorithm for identifying similar connecting parts of aero-engines: First, based on specific gestures, collect a first-person handheld part dataset within a specific range; Second, train a target detection and recognition network for similar connecting parts of aero-engines based on Yolo_v9-swin; Finally, achieve the recognition and detection of similar connecting parts of aero-engines at a specific visual distance under specific holding gestures; specifically:

[0051] Step 2.1) Build a first-person visual acquisition experimental scenario based on Hololens2, and perform image acquisition with the parts held by the human hand at the specified position based on the custom gestures in Step 1);

[0052] Step 2.2) Count the types of similar connecting parts involved, use the LabelImg tool to construct a dataset of different types of parts, and the annotation information includes the part name and its bounding box in the two-dimensional image. Automatically annotate using the trained model and manually modify to increase the volume of the dataset, and finally obtain a large training dataset sample;

[0053] Step 2.3) Design a combination of the Yolo_v9 and swin-transformer algorithms, comprehensively extract the depth information features of the image using the RepNCSPELAN4 feature extraction module and the SwinTransformerBlock module. The self-attention mechanism enhances the model's global perception ability of target features by calculating the correlation between pixels in different regions of the image, dynamically allocating weight ratios, highlighting key features, and suppressing redundant information; Yolo_v9 predicts the coordinate information and class probabilities of the target bounding box through regression to achieve accurate positioning and species recognition of similar connectors in the image. Combining with the feature optimization of the self-attention mechanism further improves the recognition accuracy and robustness of the model in complex scenarios; its structural diagram is as shown in Figure 3 shown, and its mathematical principle is shown in formulas (3), (4), (5), and (6),

[0054]

[0055] MultiHead(Q, K, V) = Concat(head1, head2,..., head h )(5)

[0056] head i = Attention(QW i Q , KW i K , VW i V )(6)

[0057] Q represents the query matrix, K represents the key matrix, V represents the value matrix, X is the embedding representation matrix of the input sequence, W Q is the projection matrix for the query, W K is the projection matrix for the key, W V is the projection matrix for the value, d k is the scaling factor, and head i is the head of the attention mechanism;

[0058] Step 2.4) Based on the image data collected in real time by Hololens2, transmit the image data to the server side in real time through the TCP communication protocol, use the pre-trained Yolo_v9-swin model to identify and locate the targets in the image, and transmit the recognition results (including target species, coordinate information, and visualization data) back to the Hololens2 side through TCP;

[0059] Step 3), build an augmented reality process decision network database for identifying similar connectors: First, according to the assembly process requirements of similar connectors of aero-engines, establish a part information storage model using the mesh distributed idea; Second, enhance the process information of the similar connector storage model in an augmented reality manner, including digital models, text push of assembly process information, animation guidance of assembly process information, assembly-related gesture information, and part feature dimension information; Finally, establish a database to store relevant information; specifically:

[0060] Step 3.1) Plan the association relationships of similar connectors of aero-engines during the assembly process, and use a mesh architecture to sort out the assembly order and assembly relevance between parts;

[0061] Step 3.2) Digitalize the process requirements of the corresponding similar connectors, establish a visual process model, and perform association modeling on the visual information, dimension information, custom-related gesture information, assembly position information, assembly method animation information, assembly result information, etc. related to the parts;

[0062] Step 3.3) Establish a database based on MySQL, design an assembly process information model for the installation of small connectors in the PowerDesigner software, establish a multi-level part information database according to the part type hierarchy, save the digitalized process information in Step 3.2), and realize information upload, retrieval, and download;

[0063] Step 4), establish an augmented reality similar connector detection and guidance system based on visual gesture assistance recognition: First, based on the custom gestures in Step 1), guide the user to hold the similar connector in the corresponding position range with the defined specific gestures; Second, call the static gesture recognition method in Step 1) to confirm whether the current gesture is correct; Third, perform target detection based on the neural network trained in Step 2) to obtain the types of similar connectors, and jointly retrieve the current part target detection results from the process information database in Step 3) to verify the category to which the current part belongs; Finally, based on the assembly information in Step 3), guide and prompt the user about the subsequent assembly operations of the current part; as Figure 4 shown; specifically:

[0064] Step 4.1) Build a on-site gesture recognition system based on Hololens augmented reality glasses. The user wears Hololens to ensure that the current gesture of the person can be captured in the first person. Establish a human hand model in the unity software, and use the augmented reality visual guidance method to display the target gesture according to the custom gesture model in Step 2), and make prompts to guide the worker to make the corresponding holding gesture of the part to the corresponding position range;

[0065] Step 4.2) Call the static gesture recognition method in Step 1) to identify whether the current gesture is the expected defined gesture;

[0066] Step 4.3) Invoke the Yolov9_swin algorithm in Step 2) to identify the current part information and combine it with the process database in Step 3) to obtain the specific type of the current part and display it on the screen of the Hololens2 glasses;

[0067] Step 4.4) Invoke the process information database in Step 3) to obtain the current assembly link, prompt and guide the worker to the assembly position of the current part, realize visual animation-guided assembly, and improve the worker's assembly efficiency.

[0068] In summary, the present invention proposes a gesture-assisted recognition method for similar connecting parts of aero engines for augmented reality-assisted assembly. First, it proposes to use augmented reality gesture recognition to assist part target detection. Secondly, it identifies the types of similar connecting parts based on a deep learning network. Then, it accurately regresses the part types in the database based on the gesture recognition information to achieve higher-precision part recognition. Finally, based on the augmented reality similar connecting part detection and guidance system assisted by visual gestures, it realizes the efficient recognition and guided assembly of similar connecting parts.

[0069] Those skilled in the art of this technology can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used here have the same meaning as the general understanding of those of ordinary skill in the art to which the present invention belongs. It should also be understood that terms defined in a general dictionary, such as those, should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless defined as here.

[0070] It should be understood that the above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A gesture-assisted aero-engine similar connector recognition method for augmented reality-assisted assembly, characterized in that: The following steps are involved: Step 1) constructs an assembly feature gesture recognition algorithm for similar connectors of aircraft engines: first, customize the holding gestures of different similar connectors of aircraft engines based on the human skeleton and assembly behavior habits; second, collect and establish a first-person gesture skeleton point dataset for holding actions of different parts; third, train a static part holding gesture classifier based on different gesture data; Finally, a static gesture recognition method is used to identify the correctness of gestures for specific assembly actions; Step 2) Construct a deep learning-based target detection network algorithm for aircraft engine similar connector type recognition: First, based on specific gestures, collect a first-person handheld parts dataset within a specific range; second, train an aircraft engine similar connector target detection and recognition network based on Yolo_v9-swin; finally, realize the recognition and detection of aircraft engine similar connectors at a specific visual distance under a specific holding gesture; Step 3) Build an augmented reality process decision network database for similar connector identification: First, based on the assembly process requirements of similar connectors for aircraft engines, a part information storage model is established using a network distributed concept; Secondly, the augmented reality similar connector storage model process information, including digital model, assembly process information text push, assembly process information animation guidance, assembly related gesture information, and part feature size information; Finally, a database is established to store relevant information; Step 4), establishing an augmented reality similar connector detection and guidance system based on visual gesture-assisted recognition: first, based on the custom gesture in step 1), guide the user to hold the similar connector to the corresponding position range with the defined specific gesture; second, call the static gesture recognition method in step 1) to confirm whether the current gesture is correct; Next, target detection is performed based on the neural network trained in step 2) to obtain similar types of connectors, and the target detection result of the current part is retrieved from the process information database in step 3) to verify the category to which the current part belongs; finally, based on the assembly information in step 3), the user is guided to perform subsequent assembly operations on the current part.

2. The method according to claim 1, characterized in that: The step 1) is specifically as follows: Step 1.1) Based on the human behavior gesture features in the assembly process of similar connectors of aircraft engines, the assembly behavior gesture information is collected, the data is deeply analyzed, and the characteristic gestures that define the assembly holding stage are extracted; based on the skeleton standard method, the gesture skeleton point matrix and vector features in the assembly process are defined, and the gesture feature matrix and vector model are constructed; Step 1.2) Define the gesture skeleton points based on Hololens2, collect the first-person custom gesture part holding skeleton point vector information, normalize the gesture skeleton point vector, use ThumbMetacarpals-Wrist as the reference vector to unitize and normalize the global vector, and establish the first-person 3D gesture skeleton point dataset; Step 1.3) Build a support vector machine classifier, use Gaussian kernel function to perform nonlinear mapping in the hyperspace, realize one-to-many classification, and obtain a static gesture recognition classifier; use the support vector machine classifier to train on a custom data set, complete the optimization adjustment of parameters, and obtain a training model; Step 1.4) During the assembly process, the current gesture data is collected in real time and transmitted to the pre-trained classifier via wired / wireless communication for prediction and classification, and the current gesture type is obtained and returned to Hololens2.

3. The method according to claim 1, characterized in that The step 2) is specifically as follows: Step 2.1) Building a first-person visual acquisition experimental scene based on Hololens2, taking the parts held by a person's hand at a specified position based on the custom gesture in step 1) as the acquisition object for image acquisition; Step 2.2) Count the types of similar connectors involved, use the LabelImg tool to build a data set of different types of parts, and label the part name and its bounding box in the two-dimensional image. Use the trained model to automatically label and manually modify to increase the size of the data set, and finally obtain a large training data set sample; Step 2.3) Design a combination of Yolo_v9 and swin-transformer algorithms, integrate the RepNCSPELAN4 feature extraction module and the SwinTransformerBlock module to extract image depth information features, and the self-attention mechanism calculates the correlation between pixels in different regions of the image, dynamically allocates weight ratios, highlights key features, and suppresses redundant information to enhance the model's global perception of target features; Yolo_v9 predicts the coordinate information and category probability of the target bounding box through regression to achieve accurate positioning and type recognition of similar connectors in the image; Step 2.4) Based on the image data collected by Hololens2 in real time, the image data is transmitted to the server in real time through the TCP communication protocol, and the pre-trained Yolo_v9-swin model is used to identify and locate the target in the image, and the recognition results including the target type, coordinate information and visualization data are transmitted back to the Hololens2 end through TCP.

4. The method according to claim 1, characterized in that The step 3) is specifically as follows: Step 3.1) Plan the association relationship of similar connectors involved in the assembly process, and use a mesh architecture to sort out the assembly order and assembly correlation between the parts; Step 3.2) Digitize the process requirements of similar connectors, establish a visual process model, and associate and model the visual information, size information, custom associated gesture information, assembly position information, assembly method animation information, and assembly result information of the parts; Step 3.3) Establish a database based on MySQL, design an assembly process information model for small connector installation in PowerDesigner software, establish a multi-level part information database based on part type levels, save the digital process information of step 3.2) and realize information upload, retrieval and download.

5. The method according to claim 1, characterized in that The step 4) is specifically as follows: Step 4.1) Build an on-site gesture recognition system based on Hololens augmented reality glasses. The user wears Hololens to ensure that the current gesture of the person can be captured in the first person perspective. A human hand model is established in the unity software. According to the custom gesture model in step 2), the target gesture is displayed using augmented reality visualization and visual guidance, and prompts are given to guide the worker to make the corresponding holding gesture of the part to the corresponding position range; Step 4.2) calling the static gesture recognition method of step 1) to identify whether the current gesture is the expected defined gesture; Step 4.3) calls the Yolov9_swin algorithm of step 2) to identify the current part information and combines it with the process database of step 3) to obtain the specific type of the current part and display it on the Hololens2 glasses screen; Step 4.4) calls the process information database of step 3) to obtain the current assembly link, prompts and guides the worker to the current part assembly position, and realizes visual animation guided assembly.

Images