A fastener tightening operation assistance method and system based on AR technology

Abstract

This invention discloses an AR-based method for assisting fastener tightening operations, comprising the following steps: Step S100: An IoT server acquires and parses the unique machine code of the vehicle to be assembled; Step S200: The IoT server sends corresponding demonstration signals and parameter information to the AR glasses and torque gun according to the parsing results; Step S300: The AR glasses identify the workbench scene and demonstrate the corresponding process operation AR video according to the demonstration signal to guide the operator to use the torque gun for tightening operations; Step S400: The AR glasses and torque gun record the operation results and dynamic error correction information, and transmit them to the IoT server for digital report display. This invention can improve the efficiency and accuracy of operators in fastener tightening operations, and intelligently assist and guide operators in their work.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing, and in particular to a fastener tightening operation assistance method and system based on AR technology. Background Technology

[0002] Due to its disruptive and revolutionary nature, AR technology has garnered significant attention. As early as the 1990s, related technologies were launched, but due to the high price, long latency, and limited computing power of devices at the time, these products ultimately failed, and the first AR boom subsided. In recent years, with continuous technological advancements, equipment capabilities have become increasingly powerful and prices have decreased accordingly, leading to more widespread applications of AR technology. Consequently, a growing number of manufacturing managers hope to utilize this technology to build intelligent factories, thereby improving enterprise production efficiency, productivity, and energy optimization capabilities.

[0003] In industrial manufacturing, the fastener tightening process is crucial for connection reliability. Accurate control of axial force requires control of the friction coefficient and proper use of various aspects. However, for operators, improving accuracy reduces operational efficiency, increases training costs, and requires bearing the losses caused by misoperation. Summary of the Invention

[0004] The main objective of this invention is to address the shortcomings of existing technologies in improving the efficiency and accuracy of operators' fastener tightening operations and the lack of intelligent operation assistance guidance, by providing a fastener tightening operation assistance method and system based on AR technology.

[0005] To achieve the above objectives, the present invention provides a fastener tightening operation assistance method based on AR technology, comprising the following steps:

[0006] Step S100: The IoT server obtains and parses the unique machine code of the vehicle to be assembled;

[0007] Step S200: The IoT server sends the corresponding demonstration signal and parameter information to the AR glasses and torque gun respectively based on the parsing results;

[0008] Step S300: The AR glasses recognize the workbench scene and demonstrate the corresponding process operation AR video based on the demonstration signal to guide the operator to use the torque gun to perform tightening operations;

[0009] Step S400: The AR glasses and torque gun record the operation results and dynamic error correction information, and transmit them to the IoT server for digital report display.

[0010] Preferably, step S100 includes: a barcode reader scanning the SN barcode on the vehicle to be assembled passing through the PBS roller conveyor, transmitting the unique machine code information contained in the SN barcode to the Internet of Things (IoT) server, and the IoT server parsing the unique machine code according to the pre-stored unique machine code parsing rules to obtain the model information of the vehicle to be assembled.

[0011] Preferably, step S200 includes: the IoT server converts and outputs a demonstration signal corresponding to the model information of the vehicle to be assembled, and at the same time retrieves the parameter information of the torque gun according to the model information of the vehicle to be assembled, and transmits it to the AR glasses and the torque gun through wireless communication, and the torque gun receives the parameter information and automatically adjusts the torque.

[0012] Preferably, step S300 includes: the AR glasses recognizing the positioning tag on the workbench, determining the demonstration coordinates of the process operation AR video, retrieving the corresponding process operation AR video according to the demonstration signal, demonstrating the process operation AR video in conjunction with the demonstration coordinates, and simultaneously, the 3D motion-sensing camera capturing and analyzing the operator's operation actions and transmitting them to the AR glasses to be loaded into the process operation AR video for real-time linkage demonstration, guiding the operator to use the torque gun to perform practical tightening operations.

[0013] Preferably, step S400 includes: AR glasses recording the position data of the fasteners tightened on the vehicle to be assembled and reminder and guidance information for incorrect actions of the operators; torque gun recording the final actual torque and final actual angle of the fasteners tightened on the vehicle to be assembled; AR glasses and torque gun transmitting the recorded data to the Internet of Things server via wireless communication; the Internet of Things server performing digital report processing on the recorded data and displaying it on a monitor.

[0014] Furthermore, this invention also provides a fastener tightening operation assistance system based on AR technology, comprising: an acquisition module for acquiring the unique machine code of the vehicle to be assembled; a parsing module for parsing the unique machine code of the vehicle to be assembled; an identification module for identifying the workbench scene; a transmission module for sending the demonstration signal and parameter information corresponding to the parsing result to the AR glasses and the torque gun respectively, and for transmitting the operation results and dynamic error correction information to the IoT server; a demonstration module for demonstrating the corresponding process operation AR video according to the demonstration signal, guiding the operator to use the torque gun for tightening operations; a recording module for recording the operation results and dynamic error correction information; and a display module for digitally displaying the operation results and dynamic error correction information; wherein the acquisition module, the parsing module, and the display module are located in the IoT server, the identification module and the demonstration module are located in the AR glasses, the IoT server, the AR glasses, and the torque gun all have the transmission module, and the AR glasses and the torque gun all have the recording module.

[0015] Preferably, the acquisition module obtains the unique machine code of the vehicle to be assembled by scanning the SN barcode on the vehicle passing through the PBS roller conveyor with a barcode reader; the parsing module parses the unique machine code according to the pre-stored unique machine code parsing rules to obtain the model of the vehicle to be assembled.

[0016] Preferably, it further includes a conversion module and a first retrieval module. The conversion module is used to convert the model information of the vehicle to be assembled into a demonstration signal corresponding to the model information. The retrieval module is used to retrieve the parameter information of the torque gun according to the model information of the vehicle to be assembled. The conversion module and the first retrieval module are located in the Internet of Things server.

[0017] Preferably, the identification module identifies the positioning tag on the workbench and determines the demonstration coordinates of the process operation AR video. The system also includes a second retrieval module, which is used to retrieve the corresponding process operation AR video according to the demonstration signal. The demonstration module combines the demonstration coordinates with the operation actions of the operator captured and analyzed by the loaded 3D motion sensing camera to perform real-time linkage demonstration of the process operation AR video, guiding the operator to use the torque gun to perform practical tightening operations. The second retrieval module is located in the AR glasses.

[0018] Preferably, "the recording module records the position data of the fasteners tightened on the vehicle to be assembled, the reminder and guidance information for incorrect actions of the operators, and the final actual torque and final actual angle."

[0019] The fastener tightening operation assistance method based on AR technology provided by this invention has the following beneficial effects: The IoT server acquires and parses the unique machine code of the vehicle to be assembled. Having acquired the unique machine code, it can parse the model information of the vehicle to be assembled according to the parsing rules, providing initial data for the AR glasses to select which process video to demonstrate. Based on the parsing results, the IoT server sends corresponding demonstration signals and parameter information to the AR glasses and the torque gun respectively. It provides signal basis for the AR glasses to select the process video by outputting signals, and intelligently adjusts the torque gun by outputting parameter information, directly assisting subsequent operators. The operation is convenient; the AR glasses recognize the workbench scene and demonstrate the corresponding process operation AR video based on the demonstration signal to guide the operator to use the torque gun to tighten the fasteners. By recognizing and locating the workbench, the demonstrated process video is integrated with the current actual working environment, and then the operator is guided to use the torque gun to complete the fastener tightening operation through AR video. The AR glasses and torque gun record the operation results and dynamic error correction information and transmit them to the IoT server for digital report display. During the operation, the AR glasses and torque gun record the position and torque data of the fasteners in actual operation and transmit them to the IoT server in real time for digital display on the monitor. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort:

[0021] Figure 1 A flowchart illustrating an AR-based fastener tightening operation assistance method according to an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of a fastener tightening operation assistance system based on AR technology is provided in one embodiment of the present invention;

[0023] Figure 3 This is a diagram illustrating the display of warning messages during traditional fastener tightening operations.

[0024] Figure 4 This is a comparative diagram of AR demonstration operation correction reminders provided in one embodiment of the present invention;

[0025] Figure 5 A schematic diagram illustrating the AR demonstration component selection assistance provided in one embodiment of the present invention. Figure 1 ;

[0026] Figure 6 A schematic diagram illustrating the AR demonstration component selection assistance provided in one embodiment of the present invention. Figure 2 ;

[0027] Figure 7 This is a schematic diagram illustrating the auxiliary guidance for selecting the location in an AR demonstration operation, provided in one embodiment of the present invention.

[0028] Figure 8 This is a schematic diagram illustrating the AR demonstration task result feedback and auxiliary guidance provided in one embodiment of the present invention; Detailed Implementation

[0029] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Typical embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0031] The general idea of ​​this invention is to provide a fastener tightening operation assistance method based on AR technology, which addresses the shortcomings of existing technologies in improving the efficiency and accuracy of operators' fastener tightening operations and the lack of intelligent operation assistance guidance. The IoT server in this invention acquires and parses the unique machine code of the vehicle to be assembled. This unique machine code allows the server to analyze the vehicle's model information using parsing rules, providing initial data for the AR glasses to select the appropriate process video for demonstration. Based on the parsing results, the IoT server sends corresponding demonstration signals and parameter information to the AR glasses and torque gun. The output signals provide a basis for the AR glasses to select the process video, and the output parameter information intelligently adjusts the torque gun, facilitating direct operation for workers. The AR glasses recognize the workbench scene and demonstrate the corresponding process operation AR video based on the demonstration signal, guiding workers to use the torque gun for tightening operations. By recognizing and locating the workbench, the demonstrated process video is integrated with the current actual working environment, allowing the AR video to guide workers in using the torque gun to complete the fastener tightening operation. The AR glasses and torque gun record operational performance and dynamic error correction information, transmitting this data to the IoT server for digital report display. During operation, the AR glasses and torque gun record the actual fastener positions and torque data, transmitting this data in real time to the IoT server for digital display.

[0032] To better understand the above technical solutions, the following will describe the above technical solutions in detail with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solutions of this application, rather than limitations on the technical solutions of this application. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0033] Reference Figure 1 , Figure 1 The diagram shown is a flowchart illustrating a fastener tightening operation assistance method based on AR technology according to an embodiment of the present invention. In this embodiment, the fastener tightening operation assistance method based on AR technology includes:

[0034] Step S100: The IoT server obtains and parses the unique machine code of the vehicle to be assembled.

[0035] After obtaining the unique machine code of the vehicle to be assembled, the IoT server parses the unique machine code using the parsing rules stored on the server to obtain the model information of the vehicle to be assembled. Knowing the model of the vehicle to be assembled, it can send out the corresponding signal, providing the original judgment data basis for the AR glasses to select which process video to demonstrate. AR technology is a technology that cleverly integrates virtual information with the real world. It widely uses various technologies such as multimedia, 3D modeling, real-time tracking and registration, intelligent interaction, and sensing to simulate and apply computer-generated text, images, 3D models, music, videos, and other virtual information to the real world. The two types of information complement each other, thereby achieving "augmentation" of the real world.

[0036] The IoT server supports connection to and data collection from PLCs, torque wrenches, sensors, instruments, robots, machine tools, and various controllers. It features data acquisition, protocol parsing, and edge computing, transmitting data via wired or wireless communication methods. This invention's fastener tightening operation primarily targets bolt tightening during automotive assembly. Before assembly, each vehicle has a unique machine code barcode. This requires reading and describing this barcode to obtain the unique machine code of the vehicle to be assembled. This can be done using a barcode reader, which functions similarly to a barcode scanner, using infrared light to scan the barcode. The barcode reader, acting as a data acquisition device, connects to the IoT server via a wireless network, transmitting the scanned unique machine code data to the IoT server for processing. This process yields the vehicle's model information, providing a basis for subsequent control.

[0037] Specifically, step S100 includes: a barcode reader scanning the SN barcode on the vehicle to be assembled passing through the PBS roller conveyor, transmitting the unique machine code information contained in the SN barcode to the Internet of Things server, and the Internet of Things server parsing the unique machine code according to the pre-stored unique machine code parsing rules to obtain the model information of the vehicle to be assembled.

[0038] After painting, the vehicles are conveyed to the final assembly area via a PBS roller conveyor. As the vehicles pass through the PBS roller conveyor, barcode readers installed on the line scan and read the unique machine code (SN) barcode on each vehicle. The barcode reader translates the unique machine code information and transmits it to an IoT server connected to the Internet of Things (IoT) server. The IoT server receives the unique machine code information, retrieves pre-stored code parsing rules, and parses the unique machine code according to the rules to obtain relevant vehicle information, such as a unique machine code of 20230. The parsing rule for code 9150102030008 is as follows: the first 8 digits represent the production date (year, month, and day); the 9th and 10th digits represent the production plant number (e.g., 01 represents the South China plant); the 11th and 12th digits represent the production line number (e.g., 02 represents production line #2); the 13th and 14th digits represent the model information (e.g., 03 represents the BYD-TF series); and the 15th to 18th digits represent the serial number of units produced that day (e.g., 00008 represents the 8th unit). The IoT server parses this unique machine code according to the parsing rule to obtain the vehicle's model information.

[0039] Step S200: The IoT server sends the corresponding demonstration signal and parameter information to the AR glasses and torque gun respectively based on the parsing results.

[0040] After analyzing the vehicle model information, the AR glasses need to be provided with signal basis for selecting process videos by outputting signals. The torque gun is also intelligently adjusted by outputting parameter information to facilitate subsequent operations for workers. The AR glasses recognize the output signals to demonstrate the corresponding process videos.

[0041] Specifically, the IoT server converts and outputs a demonstration signal corresponding to the model information of the vehicle to be assembled. At the same time, it retrieves the parameter information of the torque gun based on the model information of the vehicle to be assembled and transmits it to the AR glasses and the torque gun through wireless communication. The torque gun receives the parameter information and automatically adjusts the torque.

[0042] After the IoT server parses and obtains the model information of the vehicle to be assembled, it internally uses the I / O module to input / output and output a demonstration signal corresponding to the model information. The demonstration signal is the basis for the AR glasses to start which process video to demonstrate. For example, if there are 6 models of vehicles to be assembled, the IoT server will output 6 demonstration signals with different information corresponding to these 6 models. For example, the first model information corresponds to an input demonstration signal containing 1. The AR glasses have a Windows system with a control program installed. The control program interprets the demonstration signal and selects the process video of the fixed station corresponding to the car model that is stored in the AR glasses in advance for demonstration. The operator watches the demonstration content displayed by the AR glasses, thereby guiding the operator to perform accurate operations. At the same time, the IoT server also retrieves the working torque parameters of the car model according to the model information and transmits them to the torque gun. The torque gun has a built-in controller, which receives the torque parameters, automatically sets them, and automatically adjusts the torque gun to the torque parameter setting.

[0043] Step S300: The AR glasses recognize the workbench scene and demonstrate the corresponding process operation AR video based on the demonstration signal to guide the operator to use the torque gun to perform tightening operations.

[0044] AR glasses can identify and locate the workbench, allowing the demonstration video to blend with the actual working environment. The AR video then guides the operator to use a torque gun to tighten fasteners.

[0045] The video content demonstrated by AR glasses needs to be integrated with the actual work scene to help guide operators to complete the actual work operation. In order to accurately integrate the demonstration video with the actual workbench, the AR glasses need to locate and identify the workbench. In this way, by demonstrating the process video, operators can see both the actual work scene and the process video in the AR glasses to complete the work.

[0046] Specifically, the AR glasses identify the positioning tags on the workbench, determine the demonstration coordinates of the AR video of the process operation, retrieve the corresponding AR video of the process operation based on the demonstration signal, and demonstrate the AR video of the process operation in conjunction with the demonstration coordinates. At the same time, the 3D motion-sensing camera captures and analyzes the operator's operation actions and transmits them to the AR glasses to be loaded into the AR video of the process operation for real-time linkage demonstration, guiding the operator to use the torque gun to perform practical tightening operations.

[0047] Each workbench has a label that serves as both a workstation process identifier and an AR video positioning tool. AR glasses scan the label to obtain the current workstation process and its coordinates. After obtaining this information, the AR glasses compare it with pre-stored process videos. Once verified, the AR glasses retrieve the corresponding process video based on the received demonstration signal. Using the positioning coordinates, the process video demonstration is then integrated with the work scene viewed through AR. Simultaneously, 3D motion-sensing cameras are installed in the work area. These cameras are capable of sensing and capturing human movements. They utilize depth-sensing technologies such as infrared sensors, structured light, or time-of-flight technology to acquire real-time 3D position, skeletal posture, and motion information of the human body. Unlike traditional cameras, 3D motion-sensing cameras not only capture the external image of the human body but also analyze its depth information, enabling the tracking and recognition of human movements. Through these technologies, 3D motion-sensing cameras can capture and analyze human movements in real-time without any physical contact, facilitating interaction with virtual reality, augmented reality, and gaming applications. 3D motion-sensing cameras capture the operator's movements and analyze them using image recognition technology to obtain depth information of the human body. Image recognition technology can employ object inspection algorithms to mark the location and type of objects in the image. Commonly used object inspection algorithms include R-CNN series, YOLD, and SSD. The 3D motion-sensing camera communicates wirelessly with the IoT server and AR glasses. After capturing and analyzing the operator's movements, the 3D motion-sensing camera transmits the depth information of the movements to the AR glasses, loading it into the currently displayed process video for auxiliary demonstrations, guiding the operator to use a torque gun for practical tightening operations.

[0048] like Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the AR demonstration operation correction reminder comparison provided in one embodiment of the present invention; the AR glasses recognize a vehicle of model A with an initial assembly scene, determine the current assembly operation process to be completed, and mark the specific screw-tightening sequence of the current component on the AR glasses display screen, such as tightening screws from top to bottom. When it is recognized that the operator is aligning the torque gun with other holes, a red cross reminder will appear on the AR screen, along with correct text guidance and the correct hole location markings; for example... Figure 5 As shown, Figure 5 A schematic diagram illustrating the AR demonstration component selection assistance provided in one embodiment of the present invention. Figure 1When a worker reaches for a fastener, the AR glasses identify the type of component in the component box and compare it with the process-matching component type. If a mismatch is found, a red warning image and an "X" will appear in the component box area, while the correct component box area will be highlighted with a checkmark to indicate correctness. Figure 6 As shown, Figure 6 A schematic diagram illustrating the AR demonstration component selection assistance provided in one embodiment of the present invention. Figure 2 When other parts are mixed in with the parts box, when the operator takes a part out of the box, the AR glasses will measure the part. If the size of the part is found to be different from the standard part size in the box, an "X" will appear, along with a text explanation, such as: "Incorrect selection. The current size is 5*5mm. You should select a square with a radius of 6mm." Figure 7 As shown, Figure 7 This is a schematic diagram illustrating AR demonstration operation location selection assistance according to an embodiment of the present invention. When selecting a work process location, the system can accurately locate the target location and guide the installation position of the next step using highlighting. If the operator installs the screw in the wrong position, an alarm will be triggered to correct the error. For example, if the operator should install the screw in the first position but installs it in the second position, there will be voice, text, and animation alarms. Figure 8 As shown, Figure 8 This is a schematic diagram illustrating AR demonstration operation result feedback and auxiliary guidance provided in one embodiment of the present invention. The system can interface with tool sensor values ​​to help workers receive real-time feedback and reminders on operation results through AR glasses. For example, the torque applied by the operator in a certain step can be viewed in real time through AR glasses, and insufficient or excessive torque can be indicated with voice, text, or animation prompts. Traditional tightening operation instructions, such as... Figure 3 As shown, Figure 3 This is a schematic diagram of a traditional fastener tightening operation reminder and warning information display. The operation instructions are generally only text prompts, with little guidance and description during operation. At most, there is a warning for operation errors, but no error correction instructions. It is often necessary to repeatedly redo the operation to confirm whether it is qualified, which has the disadvantages of low efficiency and easy error.

[0049] Step S400: The AR glasses and torque gun record the operation results and dynamic error correction information, and transmit them to the IoT server for digital report display.

[0050] During the operation, AR glasses and a torque gun are used to record the position and torque data of the fasteners in actual operation, which are then transmitted to the IoT server in real time and displayed digitally on a monitor.

[0051] Specifically, step S400 includes: AR glasses recording the position data of the fasteners tightened on the vehicle to be assembled and reminder and guidance information for incorrect actions of the operators; torque gun recording the final actual torque and final actual angle of the fasteners tightened on the vehicle to be assembled; AR glasses and torque gun transmitting the recorded data to the Internet of Things server via wireless communication; the Internet of Things server performing digital report processing on the recorded data and displaying it on a monitor.

[0052] After the operators complete their work, the work process or technique needs to be optimized for further improvement. This typically requires data from the work process, such as torque data. In this invention, both the AR glasses and the torque gun have recording functions. The AR glasses record the position of the fasteners being tightened, and the torque gun records the torque and angle at that position. The AR glasses and torque gun transmit the position data, torque, and angle data to the IoT server in real time. The IoT server integrates and processes the received position information, torque information, and angle information according to the time of receipt, ultimately generating a data report which is then displayed on a monitor connected to the IoT server.

[0053] Based on the above methods, the IoT server in this invention acquires and parses the unique machine code of the vehicle to be assembled. With this unique machine code, the server can parse the model information of the vehicle according to parsing rules, providing initial data for the AR glasses to select which process video to demonstrate. The IoT server sends corresponding demonstration signals and parameter information to the AR glasses and torque gun based on the parsing results. It provides signal basis for the AR glasses to select the process video by outputting signals and intelligently adjusts the torque gun by outputting parameter information, facilitating direct operation for subsequent workers. The AR glasses recognize the workbench scene and demonstrate the corresponding process operation AR video based on the demonstration signal, guiding workers to use the torque gun for tightening operations. By recognizing and locating the workbench, the demonstrated process video is integrated with the current actual working environment, thus guiding workers to use the torque gun to complete the fastener tightening operation. The AR glasses and torque gun record operation results and dynamic error correction information, and transmit them to the IoT server for digital report display. During operation, the AR glasses and torque gun record the actual fastener position and torque data, transmitting them in real time to the IoT server for digital display on a monitor.

[0054] Accordingly, the present invention also provides a fastener tightening operation assistance system based on AR technology, referring to... Figure 2 , Figure 2The diagram shows a modular structure of an AR-based fastener tightening operation assistance system according to an embodiment of the present invention. This system assists fastener tightening operations using the AR-based fastener tightening operation assistance method described above. The system includes an acquisition module for acquiring the unique machine code of the vehicle to be assembled; a parsing module for parsing the unique machine code of the vehicle to be assembled; an identification module for identifying the workbench scene; a transmission module for sending demonstration signals and parameter information corresponding to the parsing results to the AR glasses and torque gun respectively, and for transmitting operation results and dynamic error correction information to an IoT server; a demonstration module for demonstrating the corresponding process operation AR video based on the demonstration signal, guiding the operator to use the torque gun for tightening operations; a recording module for recording operation results and dynamic error correction information; and a display module for digitally displaying the operation results and dynamic error correction information. The acquisition module, parsing module, and display module are located in the IoT server, while the identification module and demonstration module are located in the AR glasses. The IoT server, AR glasses, and torque gun all have the transmission module, and the AR glasses and torque gun all have the recording module.

[0055] Preferably, the acquisition module obtains the unique machine code of the vehicle to be assembled by scanning the SN barcode on the vehicle passing through the PBS roller conveyor with a barcode reader; the parsing module parses the unique machine code according to the pre-stored unique machine code parsing rules to obtain the model of the vehicle to be assembled.

[0056] Preferably, it further includes a conversion module and a first retrieval module. The conversion module is used to convert the model information of the vehicle to be assembled into a demonstration signal corresponding to the model information. The retrieval module is used to retrieve the parameter information of the torque gun according to the model information of the vehicle to be assembled. The conversion module and the first retrieval module are located in the Internet of Things server.

[0057] Preferably, the identification module identifies the positioning tag on the workbench and determines the demonstration coordinates of the process operation AR video. The system also includes a second retrieval module, which is used to retrieve the corresponding process operation AR video according to the demonstration signal. The demonstration module combines the demonstration coordinates with the operation actions of the operator captured and analyzed by the loaded 3D motion sensing camera to perform real-time linkage demonstration of the process operation AR video, guiding the operator to use the torque gun to perform practical tightening operations. The second retrieval module is located in the AR glasses.

[0058] Preferably, "the recording module records the position data of the fasteners tightened on the vehicle to be assembled, the reminder and guidance information for incorrect actions of the operators, and the final actual torque and final actual angle."

[0059] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0060] Similarly, it should be understood that, in order to simplify this disclosure and aid in understanding one or more of the various aspects of the invention, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0061] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0062] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0063] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0064] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

Claims

1. A fastener tightening operation assistance method based on AR technology, characterized in that, Includes the following steps: Step S100: The IoT server obtains and parses the unique machine code of the vehicle to be assembled; Step S200: The IoT server sends the corresponding demonstration signal and parameter information to the AR glasses and torque gun respectively based on the parsing results; Step S300: The AR glasses recognize the workbench scene and demonstrate the corresponding process operation AR video based on the demonstration signal to guide the operator to use the torque gun to perform tightening operations; Step S400: The AR glasses and torque gun record the actual operation results and dynamic error correction information, and transmit them to the IoT server for digital report display; Step S100 includes: The barcode reader scans the SN barcode on the vehicle to be assembled as it passes through the PBS roller conveyor, and transmits the unique machine code information contained in the SN barcode to the IoT server. The IoT server parses the unique machine code according to the pre-stored unique machine code parsing rules to obtain the model information of the vehicle to be assembled. Step S200 includes: The IoT server converts and outputs a demonstration signal corresponding to the model information of the vehicle to be assembled. At the same time, it retrieves the parameter information of the torque gun based on the model information of the vehicle to be assembled and transmits it to the AR glasses and the torque gun through wireless communication. The torque gun receives the parameter information and automatically adjusts the torque. Step S300 includes: The AR glasses identify the positioning tags on the workbench, determine the demonstration coordinates of the AR video of the process operation, retrieve the corresponding AR video of the process operation according to the demonstration signal, and demonstrate the AR video of the process operation in combination with the demonstration coordinates. At the same time, the 3D motion-sensing camera captures and analyzes the operator's operation actions and transmits them to the AR glasses to be loaded into the AR video of the process operation for real-time linkage demonstration, guiding the operator to use the torque gun to perform practical tightening operations. Step S400 includes: The AR glasses record the position data of the fasteners tightened on the vehicle to be assembled and provide reminders and guidance for incorrect actions by the operators. The torque gun records the final actual torque and final actual angle of the fasteners tightened on the vehicle to be assembled. The AR glasses and torque gun transmit the recorded data to the IoT server via wireless communication. The IoT server performs digital reporting on the recorded data and displays it on a monitor.

2. A fastener tightening operation assistance system based on AR technology, characterized in that, include: The acquisition module is used to obtain the unique machine code of the vehicle to be assembled. The parsing module is used to parse the unique machine code of the vehicle to be assembled. The recognition module is used to identify the workbench scene; The transmission module is used to send the demonstration signal and parameter information corresponding to the analysis results to the AR glasses and torque gun respectively, and to transmit the operation results and dynamic error correction information to the IoT server. The demonstration module is used to display the corresponding process operation AR video based on the demonstration signal, guiding the operator to use the torque gun to perform the tightening operation; The recording module is used to record operational performance and dynamic error correction information; The display module is used to digitally display operational performance and dynamic error correction information. The acquisition module, the parsing module, and the display module are located in the IoT server, while the identification module and the demonstration module are located in the AR glasses. The IoT server, the AR glasses, and the torque gun all have the transmission module, and the AR glasses and the torque gun all have the recording module. The acquisition module obtains the unique machine code of the vehicle to be assembled by scanning the SN barcode on the vehicle passing through the PBS roller conveyor with a barcode reader; the parsing module parses the unique machine code according to the pre-stored unique machine code parsing rules to obtain the model of the vehicle to be assembled. The system also includes a conversion module and a first retrieval module. The conversion module is used to convert the model information of the vehicle to be assembled into a demonstration signal corresponding to the model information. The retrieval module is used to retrieve the parameter information of the torque gun according to the model information of the vehicle to be assembled. The conversion module and the first retrieval module are located in the Internet of Things server. The identification module identifies the positioning tag on the workbench and determines the demonstration coordinates of the AR video of the process operation. The system also includes a second retrieval module, which is used to retrieve the corresponding AR video of the process operation according to the demonstration signal. The demonstration module combines the demonstration coordinates and loads a 3D motion-sensing camera to capture and analyze the operator's operation actions to perform real-time linkage demonstration of the AR video of the process operation, guiding the operator to use a torque gun to perform practical tightening operations. The second retrieval module is located in the AR glasses. The recording module records the position data of the fasteners tightened on the vehicle to be assembled, reminders and guidance information for incorrect actions of the operators, and the final actual torque and final actual angle.

Images