Fastener tightening operation auxiliary method and system based on AR technology

Abstract

The invention discloses a fastener tightening operation assisting method based on an AR technology. The method comprises the following steps that S100, an Internet of Things server obtains and analyzes a unique machine code of a vehicle to be subjected to final assembly; s200, the Internet of Things server sends corresponding demonstration signals and parameter information to the AR glasses and the moment gun according to the analysis result; s300, the AR glasses recognize a workbench scene, and a corresponding process operation AR video is demonstrated according to the demonstration signal to guide an operator to use a torque gun to perform tightening operation; and step S400, the AR glasses and the moment gun record the operation actual performance and the dynamic error correction information, and transmit the information to the Internet of Things server for digital report display. According to the invention, the operation efficiency and accuracy of tightening operation of a fastener by an operator can be improved, and the operation of the operator is intelligently assisted and guided.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing, and in particular to an AR technology-based fastener tightening operation auxiliary method and system. Background Art

[0002] Due to its disruptive and revolutionary nature, AR technology has received a lot of attention. As early as the 1990s, related technical applications were launched on the market, but due to the high price of AR technology at the time, its own long delay, limited computing power of the equipment and other defects, these products ended in failure, and the first AR boom subsided. In recent years, with the continuous development of science and technology, the equipment capabilities have become stronger and stronger, and the corresponding prices have become cheaper and cheaper. The application of AR technology has begun to become more and more extensive. Therefore, more and more manufacturing company managers hope to use this technology to build intelligent factories to improve the company's production efficiency, productivity and energy optimization configuration capabilities.

[0003] In industrial manufacturing, the fastener tightening process is in need of connection reliability. To achieve accurate control of axial force, it is necessary to control the friction coefficient and use all aspects correctly. However, for operators, while improving accuracy will reduce operating efficiency, increase operator training costs, and bear the losses caused by misoperation. Summary of the invention

[0004] The main purpose of the present invention is to provide a fastener tightening operation assistance method and system based on AR technology to address the shortcomings of the prior art that the operating efficiency and accuracy of operators in fastener tightening operations cannot be improved and the operation auxiliary guidance is not intelligent enough.

[0005] To achieve the above object, the present invention provides an AR technology-based fastener tightening operation auxiliary method, comprising the following steps:

[0006] Step S100: The IoT server obtains and parses the unique machine code of the vehicle to be assembled;

[0007] Step S200: The IoT server sends corresponding demonstration signals and parameter information to the AR glasses and the torque gun respectively according to the analysis results;

[0008] Step S300: The AR glasses recognize the workbench scene, and demonstrate the corresponding process operation AR video according to the demonstration signal to guide the operator to use the torque gun to perform the tightening operation;

[0009] Step S400: AR glasses and torque gun record operation performance and dynamic error correction information, and transmit them to the IoT server for digital report display.

[0010] Preferably, step S100 includes: a barcode reader scans the SN barcode on the vehicle to be assembled passing through the PBS roller, and transmits the unique machine code information contained in the SN barcode to the Internet of Things server. The Internet of Things server parses the unique machine code according to a unique machine code parsing rule stored in advance to obtain the model information of the vehicle to be assembled.

[0011] Preferably, step S200 includes: the IoT server converts and outputs a demonstration signal corresponding to the model information according to the model information of the vehicle to be assembled, and at the same time retrieves parameter information of the torque gun according to the model information of the vehicle to be assembled, and transmits it to the AR glasses and the torque gun via wireless communication. The torque gun receives the parameter information and automatically adjusts the torque.

[0012] Preferably, step S300 includes: the AR glasses identify the positioning tag of the workbench, determine the demonstration coordinates of the process operation AR video, call up the corresponding process operation AR video according to the demonstration signal, and demonstrate the process operation AR video in combination with the demonstration coordinates. At the same time, the 3D somatosensory camera captures and analyzes the operating actions of the operator, and transmits them to the AR glasses and loads them into the process operation AR video for real-time linkage demonstration, guiding the operator to use the torque gun to perform actual tightening operations.

[0013] Preferably, step S400 includes: the AR glasses record the position data of the fasteners tightened on the vehicle to be assembled and the reminder and guidance information for the operators' erroneous actions; the torque gun records the final actual torque and the final actual angle of the fasteners tightened on the vehicle to be assembled; the AR glasses and the torque gun transmit the recorded data to the Internet of Things server via wireless communication; the Internet of Things server processes the recorded data into digital reports and displays them on a display.

[0014] In addition, the present invention also provides a fastener tightening operation auxiliary system based on AR technology, including: an acquisition module, used to acquire the unique machine code of the vehicle to be assembled; a parsing module, used to parse the unique machine code of the vehicle to be assembled; an identification module, used to identify the workbench scene; a transmission module, used to send the demonstration signal and parameter information corresponding to the analysis result to the AR glasses and the torque gun respectively, and used to transmit the operation performance and dynamic error correction information to the Internet of Things server; a demonstration module, used to demonstrate the corresponding process operation AR video according to the demonstration signal, and guide the operator to use the torque gun to perform tightening operations; a recording module, used to record the operation performance and dynamic error correction information; a display module, used to display the operation performance and dynamic error correction information in digital reports; the acquisition module, the parsing module, and the display module are located in the Internet of Things server, the identification module and the demonstration module are located in the AR glasses, the Internet of Things server, the AR glasses, and the torque gun all have the transmission module, and the AR glasses and the torque gun all have the recording module.

[0015] Preferably, the acquisition module obtains the unique machine code of the vehicle to be assembled by scanning the SN barcode on the vehicle to be assembled passing through the PBS roller with a barcode reader; the parsing module parses the unique machine code according to the unique machine code parsing rules stored in advance to obtain the model of the vehicle to be assembled.

[0016] Preferably, it also includes a conversion module and a first retrieval module, the conversion module is used to convert the model information of the vehicle to be assembled into a demonstration signal corresponding to the model information, the retrieval module is used to retrieve the parameter information of the torque gun according to the model information of the vehicle to be assembled, and the conversion module and the first retrieval module are located in the Internet of Things server.

[0017] Preferably, the recognition module identifies the positioning tag of the workbench and determines the demonstration coordinates of the process operation AR video. The system also includes a second calling module for calling the corresponding process operation AR video according to the demonstration signal. The demonstration module combines the demonstration coordinates and loads a 3D somatosensory camera to capture and analyze the operating actions of the operator to perform real-time linkage demonstration of the process operation AR video, thereby guiding the operator to use a torque gun to perform actual tightening operations. The second calling module is located in the AR glasses.

[0018] Preferably, "the recording module records the position data of the fasteners tightened on the vehicle to be assembled, the reminder and guidance information for the operator's erroneous actions, and the final actual torque and the final actual angle.

[0019] The invention provides an AR technology-based fastener tightening operation auxiliary method with the following beneficial effects: the Internet of Things server obtains and analyzes the unique machine code of the vehicle to be assembled, and can analyze and store the model information of the vehicle to be assembled through the analysis rules after obtaining the unique machine code of the vehicle to be assembled, so as to provide the original judgment data basis for the subsequent AR glasses to select which process video to demonstrate; the Internet of Things server sends the corresponding demonstration signal and parameter information to the AR glasses and the torque gun respectively according to the analysis results, and provides the signal basis for the AR glasses to select the process video by means of output signal, and completes the intelligent adjustment of the torque gun by outputting the parameter information, so as to provide the subsequent operators with direct judgment data basis for the subsequent operators The AR glasses can identify the workbench scene, demonstrate the corresponding process operation according to the demonstration signal, and use the AR video to guide the operator to use the torque gun to perform the tightening operation. By identifying and locating the workbench, the demonstrated process video is integrated with the current actual working environment, and then the AR video is used to guide the operator to use the torque gun to complete the fastener tightening operation. The AR glasses and torque gun record the operation performance and dynamic error correction information, and transmit them to the IoT server for digital report display. During the operation, the actual fastener position and torque data of the operation are recorded by AR glasses and torque guns, transmitted to the IoT server in real time, and displayed digitally on the display. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without creative work:

[0021] Figure 1 A schematic diagram of a process flow of a fastener tightening operation assisting method based on AR technology provided by one embodiment of the present invention;

[0022] Figure 2 A schematic diagram of a module of a fastener tightening operation auxiliary system based on AR technology provided by one embodiment of the present invention;

[0023] Figure 3 A schematic diagram showing a reminder warning message for a conventional fastener tightening operation;

[0024] Figure 4 A schematic diagram showing a comparison of correction reminders for an AR demonstration operation provided by an embodiment of the present invention;

[0025] Figure 5 Schematic diagram of an AR demonstration component selection auxiliary guidance provided by an embodiment of the present invention Figure 1 ;

[0026] Figure 6 Schematic diagram of an AR demonstration component selection auxiliary guidance provided by an embodiment of the present invention Figure 2 ;

[0027] Figure 7 A schematic diagram of an auxiliary guidance for selecting an AR demonstration operation position provided by an embodiment of the present invention;

[0028] Figure 8 A schematic diagram of an AR demonstration operation result feedback reminder auxiliary guidance provided by an embodiment of the present invention; DETAILED DESCRIPTION

[0029] In order to facilitate understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Typical embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0031] The general idea of ​​the present invention is to provide a fastener tightening operation assistance method based on AR technology to address the shortcomings of the prior art that the operating efficiency and accuracy of operators in fastener tightening operations cannot be improved and the operation auxiliary guidance is not intelligent enough. The Internet of Things server in the present invention obtains and parses the unique machine code of the vehicle to be assembled. After obtaining the unique machine code of the vehicle to be assembled, the model information of the vehicle to be assembled can be parsed by parsing rules, providing original judgment data basis for the subsequent AR glasses to select which process video to demonstrate; the Internet of Things server sends corresponding demonstration signals and parameter information to the AR glasses and the torque gun according to the parsing results, provides signal basis for the AR glasses to select the process video by outputting signals, and completes intelligent adjustment of the torque gun by outputting parameter information, which provides convenience for subsequent operators to directly operate; the AR glasses recognize the workbench scene, and guide the operators to use the torque gun to perform tightening operations according to the demonstration signal. Through the identification and positioning of the workbench, the demonstrated process video is integrated with the current actual working environment, and then the AR video guides the operators to use the torque gun to complete the fastener tightening operation; the AR glasses and the torque gun record the operation performance and dynamic error correction information, and transmit them to the Internet of Things server for digital report display. During the operation, the fastener position and torque data of the actual operation are recorded by the AR glasses and the torque gun, transmitted to the Internet of Things server in real time, and displayed digitally on the display.

[0032] In order to better understand the above-mentioned technical scheme, the above-mentioned technical scheme will be described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical scheme of the present application, rather than limitations on the technical scheme of the present application. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0033] Reference Figure 1 , Figure 1 The figure is a flow chart of a fastener tightening operation assisting method based on AR technology provided by an embodiment of the present invention. In this embodiment, a fastener tightening operation assisting method based on AR technology includes:

[0034] Step S100: The IoT server obtains and parses the unique machine code of the vehicle to be assembled.

[0035] After the IoT server obtains the unique machine code of the vehicle to be assembled, it parses the unique machine code through the parsing rules stored in the server to obtain the model information of the vehicle to be assembled. Knowing the model of the vehicle to be assembled, it can send out the corresponding signal to provide the original judgment data basis for the subsequent AR glasses to select which process video to demonstrate. AR technology is a technology that cleverly integrates virtual information with the real world. It widely uses multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, sensing and other technical means to simulate computer-generated text, images, three-dimensional models, music, videos and other virtual information, and apply them to the real world. The two types of information complement each other, thereby achieving "enhancement" of the real world.

[0036] The Internet of Things server supports connecting and collecting data from PLCs, torque wrenches, sensors, instruments, robots, machine tools, and various controllers. It has data collection, protocol analysis, and edge computing, and transmits data through a variety of wired or wireless communication methods. The fastener tightening operation of the present invention is mainly for the bolt tightening operation of automobile assembly. Before the assembly, the car is affixed with a barcode with the unique machine code of the car. This requires describing and reading the barcode containing the unique machine code to obtain the unique machine code of the vehicle to be assembled. It can be read by a barcode reader. The function of the barcode reader is the same as that of a scanning gun. It scans the barcode through infrared rays. The barcode reader is used as a collection device. It is connected to the Internet of Things server through wireless network communication. It transmits the scanned unique machine code data information to the Internet of Things server for processing, thereby obtaining the model information of the vehicle to be assembled, providing a basis for subsequent control.

[0037] Specifically, step S100 includes: a barcode reader scans the SN barcode on the vehicle to be assembled passing through the PBS roller, and transmits the unique machine code information contained in the SN barcode to the Internet of Things server. The Internet of Things server parses the unique machine code according to the unique machine code parsing rules stored in advance to obtain the model information of the vehicle to be assembled.

[0038] The painted vehicle is transported to the final assembly area via the PBS roller conveyor. When the vehicle passes through the PBS roller conveyor, the barcode reader installed on the PBS roller conveyor line scans and reads the SN barcode of the unique machine code on the vehicle. The barcode reader translates the unique machine code information of the vehicle and transmits it to the IoT server. The IoT service receives the unique machine code information and retrieves the code parsing rules stored in advance. The unique machine code is parsed according to the rules to obtain relevant information of the vehicle, such as the unique machine code is 20230. 9150102030008, the parsing rule is that the first 8 digits of the code are the year, month and day of production, the 9th to 10th digits are the production plant number, such as 01 for the South China plant, the 11th to 12th digits are the production line number, such as 02 for the 2# production line, the 13th to 14th digits are the model information, such as 03 for the BYD-TF series, the 15th to 18th digits represent the serial number of units produced that day, such as 00008 for the 8th unit. The IoT server parses the unique machine code according to the parsing rules to obtain the model information of the vehicle.

[0039] Step S200: The IoT server sends corresponding demonstration signals and parameter information to the AR glasses and torque gun respectively according to the analysis results.

[0040] After parsing the vehicle model information, it is necessary to provide a signal basis for the AR glasses to select the process video by means of output signals. The torque gun is also intelligently adjusted through the output parameter information to facilitate direct operations for subsequent operators. The AR glasses recognize the output signals and perform corresponding process video demonstrations.

[0041] Specifically, the IoT server converts and outputs a demonstration signal corresponding to the model information of the vehicle to be assembled, and at the same time retrieves the parameter information of the torque gun according to the model information of the vehicle to be assembled, and transmits it to the AR glasses and the torque gun through wireless communication. The torque gun receives the parameter information and automatically adjusts the torque.

[0042] After the IoT server parses and obtains the model information of the vehicle to be assembled, it internally outputs a demonstration signal corresponding to the model information through the IO module input and output conversion. The demonstration signal is the basis for the start signal of the AR glasses to demonstrate which process video. For example, there are 6 models of vehicles to be assembled, and the IoT server corresponds to these 6 models and outputs six demonstration signals with different information. For example, the first model information corresponds to the input of a demonstration signal containing 1. The AR glasses have a Windows system with a control program installed. The control program interprets the demonstration signal and selects the process video of the fixed workstation corresponding to the car model stored in the AR glasses in advance for demonstration. The operator wears the AR glasses to watch the demonstration content that appears on the AR glasses, thereby guiding the operator to perform accurate operations. At the same time, the IoT server also retrieves the operating torque parameters of the model car according to the model information and transmits them to the torque gun. The torque gun has its own controller, which receives the torque parameters and automatically sets them to automatically adjust the torque gun to the torque parameter gear.

[0043] Step S300: The AR glasses identify the workbench scene and demonstrate the corresponding process operation AR video according to the demonstration signal to guide the operator to use the torque gun to perform tightening operations.

[0044] AR glasses can identify and locate the workbench, allowing the demonstrated process video to be integrated with the current actual working environment, and then guide the operator to use the torque gun to complete the fastener tightening operation through AR video.

[0045] The video content demonstrated by AR glasses needs to be integrated with the actual work scene in order to assist and guide the workers to complete the actual work operations through the demonstration content. In order to accurately integrate the demonstration video with the actual workbench, AR glasses are needed to locate and identify the workbench. In this way, through the demonstration process video, the workers can see both the actual work scene and the process video in the AR glasses and complete the work.

[0046] Specifically, the AR glasses identify the positioning label of the workbench, determine the demonstration coordinates of the process operation AR video, call up the corresponding process operation AR video according to the demonstration signal, and demonstrate the process operation AR video in combination with the demonstration coordinates. At the same time, the 3D somatosensory camera captures and analyzes the operating actions of the operators, and transmits them to the AR glasses and loads them into the process operation AR video for real-time linkage demonstration, guiding the operators to use the torque gun to perform actual tightening operations.

[0047] There is a label on the workbench, which can be used to determine the workstation process and AR video positioning. AR glasses can obtain the current workstation process of the workbench by scanning the label on the workbench, and can also determine the positioning coordinates of the current workbench. After obtaining the current workstation information, AR glasses compare it with the process video stored in advance. After confirming that it is correct, AR glasses call up the process video of the corresponding model information according to the received demonstration signal. Through positioning coordinates, the process video demonstration is integrated with the operation scene seen through AR. At the same time, a 3D somatosensory camera is also installed in the work area. The 3D somatosensory camera is a camera device that can sense and capture human movements. It uses depth perception technology, such as infrared sensors, structured light or time-of-flight technology, to obtain the three-dimensional position, skeletal posture and movement information of the human body in real time. Unlike traditional cameras, 3D somatosensory cameras can not only capture the appearance image of the human body, but also analyze the depth information of the human body, thereby realizing the tracking and recognition of human movements. Through these technologies, 3D somatosensory cameras can capture and analyze human movements in real time without any physical contact, and realize interaction with virtual reality, augmented reality, games and other applications. The 3D somatosensory camera captures the operator's operating actions and analyzes these operating actions through image recognition technology to obtain the depth information of human body movements. Image recognition technology can use the target inspection type algorithm to mark the location of objects in the image and identify the type. Common target inspection algorithms include RCNN series, YOLD, SSD, etc. The 3D somatosensory camera communicates with the IoT server and AR glasses through wireless communication. After capturing and analyzing the operator's operating actions, the 3D somatosensory camera transmits the depth information of the operating actions to the AR glasses, loads it into the currently demonstrated process video, and performs auxiliary linkage demonstration to guide the operator to use the torque gun for actual tightening operations.

[0048] like Figure 4 As shown, Figure 4 A schematic diagram of a comparison of correction reminders for AR demonstration operations provided by an embodiment of the present invention; the AR glasses recognize the initial assembly scene of a vehicle of model A, determine the current assembly process that needs to be completed, and mark the specific screw tightening sequence of the current component on the display screen of the AR glasses, such as tightening the screws from top to bottom. When it is recognized that the operator is aiming the torque gun at other holes, a red cross reminder will appear on the AR screen, and the correct text guidance and the correct hole position mark will appear at the same time; Figure 5 As shown, Figure 5 Schematic diagram of an AR demonstration component selection auxiliary guidance provided by an embodiment of the present invention Figure 1When the operator reaches out to take the fastener, the AR glasses will identify the type of parts in the parts box and compare them with the types of parts that match the process. If a mismatch is found, a red reminder pattern and a cross mark will appear in the part box area, and the correct part box area will be covered with color and a check mark will be marked to indicate correctness; Figure 6 As shown, Figure 6 Schematic diagram of an AR demonstration component selection auxiliary guidance provided by an embodiment of the present invention Figure 2 ; When other parts are mixed in the parts box, when the operator takes the parts out of the parts box, the AR glasses will measure the parts. If it is found that the size of the parts is different from the standard size of the parts in the parts box, a cross prompt will appear, and a text explanation will appear, such as: "Wrong selection, the current size is 5*5mm, and the square radius of 6mm should be selected"; Figure 7 As shown, Figure 7 A schematic diagram of an AR demonstration of auxiliary guidance for selecting a work position provided by an embodiment of the present invention; when selecting a process position for operation, the system can accurately locate the target position and guide the installation position of the next step in a highlight manner. If the operator installs the screw in the wrong position, an alarm will be issued for correction. For example, if the operator should install the screw in the first position but installs it in the second position, a voice, text, and animation alarm will be issued at this time; Figure 8 As shown, Figure 8 A schematic diagram of an AR demonstration of operation result feedback reminder auxiliary guidance provided by an embodiment of the present invention; the system can connect to the tool sensor value to help workers to feedback the operation results in real time in AR glasses and give reminders, such as: the operation torque of the operator in this step can be viewed in real time through AR glasses, and voice, text, and animation prompts are given for insufficient or over-prompted. Traditional tightening operation instructions such as Figure 3 As shown, Figure 3 This is a schematic diagram of the warning information displayed during the tightening operation of traditional fasteners. The instructions for the operation are generally only text prompts, and there are few instructions for operation. If the operation is wrong, there will be a warning at most, but no correction instructions. It is often necessary to repeat the operation repeatedly to confirm whether it is qualified, which has the disadvantages of low efficiency and easy error.

[0049] Step S400: AR glasses and torque gun record operation performance and dynamic error correction information, and transmit them to the IoT server for digital report display.

[0050] During the operation, the actual fastener position and torque data are recorded through AR glasses and torque guns, transmitted to the IoT server in real time, and displayed digitally on the monitor.

[0051] Specifically, step S400 includes: the AR glasses record the position data of the fasteners tightened on the vehicle to be assembled and the reminder and guidance information for the operators' wrong actions; the torque gun records the final actual torque and the final actual angle of the fasteners tightened on the vehicle to be assembled; the AR glasses and the torque gun transmit the recorded data to the Internet of Things server via wireless communication; the Internet of Things server processes the recorded data into digital reports and displays them on a display.

[0052] After the operator has finished the work, the operation process or operation process needs to be optimized. For further improvement, it is usually necessary to have data during the operation process, such as torque data. In the present invention, both AR glasses and torque guns have a recording function. AR glasses record the position of the fastener tightening, and the torque gun records the torque and angle at the position. AR glasses and torque guns transmit the position data and torque and angle data to the Internet of Things server in real time. The Internet of Things server receives the position information and the torque information and angle information of the position, and integrates and processes these data according to the receiving time, and finally forms a data report, which is finally displayed on a display connected to the Internet of Things server.

[0053] Based on the above method, the Internet of Things server in the present invention obtains and parses the unique machine code of the vehicle to be assembled. After obtaining the unique machine code of the vehicle to be assembled, the model information of the vehicle to be assembled can be parsed by parsing rules, providing original judgment data basis for the subsequent AR glasses to select which process video to demonstrate; the Internet of Things server sends corresponding demonstration signals and parameter information to the AR glasses and the torque gun according to the parsing results, provides signal basis for the AR glasses to select the process video by outputting signals, and completes intelligent adjustment of the torque gun by outputting parameter information, which provides convenience for subsequent operators to directly operate; the AR glasses recognize the workbench scene, and guide the operators to use the torque gun to tighten the work according to the demonstration signal. Through the identification and positioning of the workbench, the demonstrated process video is integrated with the current actual working environment, and then the AR video guides the operators to use the torque gun to complete the fastener tightening operation; the AR glasses and the torque gun record the operation performance and dynamic error correction information, and transmit them to the Internet of Things server for digital report display. During the operation, the actual operation fastener position and torque data are recorded by the AR glasses and the torque gun, transmitted to the Internet of Things server in real time, and displayed digitally on the display.

[0054] Accordingly, the present invention also provides a fastener tightening operation auxiliary system based on AR technology, referring to Figure 2 , Figure 2The figure shows a schematic diagram of the module structure of a fastener tightening operation auxiliary system based on AR technology provided by an embodiment of the present invention. The system assists the fastener tightening operation through the fastener tightening operation auxiliary method based on AR technology described above. The system includes an acquisition module for acquiring a unique machine code of a vehicle to be assembled; a parsing module for parsing the unique machine code of the vehicle to be assembled; an identification module for identifying a workbench scene; a transmission module for sending a demonstration signal and parameter information corresponding to the parsing result to the AR glasses and the torque gun respectively, and for transmitting the operation performance and dynamic error correction information to the Internet of Things server; a demonstration module for demonstrating the corresponding process operation AR video according to the demonstration signal to guide the operator to use the torque gun for tightening operation; a recording module for recording the operation performance and dynamic error correction information; a display module for displaying the operation performance and dynamic error correction information in a digital report; the acquisition module, the parsing module, and the display module are located in the Internet of Things server, the identification module and the demonstration module are located in the AR glasses, the Internet of Things server, the AR glasses, and the torque gun all have the transmission module, and the AR glasses and the torque gun all have the recording module.

[0055] Preferably, the acquisition module obtains the unique machine code of the vehicle to be assembled by scanning the SN barcode on the vehicle to be assembled passing through the PBS roller with a barcode reader; the parsing module parses the unique machine code according to the unique machine code parsing rules stored in advance to obtain the model of the vehicle to be assembled.

[0056] Preferably, it also includes a conversion module and a first retrieval module, the conversion module is used to convert the model information of the vehicle to be assembled into a demonstration signal corresponding to the model information, the retrieval module is used to retrieve the parameter information of the torque gun according to the model information of the vehicle to be assembled, and the conversion module and the first retrieval module are located in the Internet of Things server.

[0057] Preferably, the recognition module identifies the positioning tag of the workbench and determines the demonstration coordinates of the process operation AR video. The system also includes a second calling module for calling the corresponding process operation AR video according to the demonstration signal. The demonstration module combines the demonstration coordinates and loads a 3D somatosensory camera to capture and analyze the operating actions of the operator to perform real-time linkage demonstration of the process operation AR video, thereby guiding the operator to use a torque gun to perform actual tightening operations. The second calling module is located in the AR glasses.

[0058] Preferably, "the recording module records the position data of the fasteners tightened on the vehicle to be assembled, the reminder and guidance information for the operator's erroneous actions, and the final actual torque and the final actual angle.

[0059] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0060] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0061] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0062] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0063] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0064] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

Claims

1. A fastener tightening operation auxiliary method based on AR technology, characterized in that: The following steps are involved: Step S100: The IoT server obtains and parses the unique machine code of the vehicle to be assembled; Step S200: The IoT server sends corresponding demonstration signals and parameter information to the AR glasses and the torque gun respectively according to the analysis results; Step S300: The AR glasses recognize the workbench scene, and demonstrate the corresponding process operation AR video according to the demonstration signal to guide the operator to use the torque gun to perform the tightening operation; Step S400: AR glasses and torque gun record operation performance and dynamic error correction information, and transmit them to the IoT server for digital report display.

2. The fastener tightening operation assisting method based on AR technology according to claim 1, characterized in that: The step S100 includes: The barcode reader scans the SN barcode on the vehicle to be assembled passing through the PBS roller, and transmits the unique machine code information contained in the SN barcode to the IoT server. The IoT server parses the unique machine code according to the unique machine code parsing rules stored in advance to obtain the model information of the vehicle to be assembled.

3. The fastener tightening operation assisting method based on AR technology according to claim 2, characterized in that: The step S200 includes: The IoT server converts and outputs a demonstration signal corresponding to the model information of the vehicle to be assembled. At the same time, it retrieves the parameter information of the torque gun according to the model information of the vehicle to be assembled, and transmits it to the AR glasses and the torque gun through wireless communication. The torque gun receives the parameter information and automatically adjusts the torque.

4. The fastener tightening operation assisting method based on AR technology according to claim 3, characterized in that: The step S300 includes: The AR glasses recognize the positioning label of the workbench, determine the demonstration coordinates of the process operation AR video, call up the corresponding process operation AR video according to the demonstration signal, and demonstrate the process operation AR video in combination with the demonstration coordinates. At the same time, the 3D somatosensory camera captures and analyzes the operating actions of the operators, and transmits them to the AR glasses and loads them into the process operation AR video for real-time linkage demonstration, guiding the operators to use the torque gun to perform actual tightening operations.

5. The fastener tightening operation assisting method based on AR technology according to claim 4, characterized in that: The step S400 includes: AR glasses record the position data of fasteners tightened on the vehicle to be assembled and the reminder and guidance information for operators' incorrect actions. The torque gun records the final actual torque and final actual angle of fasteners tightened on the vehicle to be assembled. AR glasses and torque gun transmit the recorded data to the IoT server via wireless communication. The IoT server processes the recorded data into digital reports and displays them on the monitor.

6. A fastener tightening operation auxiliary system based on AR technology, characterized in that: include: An acquisition module, used to acquire a unique machine code of a vehicle to be assembled; A parsing module, used to parse the unique machine code of the vehicle to be assembled; Identification module, used to identify the workbench scene; The transmission module is used to send the demonstration signal and parameter information corresponding to the analysis result to the AR glasses and the torque gun respectively, and to transmit the operation performance and dynamic error correction information to the IoT server; Demonstration module, used to demonstrate the corresponding process operation AR video according to the demonstration signal, and guide the operator to use the torque gun to perform tightening operations; Recording module, used to record operation performance and dynamic error correction information; Display module, used to display digital reports of operation performance and dynamic error correction information; The acquisition module, the analysis module, and the display module are located in the Internet of Things server, the identification module and the demonstration module are located in the AR glasses, the Internet of Things server, the AR glasses, and the torque gun all have the transmission module, and the AR glasses and the torque gun all have the recording module.

7. The fastener tightening operation auxiliary system based on AR technology according to claim 6, characterized in that: The acquisition module obtains the unique machine code of the vehicle to be assembled by scanning the SN barcode on the vehicle to be assembled passing through the PBS roller through a barcode reader; the parsing module parses the unique machine code according to the unique machine code parsing rules stored in advance to obtain the model of the vehicle to be assembled.

8. The fastener tightening operation auxiliary system based on AR technology as claimed in claim 7, characterized in that: It also includes a conversion module and a first retrieval module, wherein the conversion module is used to convert the model information of the vehicle to be assembled into a demonstration signal corresponding to the model information, and the retrieval module is used to retrieve parameter information of the torque gun according to the model information of the vehicle to be assembled, and the conversion module and the first retrieval module are located in the Internet of Things server.

9. The fastener tightening operation auxiliary system based on AR technology as claimed in claim 8, characterized in that: The recognition module identifies the positioning tag of the workbench and determines the demonstration coordinates of the process operation AR video. The system also includes a second retrieval module, which is used to retrieve the corresponding process operation AR video according to the demonstration signal. The demonstration module combines the demonstration coordinates and loads a 3D somatosensory camera to capture and analyze the operating actions of the operator to perform real-time linkage demonstration of the process operation AR video, and guides the operator to use a torque gun to perform actual tightening operations. The second retrieval module is located in the AR glasses.

10. The fastener tightening operation auxiliary system based on AR technology according to claim 9, characterized in that: The recording module records the position data of the fasteners tightened on the vehicle to be assembled, the reminder and guidance information for the operator's wrong actions, and the final actual torque and the final actual angle.

Images