Tightening method
The integration of augmented and virtual reality techniques for precise 3D tool localization and component identification addresses the limitations of existing solutions, enhancing operator guidance and reducing errors in mechanical equipment tightening operations.
Patent Information
- Application Number
- PCT/FR2025/050427
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-08-01
- Filing Date
- 2025-05-20
- Publication Date
- 2026-02-05
AI Technical Summary
Existing smart workstation solutions for mechanical equipment, such as those used in aircraft maintenance, struggle with precise 3D component identification and tool localization, especially when operated by human operators, due to limitations in 2D imaging, visibility issues, and accuracy requirements that current technologies cannot meet.
A method combining augmented and virtual reality to calculate the distance between a clamping tool and the component to be clamped, using a digital model and trackers to ensure precise positioning and orientation, enabling continuous tool visibility and operator assistance through augmented reality displays.
Reduces human error and operation time while ensuring the quality of tightening operations by providing precise guidance and traceability, with an accuracy of less than 10 mm throughout the process.
Smart Images

Figure FR2025050427_05022026_PF_FP_ABST
Abstract
Description
DESCRIPTION TITLE: Clamping Method TECHNICAL FIELD OF THE INVENTION
[0001] The technical field of the invention is that of tightening operations carried out by an operator in the context of production and maintenance activities of mechanical equipment, in particular mechanical equipment of an aircraft.
[0002] The present invention relates to a method of clamping one or more elements to be clamped of a mechanical equipment. TECHNOLOGICAL BACKGROUND OF THE INVENTION
[0003] Mechanical equipment, such as aircraft mechanical equipment, sometimes requires lengthy and complex operations performed as part of production and maintenance activities. These operations are carried out by one or more operators. Therefore, the operator must follow precise instructions and rules to ensure that the mechanical equipment is in perfect working order after the intervention. For example, during a tightening operation, the operator must use the correct tightening tool for each component being tightened. Furthermore, the operator must apply a specific tightening torque within a defined tolerance range to each component, such as a screw. Finally, when the tightening operation involves several components, there is sometimes a specific order to follow for tightening them (for example, for cross-tightening).Thus, in order to guarantee the quality of a tightening operation, it is necessary to tighten the correct element to be tightened, with the correct tightening tool, at the right time and with the correct tightening torque.
[0004] To assist the operator during operations on mechanical equipment, various solutions, called intelligent workstation solutions, exist.
[0005] One category of smart workstation solutions, such as the one offered by Arkite, uses a camera perpendicular to the work surface to observe the operator's actions. These solutions can, for example, guide the operator step by step during the operation. However, recognizing the components to be clamped requires positioning the parts to be assembled very precisely and repeatably on the work surface. These solutions are therefore applicable to small mechanical equipment that can be assembled on a work surface such as a table. Furthermore, the identification of the components to be clamped and the clamping tools can only be performed in two dimensions (2D), parallel to the work surface. Thus, these systems cannot guide assembly in three dimensions (3D) on complex and / or large equipment.
[0006] Another category of smart workstation solutions, such as those offered by Inbolt, specializes in 3D tool localization, particularly for clamping operations. These solutions consist of an instrumented housing mounted on the clamping tool. The housing contains 3D vision sensors and inertial sensors. These solutions use the data from the housing to locate the tool relative to the equipment. They also utilize a digital model of the mechanical equipment—that is, a 3D digital model that accurately represents the mechanical equipment geometrically—to feed a 3D shape recognition algorithm to locate the housing relative to the equipment. However, to enable the localization of the clamping tool relative to the equipment, it is necessary to position the housing so that there are no obstacles between the housing and the mechanical equipment.Furthermore, the clamping tool's movements must be stable, slow, and controlled to maintain accurate positioning information relative to the mechanical equipment. This solution is therefore implemented in clamping operations that can be automated with robotic arms but is not suitable for use by a human operator.
[0007] A final category of smart workstation solutions, such as the one offered by Diota, uses augmented reality to assist in the assembly or inspection of mechanical equipment. These solutions use a 2D camera to film the work area. From the video stream obtained with the 2D camera, an algorithm, known as a "model-based augmented reality algorithm," can locate a digital model of the equipment within the augmented reality environment by determining the position and orientation of the mechanical equipment relative to the 2D camera. To determine the position and orientation of the mechanical equipment relative to the 2D camera, the model-based augmented reality algorithm analyzes the video stream to determine the position and The orientation of areas and / or edges of mechanical equipment for any image in the video stream. Some solutions in this third category attempt to locate mechanical equipment and a tool within the same augmented reality environment, for example, to assist operators during tightening operations. However, in real-world situations, the tightening tool being used is barely visible to the 2D camera during the operation, making it difficult, if not impossible, to use augmented reality to locate it. Therefore, the loss of tool location information by solutions in this third category prevents effective operator assistance during tightening operations.
[0008] Finally, there are solutions for locating components in industrial environments, designed to equip tools with trackers for precise location within workshops, for example. These solutions utilize technologies capable of detecting components over long distances, enabling them to be located within workshops or on industrial sites. However, these solutions offer an accuracy of approximately ten centimeters. They do not achieve the precision required for guiding tightening operations, which is typically within tens of millimeters.
[0009] There is therefore a need to provide a method for clamping a component of mechanical equipment that at least partially resolves the drawbacks of currently known solutions. SUMMARY OF THE INVENTION
[0010] The invention provides a solution to the problems mentioned above by assisting an operator during a tightening operation, performed using a clamping tool, on a component of mechanical equipment. This assistance is provided by using a distance between the clamping tool and the component to be clamped. This distance is calculated in the method according to the invention by combining augmented reality and virtual reality techniques. The use of virtual reality to determine the position of the clamping tool ensures, in particular, continuous visibility of the clamping tool within the virtual reality environment. Thus, the position and orientation of the clamping tool can be reliably determined at any time during the tightening operation.Finally, the use of augmented reality to assist the operator is particularly suited to maintenance and production operations of mechanical equipment since. The operator can easily view the screen displaying assistance information in the augmented reality environment during the tightening operation.
[0011] One aspect of the invention relates to a method for clamping a component of mechanical equipment, the clamping operation being carried out by an operator using a clamping tool in a work area, the method comprising the following steps: Obtaining a digital model of the mechanical equipment, Obtaining an augmented reality environment, displayed on a computer screen visible to the operator during the tightening operation, and a virtual reality environment, Determination of the position and orientation of the digital model in the augmented reality environment, the determination of the position and orientation of the digital model in the augmented reality environment being carried out: by filming the mechanical equipment in the work area during the tightening operation using a 2D camera, and by spatially superimposing the work area and the augmented reality environment, the spatial superposition of the work area and the augmented reality environment being carried out with a model-based augmented reality algorithm using the film of the mechanical equipment in the work area, Determining the position and orientation of the clamping tool within the virtual reality environment, and Calculating the distance between the clamping tool and the element to be clamped, the calculation of the distance including sub-steps of: Calculation of the position of the element to be clamped in the augmented reality environment based on the determined position and orientation of the digital model, Achieving a common digital environment by spatially superimposing the augmented reality environment and the virtual reality environment, Calculate the position of the clamping tool and the element to be clamped in the common digital environment, and Operator assistance during the tightening operation, the assistance being: carried out via the display of the augmented reality environment on the computer screen visible to the operator, carried out via the control of the tightening tool, and dependent on the calculated distance between the tightening tool and the element to be tightened.
[0012] Thanks to the invention, the risk of errors due to human error during a tightening operation is reduced. Furthermore, the time required for a tightening operation is also reduced. Finally, the process according to the invention improves traceability to guarantee the quality of tightening operations performed on mechanical equipment.
[0013] In addition to the features mentioned in the preceding paragraph, the method according to one aspect of the invention may have one or more of the following complementary features, considered individually or in all technically possible combinations: an error in calculating the distance between the clamping tool and the element to be clamped is less than 10 or 20 millimeters throughout the clamping operation; a display, in the augmented reality environment, of information on the location of the element to be clamped of the mechanical equipment; the clamping tool is configurable by the computer; a configuration of the clamping tool including a setting of a target clamping torque applicable by the clamping tool; the operator assistance (160) includes the configuration of the target clamping torque of the clamping tool as a function of the calculated distance (150) between the clamping tool and the element to be clamped. A predetermined tightening torque value is applied to the element to be tightened. The clamping tool is adapted to send to the computer a clamping torque induced by the clamping tool during the clamping operation, and Operator assistance includes: determining the end of a tightening operation for the component to be tightened by determining that the tightening torque induced by the tightening tool during the operation is within a predetermined tightening torque tolerance range, and displaying information, in the augmented reality environment, indicating that the tightening operation for the component to be tightened is complete and whether this tightening is compliant or not; the tightening operation consists of tightening an additional component of the mechanical equipment according to a tightening sequence that is at least partially predetermined; and operator assistance includes displaying, in the augmented reality environment, information indicating the next component to be tightened from among the components to be tightened, respecting the tightening sequence; the mechanical equipment is equipment of an aircraft, such as: an aircraft engine, a landing gear,an engine nacelle, a transmission gearbox, and cabin equipment from an aircraft.
[0014] Another aspect of the invention relates to a clamping system comprising A computer configured to implement the process according to the invention, A screen connected to the computer and adapted to display the augmented reality environment, A clamping tool: computer configurable, a clamping tool configuration including a setting of a target tightening torque applicable by the clamping tool, adapted to send to the computer a tightening torque induced by the clamping tool during the clamping operation, An augmented reality device comprising a 2D camera adapted to film a work area in which the mechanical equipment and the element to be clamped are included during the clamping operation, A virtual reality device comprising at least one tracker suitable for locating the clamping tool and one tracker suitable for locating the camera.
[0015] The invention and its various applications will be better understood by reading the following description and examining the accompanying figures. BRIEF DESCRIPTION OF THE FIGURES
[0016] The figures are presented for illustrative purposes only and are in no way limiting to the invention. Figure 1 shows a synoptic diagram illustrating the steps of an example of the method for assisting a clamping operation of a component to be clamped of a mechanical equipment according to the invention. Figure 2 shows a schematic representation of an example of a clamping system for mechanical equipment according to the invention. Figure 3 is a synoptic diagram illustrating the sub-steps of an example of the step of calculating a distance between the clamping tool and the element to be clamped according to the invention. Figure 4 is a synoptic diagram illustrating the sub-steps of an example of the operator assistance step according to the invention. DETAILED DESCRIPTION
[0017] Unless otherwise specified, the same element appearing on different figures has a unique reference.
[0018] Figure 1 is a synoptic diagram illustrating the steps of an example of process 100 according to the invention.
[0019] Process 100 is a tightening method for one or more components with a threaded connection, such as a screw, a grease fitting, or a screw-on plug. For simplicity, the following description will detail an example of Process 100 with a single component. However, Process 100 can, of course, be applied to multiple components. The component being tightened using Process 100 is tightened onto a mechanical device by an operator. To perform the tightening operation, the operator uses a tightening tool. During the tightening operation, the operator is assisted, and the steps of Process 100 are carried out while the operator tightens the component. In one example, the mechanical device is an aircraft component. For example, the mechanical device is one of the following:
[0020] an aircraft engine,
[0021] landing gear,
[0022] an engine nacelle,
[0023] a transmission box, and
[0024] aircraft cabin equipment such as seats.
[0025] Method 100 can be implemented by computer. By "implemented by computer," it is understood that all, or virtually all, of the steps in Method 100 are executed by at least one computer or processor, or some other similar system, such as a smartphone, tablet, or virtual or augmented reality headset. Thus, steps are performed by the computer, possibly fully or semi-automatically. In some examples, the triggering of at least some of the steps in these methods can be achieved through user-computer interaction. The level of user interaction required may depend on the level of automation expected and weighed against the need to implement the User preferences. In some examples, this level can be user-defined and / or predefined. User interaction can be performed using any device. When the computer includes a touchscreen or camera, user interaction can be performed with any part of the user's body, such as their hand or one or more fingers. The computer includes a screen suitable for displaying the augmented reality environment. This screen is visible to the operator during at least part of the clamping operation.
[0026] A typical example of a computer implementation of Process 100 involves running Process 100 with a system adapted for this purpose. The system may include a processor coupled with memory and a graphical user interface (GUI), the memory containing a computer program with instructions for implementing the process. The memory may also store a database. Memory is any hardware adapted for such storage, possibly comprising several distinct physical parts. In particular, the memory may store a digital model of the mechanical equipment. The digital model of the mechanical equipment accurately reproduces its outer casing. The term "accurate" here means that the digital model reproduces the outer casing of the mechanical equipment identically, with an error of less than 1, 5, or 10 millimeters.The digital model can also include all or some of the components to be tightened on the mechanical equipment. The memory can also contain a digital model of the tightening tool. This digital model of the tightening tool can then be optionally displayed in the augmented reality environment. The memory can also contain all the rules to be followed for the tightening operation. For example, the memory can contain, for each component to be tightened, the minimum tightening torque to be applied, as well as, complementaryly, the maximum tightening torque to be applied. It is worth noting that these minimum and maximum tightening torques can be different for each component during the tightening operation. The memory can also contain a tightening sequence to be followed. This stored tightening sequence can be complete within a strict, ordered sequence.A strict, ordered sequence dictates, in a predetermined manner, the position of each component to be tightened within the tightening order. The memorized tightening order can also be partial, within the framework of an ordered sequence. According to a business rule, a sequence ordered according to a business rule determines which elements are authorized to be tightened during the tightening operation, based on the business rules specific to the current operation, for example, cross tightening. Alternatively, the stored tightening order can be free, within the framework of a free sequence. A free sequence does not impose any tightening order. Finally, the memory can contain the tightening tool(s), such as one or more wrenches or screwdrivers, suitable for tightening each element.
[0027] An example of a system adapted to implement Process 100 is illustrated in Figure 2. More specifically, Figure 2 illustrates a possible implementation of such a system at an operator's workstation or in their work area during the implementation of Process 100. In Figure 2, solid-line arrows of type 250 represent a superposition relationship, dashed arrows of type 260 represent a location relationship, and dashed arrows of type 270 represent a distance relationship. The superposition relationship links two elements that are superimposed, i.e., have the same position and orientation, in their respective environments, using, for example, augmented reality algorithms or calibration of the transformation matrix to allow the transition between the two elements.For example, the type 250 arrow between the virtual reality tracker 212 and the 2D camera 202 means that the tracker 212 is superimposed, in the virtual reality digital environment 211, with the 2D camera 202 (which is in the work area). A location relationship links two elements that allow the calculation of the relative position and orientation between the two elements in the corresponding digital environment. For example, the type 260 arrow between the mechanical equipment 204 and the 2D camera 202 means that the mechanical equipment 204 is located relative to the 2D camera 202; that is, its position and orientation relative to the 2D camera 202 are determined in the work area. The distance relationship designates the gap between two physical elements that the process 100 allows to be calculated in order to assist the clamping operations.For example, the arrow of type 270 between the clamping element 205 and the connected clamping tool 206 means that the actual distance between the clamping element 205 and the connected clamping tool 206 is determined in the virtual reality digital environment 211.
[0028] The system, as illustrated in Figure 2, also includes at least one clamping tool. Each clamping element among the set of elements to be clamped can be compatible with one or more clamping tools. Thus, the operator may need to use several clamping tools during a clamping operation. The clamping tools can be connected. When connected, the clamping tool can be configured remotely by the computer implementing process 100. The clamping tool configuration can, for example, include setting a target clamping torque that the tool can apply. In other words, the computer can remotely configure the target clamping torque that the connected clamping tool can apply during a clamping operation. The target clamping torque is within a tolerance range, which defines the minimum and maximum clamping torques.The connected clamping tool can also send information about its use to the computer implementing process 100, such as the clamping torque applied or induced by the clamping tool during the clamping operation.
[0029] The system, as illustrated in Figure 2, further includes an augmented reality device. The augmented reality device includes at least one 2D camera, labeled 202 in Figure 2. The 2D camera 202 is adapted to film a work area containing the mechanical equipment 204 and the clamping element 205 throughout the clamping operation. The position and orientation of the clamping element 205 relative to the mechanical equipment 204 are predetermined. The digital model 203 of the mechanical equipment 204 may include a digital representation of the element to be clamped 205. The position and orientation of the digital equipment, relative to the 2D camera 202, can be determined using a model-based synchronization algorithm. It is worth noting that the mechanical equipment 204 can thus be moved during the tightening operation and as a result, the position and orientation of the digital model 203 will be updated in real time in the augmented reality digital environment 210. Elements 202 to 205 are included in the augmented reality digital environment 210.
[0030] The system, as illustrated in Figure 2, also includes a virtual reality device. The virtual reality device comprises at least two trackers, 209 and 212. The first tracker, 209, locates the clamping tool 206 within the virtual reality digital environment 211. The second tracker, 212, allows to locate the 2D camera 202 within the virtual reality digital environment 211. Thus, this tracker 212 can be used to synchronize the digital environments 210 and 211, i.e., to determine the position and orientation of all the elements of the augmented reality digital environment 210 within the virtual reality digital environment 211. In this example, the position and orientation of the 2D camera 202 can correspond to a first reference frame, the reference frame of the augmented reality digital environment 210. Therefore, thanks to the tracker 212, it is possible to determine the transformation matrix that allows us to go from this first reference frame to a second reference frame, the reference frame of the virtual reality environment 211. This transformation matrix is determined from the position and orientation of the tracker 212 within the second reference frame.Determining the position and orientation of the trackers 209 and 212 within the virtual reality environment can be achieved using one or more beacons 207. Preferably, four beacons 207 can be used and arranged horizontally at the four corners of the work area 201 to form a rectangle. The beacons 207 can, for example, be placed at a height of between 2 and 2.5 meters. These beacons 207 can also be positioned, as illustrated in Figure 2, outside the physical area 208 managed by the virtual reality digital environment 211. In the example shown in Figure 2, the operator 214 has their back to the 2D camera 202. Thus, the operator 214 can see the screen 213, on which a real-time rendering of the augmented reality environment 210 is displayed, while performing the clamping operation.Furthermore, in this configuration, the operator can see himself on the computer screen performing the tightening operation.
[0031] Step 110 of process 100 involves obtaining, for example by computer, the digital model of the mechanical equipment. The term "obtain" in this application may mean "generate" using the computer, "receive" by the computer, or "access" by the computer (when the digital model is already stored in the computer's memory).
[0032] Step 120 of process 100 involves, for example, obtaining an augmented reality environment using a computer. An augmented reality environment is a technology that overlays virtual information and objects onto the real world, usually through the use of devices such as screens. Smartphones, tablets, smart glasses, or augmented reality headsets. Augmented reality thus enriches the user experience by integrating interactive digital elements into their physical environment. In the example in Figure 2, the augmented reality environment is displayed on a computer screen 213. This screen 213 is visible to the operator 214 during the tightening operation. Operator assistance can therefore include the display of information and virtual objects within the augmented reality environment, which are easily visible to the operator throughout the tightening process.
[0033] Step 120 of process 100 also includes obtaining, for example by computer, a virtual reality environment. A virtual reality environment is an immersive computer simulation that recreates a realistic or fictional environment. It is worth noting that in process 100, the virtual reality environment does not need to be displayed. In fact, the virtual reality environment is used, in particular, to locate the clamping tool using, for example, a virtual reality tracker placed on the clamping tool.
[0034] An augmented reality environment can include a first reference frame. A virtual reality environment can include a second reference frame. Furthermore, a transformation matrix allowing the conversion from the second frame to the first frame can be known. For example, the transformation matrix can be calculated using a virtual reality tracker placed on the 2D camera of the augmented reality environment device. Thus, the position and orientation of the first reference frame, for example, the virtual reality tracker placed on the 2D camera, are determined in the second reference frame.
[0035] Step 130 of process 100 involves determining, for example by computer, the position and orientation of the digital model in the augmented reality environment. This determination is performed by superimposing the position and orientation of the mechanical equipment in the work area onto the position and orientation of the digital model in the augmented reality environment. The determination can be carried out continuously in real time throughout the clamping operation. Alternatively, it can be performed at a predetermined frequency, for example, 1, 5, or 10 hertz. Given that the position and orientation of the digital model in the reality environment The augmented reality environment is updated throughout the tightening operation. The mechanical equipment can be moved while remaining visible to the 2D camera 202 during the tightening process. This step 230 can be implemented by filming the work area, including the mechanical equipment, with the 2D camera during the tightening operation. Then, using a model-based augmented reality algorithm, the work area and the augmented reality environment are spatially superimposed. Thus, the position and orientation of the digital model in the augmented reality environment are synchronized with the position and orientation of the mechanical equipment in the work area.
[0036] Step 140 of process 100 involves determining, for example by a computer, the position and orientation of the clamping tool within the virtual reality environment. To determine the position and orientation of the clamping tool, a virtual reality tracker can be placed on it. The update frequency of the position and orientation of the virtual reality tracker, and therefore of the clamping tool, depends on the characteristics of the virtual reality device used. For example, the frequency can range from 10 to 300 hertz. It is worth noting that this step 140 may include calculating the position, and optionally even the orientation, of a part of interest of the clamping tool based on the position and orientation of the virtual reality tracker. The part of interest is, for example, the part in contact with the element to be clamped during clamping.The position of the part of interest can be calculated using a predetermined vector that defines the position of the part of interest relative to the virtual reality tracker in the second reference frame. When the orientation of the part of interest is also calculated, a transformation matrix can be used to define the position and orientation of the part of interest relative to the virtual reality tracker in the second reference frame.
[0037] A step 150 of the process 100 includes calculating, for example by computer, the distance between the clamping tool and the element to be clamped. This step 150 comprises three substeps. Figure 3 is a block diagram illustrating the steps of an example of step 150 according to the invention. The first substep 151 of step 150 consists of calculating the position of the element to be clamped in the augmented reality environment based on the position and orientation of the digital model determined in step 130. The calculation 151 of the position of the element to be clamped in The augmented reality environment can be achieved using at least one second predetermined vector defining the position, relative to the digital model, of the element to be clamped in the first reference frame.
[0038] The second substep, 152, consists of obtaining a common digital environment. This common digital environment is obtained by spatially superimposing the augmented reality environment and the virtual reality environment. The spatial superposition of the two environments can be achieved by determining, in the second reference frame, the position and orientation of the first reference frame. Alternatively, the spatial superposition of the two environments can be achieved by determining, in a common frame of reference, the position and orientation of both the first and second reference frames. Substep 152 thus generates a common digital environment, i.e., one that shares the same reference frame for both the augmented reality and virtual reality environments.Thus, the various calculations of positions and orientations, as well as the distance between the clamping tool and the element to be clamped, can be carried out in this common digital environment.
[0039] The third substep, 153, consists of calculating the position of the clamping tool and the clamping element in the shared digital environment, based on the position of the clamping element in the shared environment calculated in step 151. The calculation of the clamping element's position in the shared digital environment can be performed using a transformation matrix that allows conversion from the augmented reality environment to the shared digital environment, i.e., from the first reference frame to the frame of the shared digital environment. Similarly, the calculation of the clamping tool's position in the shared digital environment can be performed using a transformation matrix that allows conversion from the virtual reality environment to the shared digital environment, i.e., from the second reference frame to the frame of the shared digital environment.
[0040] Finally, since the position of the clamping tool and the position of the element to be clamped are known in the common digital environment, it is possible to calculate the distance between the clamping tool and the element to be clamped.
[0041] In an example consistent with the previous examples, the calculation error, performed in step 150, of the distance between the clamping tool and the workpiece is less than or equal to 10 millimeters throughout the entire clamping operation. In other words, the calculated distance 150 corresponds to the actual distance between the clamping tool and the workpiece with an error of less than or equal to 10 millimeters throughout the entire clamping operation. This level of precision makes it possible to differentiate the workpiece being clamped from other nearby workpieces, or the next workpiece the operator will clamp from other nearby workpieces, on mechanical equipment of varying sizes, for example, ranging from a few centimeters to several meters in length. For instance, the size of a clamping pitch cannot be less than 30 millimeters.
[0042] Step 160 of process 100 involves assisting the operator during the clamping operation. The assistance provided is dependent on the distance, calculated in step 150, between the clamping tool and the workpiece. Therefore, the type of assistance and / or its presentation varies depending on this distance. This distance allows the operator to determine the workpiece closest to the clamping tool and thus provide appropriate assistance at any time during the clamping operation, depending on the task being performed. Furthermore, the assistance can be provided in different ways. For example, it can be delivered via an augmented reality display on the screen visible to the operator and / or through the operator's control of the clamping tool.The term "control" in this application should be understood broadly as assistance in the use and / or automatic configuration of certain parameters of the clamping tool. Several examples of such control are provided later in the application.
[0043] In an example consistent with the previous examples, the display in the augmented reality environment includes information about the location of the component to be clamped on the mechanical equipment. For example, additional information and / or virtual objects related to the location of the component to be clamped are displayed in the augmented reality environment.
[0044] In an example consistent with the previous examples, when there are at least two elements to clamp in the clamping operation, the display, in the augmented reality environment, assists the operator in following a sequence The clamping sequence is at least partially predetermined. For example, the sequence can be a strictly ordered sequence or a sequence ordered according to a business rule. Thus, the display in the augmented reality environment includes information indicating the next element(s) to be clamped according to the clamping sequence. For example, a number is displayed for each of the next elements to be clamped on the mechanical equipment. The number corresponds, for example, to the position of the element to be clamped in the clamping sequence. The number 1 thus corresponds to the next element to be clamped according to the clamping sequence. Furthermore, it is worth noting that the number can be displayed even if the clamping element is not visible on the screen, for example, if the element is hidden by the operator or the mechanical equipment from the perspective of the 2D camera.
[0045] In an example consistent with the previous examples, when the target tightening torque setting applicable by the clamping tool is computer-configurable, the operator's assistance 160 can understand the target tightening torque setting of the clamping tool. This setting is determined based on the calculated distance 150 between the clamping tool and the workpiece. For example, when the clamping tool is more than 10 or 20 millimeters from a workpiece, the maximum tightening torque can be zero. In other words, when the distance between the clamping tool and a workpiece exceeds a predetermined distance, the clamping tool is deactivated, i.e., its tightening torque is zero.When the clamping tool is close to a component to be clamped, for example, at a distance of less than 10 or 20 millimeters, the clamping torque of the tool can be automatically set to the specific torque of the component closest to the tool. However, when this closest component is not the next component to be clamped according to the clamping sequence, or is not one of the components permitted by the clamping sequence, the clamping tool can be deactivated.
[0046] In an example consistent with the preceding examples, when the clamping tool is adapted to send the computer a clamping torque induced by the clamping tool during the clamping operation, the operator assistance 160 can include determining the end of a clamping operation for one element to be clamped from among all the elements to be clamped. Figure 4 is a block diagram illustrating the steps of an example of step 160 according to the invention. In order to determine that Once the clamping operation of a component is completed in substep 161, a target tightening torque value can be applied to said component. This target tightening torque value can be predetermined. By comparing this target tightening torque value—or more precisely, the minimum and maximum tightening torque values that define the tolerance range around the target torque—with the tightening torque induced by the clamping tool during the operation, it is possible to determine that the clamping operation of said component has been successfully completed and is therefore finished. When the completion of the clamping operation has been determined, this information can be displayed in the augmented reality environment in substep 162.
Claims
DEMANDS
1. A method (100) for clamping a component of mechanical equipment, the clamping operation being performed by an operator using a clamping tool in a work area, the method comprising the steps of: - Obtaining (110) a digital model of the mechanical equipment, - Obtaining (120) an augmented reality environment, displayed on a computer screen visible to the operator during the tightening operation, and a virtual reality environment, - Determination (130) of a position and orientation of the digital model in the augmented reality environment, the determination of the position and orientation of the digital model in the augmented reality environment being carried out: o by filming the mechanical equipment in the work area during the tightening operation using a 2D camera, and o by spatially superimposing the work area and the augmented reality environment, the spatial superposition of the work area and the augmented reality environment being carried out with a model-based augmented reality algorithm using the film of the mechanical equipment in the work area, - Determination (140) of a position and orientation of the clamping tool in the virtual reality environment, the determination of the position and orientation of the clamping tool in the virtual reality environment being carried out from a virtual reality tracker placed on the clamping tool in order to ensure continuity of visibility of the clamping tool in the virtual reality environment, and - Calculation (150) of a distance between the clamping tool and the element to be clamped, the calculation (150) of the distance comprising sub-steps of: o Calculation (151) of a position of an element to be clamped in the augmented reality environment from the determined position and orientation (130) of the digital model, o Obtaining (152) a common digital environment by spatially superimposing the augmented reality environment and the virtual reality environment, o Calculating (153) the position of the clamping tool and the element to be clamped in the common digital environment, and - Assistance (160) to the operator during the tightening operation, the assistance being: o carried out via the display of the augmented reality environment on the computer screen visible to the operator, o carried out via the control of the tightening tool, and o dependent on the calculated distance (150) between the tightening tool and the element to be tightened.
2. A method according to claim 1 wherein an error in the calculation (150) of the distance between the clamping tool and the element to be clamped is less than 10 or 20 millimeters during the entire clamping operation.
3. A method according to any one of the preceding claims wherein the operator assistance (160) comprises a display, in the augmented reality environment, of information on the location of the element to be clamped of the mechanical equipment.
4. A method according to any one of the preceding claims, wherein: - The clamping tool is computer-configurable; a clamping tool configuration includes setting a target tightening torque applicable by the clamping tool, and - the operator assistance (160) includes the configuration of the target tightening torque of the clamping tool as a function of the calculated distance (150) between the clamping tool and the element to be clamped.
5. A method according to any one of the preceding claims, wherein: - A predetermined tightening torque value is applied to the element to be tightened. - The clamping tool is adapted to send to the computer a clamping torque induced by the clamping tool during the clamping operation, and - The operator assistance (160) includes: o determining (161) the end of a tightening operation of the element to be tightened by determining that the tightening torque induced by the tightening tool during the tightening operation is within a predetermined tightening torque tolerance range, and o displaying (162) information, in the augmented reality environment, indicating that the tightening operation of the element to be tightened is complete and whether this tightening is compliant or not.
6. A method according to any one of the preceding claims, wherein: - The tightening operation consists of tightening an additional component of the mechanical equipment according to a tightening sequence that is at least partially predetermined, and - operator assistance (160) includes displaying, in the augmented reality environment, information indicating the next element to be tightened among the elements to be tightened, respecting the tightening sequence.
7. A method according to any one of the preceding claims, wherein the mechanical equipment is equipment of an aircraft, among: - an aircraft engine, - a landing gear, - an engine nacelle, a transmission box, and cabin equipment from an aircraft.
8. Clamping system comprising: - A computer configured to implement the process according to any one of the preceding claims, - A screen connected to the computer and adapted to display the augmented reality environment, - A clamping tool: o configurable by the computer, a clamping tool configuration including a setting of a target tightening torque applicable by the clamping tool, o adapted to send to the computer a tightening torque induced by the clamping tool during the clamping operation, - An augmented reality device including a 2D camera adapted to film a work area in which the mechanical equipment and the element to be clamped are included during the clamping operation, - A virtual reality device including at least one tracker adapted to locate the clamping tool and one tracker adapted to locate the 2D camera.
Citation Information
Patent Citations
Augmented reality spatial guidance and procedure control system
US20210158587A1
Mechanical fastening unit management method using augmented reality
US20210350588A1