Method and control device for displaying a model of an object to be processed by a manually operated tool

Abstract

A method for controlling an operator display (3) arranged to display a model of an object (6) to be processed by a manual operating tool (7) is provided. The displayed model is part of an operator instruction on how to process the object using the tool. The method includes determining a position of the tool relative to the object and automatically selecting a viewing direction of the model of the object displayed on the operator display based on the determined position of the tool relative to the object. The operator does not need to manually select the image / field of view angle of the model. Instead, the selection is made automatically depending on where the operator holding the tool is located relative to the object. Thus, the use of operator instructions is facilitated and time-consuming manual work can be reduced.

Description

Technical Field

[0001] The present invention generally relates to the field of controlling operator instructions for manual operating tools.

Background Art

[0002] In industrial factories, operator instructions are presented to instruct operators on how to process the objects to be manufactured. The operator instructions can include, for example, an instruction to enable the operator to understand how to process the object, and an image of the object to be processed. As an example, the operator instruction may depict a vehicle engine including markings / marking of a location where a joint needs to be tightened by a tool operated by the operator. The operator instructions often include a plurality of photos depicting the object from different viewpoints to visualize a plurality of processing steps, and it is cumbersome and time-consuming for the operator to select the appropriate photo.

Summary of the Invention

Means for Solving the Problems

[0003] It would be advantageous to implement a method and a control device that eliminate or at least mitigate the above-mentioned drawbacks. Specifically, it would be desirable to enable a method and a control device that facilitate the use of operator instructions for the operator.

[0004] To better address one or more of these problems, a method and a control device having the features defined in the independent claims are provided. Preferred embodiments are defined in the dependent claims.

[0005] Accordingly, according to the first aspect, a method for controlling an operator display is provided. The operator display is arranged to display a model of an object to be processed by a manually operated tool, and the displayed model is part of an operator instruction on how to process the object using the tool. The method includes determining the position of the tool relative to the object and automatically selecting the viewing direction of the model of the object displayed on the operator display based on the determined position of the tool relative to the object.

[0006] According to a second aspect, a control device for controlling an operator display is provided. The operator display is arranged to display a model of an object to be processed by a manually operated tool, and the displayed model is part of an operator instruction on how to process the object using the tool. The control device is configured to execute the method defined in the first aspect.

[0007] By using the position information of the tool relative to the object, automatic selection of the viewing angle of the model of the object can be performed. For example, when the operator holding the tool stands on a specific side of the object to be processed, the model displayed in the operator instruction can be displayed as seen from that specific side. In this case, the operator does not need to manually select the image / viewing angle of the model. Instead, the selection is automatically made according to where the operator holding the tool is located relative to the object. Accordingly, the use of operator instructions is made easier for the operator, time-consuming manual work is reduced, and ultimately it leads to a reduction in manufacturing time and cost. Furthermore, the positioning means is often already provided on the tool to determine the position of the processing point (such as a joint) to be processed by the tool, so that there is no need to add additional hardware to the tool.

[0008] The operator instruction may alternatively be referred to as a work flow instruction, an operation sequence, a work instruction, a work task instruction, etc.

[0009] As used herein, the term "manual operating tool" may include any tool that can be at least partially operated by a human. It can include any tool that can be moved, optionally using some type of machine, by a human. The tool can be, for example, fully handheld or manually operable using a torque reaction arm.

[0010] According to one embodiment, the viewing direction of the model of the object can be automatically selected, and the viewing point on the model of the object displayed on the operator display is adapted to reflect the determined position of the tool relative to the object.

[0011] As used herein, the term "reflect" may (at least roughly) include corresponding. Thus, the viewing point from which the model on the display is substantially viewed may (or be equal to) essentially correspond to the determined position of the tool.

[0012] The degree to which the selected viewing direction faithfully reflects the actual position determined with respect to the tool may vary depending on the embodiment.

[0013] According to one embodiment, the method can further include the step of storing at least two images depicting the model from at least two different viewpoints. Next, the step of automatically selecting the line-of-sight direction of the model of the object displayed on the operator display can include the step of selecting one of the at least two images displayed on the operator display based on the determined position of the tool relative to the object. When more images depicting the model from unique viewpoints are provided, the selected viewing direction of the model can more accurately reflect the actual detection position of the tool. This embodiment is advantageous in that it can reduce the storage space because the images of the model that need to be stored can simply be two-dimensional images.

[0014] According to another embodiment, the method can further include the step of storing a 3D (three-dimensional) model of the object, and the step of automatically selecting the viewing direction of the model of the object on the operator display can include the step of rotating the 3D model based on the determined position of the tool relative to the object, preferably rotating the 3D model so that the viewing point of the 3D model reflects the determined position of the tool. In this way, the model of the object can be a 3D model. By rotating the 3D model (preferably continuously), an (at least approximately) infinite number of viewpoints can be selected, so it is possible to select a view that more faithfully reflects the actual detection position of the tool. The 3D model can be, for example, a CAD (computer-aided design) model, or some other type of 3D visualization of the object.

[0015] According to one embodiment, the position of the tool relative to the object can be determined by the same means used to determine the position of the processing point (such as a joint) to be processed (such as tightened) by the tool. Therefore, there is no need to add hardware for tool positioning to the tool / system.

[0016] For example, tool positioning can be detected by camera-based techniques such as a camera attached to the roof above the processing area and / or a camera attached to a tool that recognizes an image of the object, optionally including the use of tags (such as having a QR code or other matrix code) on the tool and the object that can be detected by the camera(s). Other techniques that can be envisioned are wireless-based techniques such as UWB (ultra-wideband radio), where the time of flight (or the like) of a wireless signal between the tool and the object is determined to calculate the relative position of the tool relative to the object. If the tool is held by a torque reaction arm, the positioning can be done by detecting the angle of the joint of the arm. Other conventionally known positioning techniques can also be envisioned.

[0017] According to one embodiment, the method can be executed in an industrial assembly plant such as an automobile assembly plant.

[0018] According to one embodiment, the tool may be a tightening tool. The treatment of the object may include tightening joints (fastening means) such as screws / bolts / nuts in the object.

[0019] According to one embodiment, when a program is executed by a computer, there is provided a computer program including instructions for causing the computer to execute a method defined according to the first aspect or any one of the above embodiments.

[0020] According to one embodiment, there is provided a computer-readable storage medium including instructions for causing a computer to execute a method defined according to the first aspect or any one of the above embodiments when executed by the computer.

[0021] According to one embodiment, there is provided a system including a manually operable tool and an operator display arranged to display a model of an object to be processed by the manually operable tool, wherein the displayed model is part of an operator instruction on how to process the object using the tool, and a control device defined according to the second aspect.

[0022] Note that embodiments of the present invention relate to all possible combinations of the features described in the claims. Further, it should be understood that all of the various embodiments described for the method can be combined with the control device defined according to the second aspect of the present invention.

[0023] These aspects and other aspects will be described in more detail by the following detailed description of exemplary and non-limiting embodiments with reference to the accompanying drawings.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2A

Figure 2B

Figure 3

Mode for Carrying Out the Invention

[0025] All the drawings are schematic views and are not necessarily to scale. Generally, only the parts necessary for explaining the embodiments are shown, and other parts may be omitted. Throughout the description, like reference numerals refer to like elements.

[0026] Referring to FIG. 1, a system 1 according to an embodiment will be described. The system 1 includes a tool 7 that can be operated by an operator 5. The tool 5 is, for example, a power tool such as a tightening tool, a drill, or a grinder. The system 1 may also be called an industrial production system 1. The operator 5 uses the tool 7 to process an object 6. For the purpose of explanation and simplification, in this example, the object 6 is depicted simply as a solid cylinder. However, it should be understood that the object 6 processed by the operator 5 can be substantially any product manufactured in an industrial factory, such as vehicle parts, consumer products, etc., to name a few examples.

[0027] The system 1 further includes an operator display 3 for displaying operator instructions to the operator 5. The operator instructions displayed on the display 3 include a model of the object 6 and, preferably, marking / marking / text information useful for the operator 5 to understand how to process the object 6.

[0028] System 1 further comprises a control device 2 configured to control other parts of System 1, such as a display 3 and optionally a tool 7. The control device 2 may also be referred to as a computer, a control unit, a control system, etc. The control device 2 can comprise a memory for storing executable computer program code and a processor. The control device 2 can be connected, either wired or wirelessly, to the device to be controlled and can send (optionally receive) data thereto (or therefrom).

[0029] System 1 can further comprise means 4 for determining the position of the tool 7 relative to the object 6. The means 4 for determining the position of the tool 7 sends information to the control device 2 that enables the control device 2 to calculate / determine the position of the tool 7 relative to the object 6. For example, the position of the tool 7 can be determined in one step, and the position of the object 6 can be determined in another step or pre-determined (such as a pre-set fixed position). Thereafter, the interrelationship between the tool 7 and the object 6 can be determined.

[0030] The means 4 for determining the position of the tool 7 can be the same means used to determine the position of the tool 7 for the purpose of determining the position of the processing point on the object 6. This is often done in industrial production to provide traceability for each processing point. For example, the control device 2 can be configured to store, together with the positioning information of each processed joint, the tightening torque and angle used and information on whether the tightening is OK / NOK.

[0031] For example, System 1 can comprise one or more cameras 4 arranged on the roof above the processing area. The camera 4 can send image data to the control device 2. From this image data, the control device 2 can identify the tool 7 and the object 6, for example, by identifying tags / markings arranged on the tool 7 and the object 6, and calculate the relative position of the tool 7 and the object 6. Other suitable means for conventional tool positioning are also envisaged.

[0032] Next, the operation of the system 1 will be described with reference to FIGS. 1, 2A, and 2B. When the operator 5 needs to start processing the object 6, the position of the tool 7 with respect to the object is determined by the control device 2, for example, by analyzing the image data from the camera 4. According to the first embodiment, the operator is located at position A (the left side in FIG. 1), that is, on the short end of the columnar object 6. Next, as shown in FIG. 2A, the control device 2 automatically selects the viewing direction of the model 10 of the object 6 and instructs the display 3 to display the model 10 from that viewing direction. Therefore, the operator 5 will see the short side of the modeled object 10 on the display 3. For example, the display 3 can also display the marking 11 of the joint that needs to be tightened on the object 6 so that the operator 5 can know how to process the object 6.

[0033] For example, based on the determined position of the tool 7, the control device 2 can select one of at least two photos of the model 10 that depicts the object 6 from a specific angle, and the selected photo depicts the model 10 from the side that coincides with the viewpoint of the operator 5 holding the tool 7.

[0034] Alternatively, the control device 2 can store the 3D model 10 of the object 6, and the 3D model is rotated to a position based on the determined position of the tool 7 so that the viewpoint on the model 10 on the display reflects the determined position of the tool 7.

[0035] According to the second embodiment, the operator 5 moves to position B (the right side in FIG. 1), that is, the long side of the object 6 formed in a columnar shape. Next, as shown in FIG. 2B, the control device 2 automatically selects the viewing direction of the model 10 of the object 6 and commands the display 3 to display the model 10 from that viewing direction. Accordingly, the operator 5 will see the long side of the modeled object 10 on the display 3. For example, the display 3 can also display the marking 11 of the joint that needs to be tightened on the object 6 so that the operator 5 can know how to process the object 6.

[0036] The determination of the position of the tool 7 and the automatic selection of the viewing direction of the displayed model 10 can be repeatedly executed at a predetermined time interval or the like, or can be executed in response to some trigger signal, such as in response to the operator 5 finishing a specific processing step.

[0037] Referring to FIG. 3, a method 100 for controlling an operator display (such as the display 3 of the system 1 described with reference to FIG. 1) according to an embodiment will be described. The method 100 can be executed, for example, by a control device (such as the control device 2 of the system 1 described with reference to FIG. 1). The method can include storing a model of an object to be processed, such as step 101a of storing at least two images depicting the model from at least two different viewpoints, and / or step 101b of storing a 3D model of the object. The method 100 further includes step 102 of determining the position of a tool relative to the object, and step 103 of automatically selecting the viewing direction of the model of the object displayed on the operator display based on the determined position of the tool relative to the object. Step 103 of automatically selecting the viewing direction of the model of the object displayed on the operator display can include step 103a of selecting one of the at least two images (stored in step 101a) displayed on the operator display based on the determined position of the tool relative to the object. Alternatively (or complementarily), step 103 of automatically selecting the viewing direction of the model of the object on the operator display can include step 103b of rotating the 3D model based on the determined position of the tool relative to the object.

[0038] Those skilled in the art will understand that the present invention is not limited to the above-described embodiments. On the contrary, many modifications and variations are possible within the scope of the claims.

[0039] In addition, by considering the drawings, the disclosure, and the claims, those skilled in the art can understand and implement variations of the disclosed embodiments when implementing the invention described in the claims. In the claims, the term "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that certain means are described in mutually different dependent claims does not indicate that a combination of these means cannot be used advantageously.

Claims

1. A method (100) for controlling an operator display arranged to display a model of an object to be processed by a manual operation tool, wherein the displayed model is part of an operator instruction on how to process the object using the tool, a step (102) of determining the position of the tool relative to the object, a step (103) of automatically selecting a viewing direction of the model of the object displayed on the operator display based on the determined position of the tool relative to the object, The method comprising.

2. The viewing direction of the model of the object is automatically selected such that a viewing point on the model of the object displayed on the operator display reflects the determined position of the tool relative to the object, according to the method of claim 1.

3. The method further comprises a step (101a) of storing at least two images depicting the model from at least two different viewpoints, The step of automatically selecting a viewing direction of the model of the object displayed on the operator display, comprises a step (103a) of selecting one of the at least two images to be displayed on the operator display based on the determined position of the tool relative to the object, according to the method of claim 1 or 2.

4. The method further comprises a step (101b) of storing a 3D model of the object, The step of automatically selecting a viewing direction of the model of the object on the operator display, comprises a step (103b) of rotating the 3D model based on the determined position of the tool relative to the object, The method according to claim 1 or 2, comprising.

5. The position of the tool relative to the object is determined by the same means as those used to determine the position of a processing point, such as a joint, where processing such as tightening is performed by the tool, according to any one of claims 1 to 4.

6. The method is executed in an industrial assembly plant, according to any one of claims 1 to 5.

7. The tool is a tightening tool, according to any one of claims 1 to 6.

8. A control device (2) for controlling an operator display (3) arranged to display a model (10) of an object (6) to be processed by a manually operable tool (7), wherein the displayed model is part of an operator instruction for a method of processing the object using the tool, and the control device is configured to execute the method according to any one of claims 1 to 7. **Claim 9** A computer program comprising instructions which, when the program is executed by a computer, cause the computer to execute the method according to any one of claims 1 to 8. **Claim 10** A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to execute the method according to any one of claims 1 to 8. **Claim 11** A manually operable tool (7), an operator display (3) arranged to display a model (10) of an object to be processed by the manually operable tool, wherein the displayed model is part of an operator instruction for a method of processing the object using the tool, the control device (2) according to claim 8, and a system (1) comprising the same.

Images