Method and control device for displaying model of object machined by manually operated tool

Abstract

A method of controlling an operator display (3) arranged to display a model of an object (6) machined by a manually operated tool (7) is provided. The displayed model is part of operator instructions for how the object is machined using the tool. The method includes determining a position of a tool relative to an object; and automatically selecting a viewing direction of the model of the object to be displayed on the operator display based on the determined position of the tool relative to the object. The operator may then not have to manually select the picture / view angle of the model. Instead, the selection takes place automatically as a function of where the operator holding the tool is relative to the object. Therefore, the operator can use the operator instruction conveniently, and time-consuming manual processing can be reduced.

Description

Technical Field

[0001] The present invention relates generally to the field of controlling the display of operator instructions for manually operated tools. Background Art

[0002] In an industrial plant, operator instructions are provided to instruct an operator how to process an object being manufactured. The operator instructions may, for example, include a picture of the object to be processed and instructions that enable the operator to understand how he / she should process the object. As an example, the operator instructions may depict a vehicle engine with symbols / markings, where joints should be tightened with the aid of a tool operated by the operator. The operator instructions may typically include several pictures depicting the object from different perspectives to visualize several processing steps, making it cumbersome and time consuming for the operator to select the correct picture. Summary of the invention

[0003] It would be beneficial to achieve a method and control device that overcomes or at least mitigates the above-mentioned disadvantages. In particular, it would be desirable to achieve a method and control device that facilitates the use of operator commands by an operator.

[0004] In order to better solve one or more of these problems, a method and a control device are provided having the features defined in the independent claims. Preferred embodiments are defined in the dependent claims.

[0005] Thus, according to a first aspect, there is provided a method of controlling an operator display. The operator display is arranged to display a model of an object to be machined by a manually operated tool, the displayed model being part of operator instructions on how to use the tool to machine the object. The method comprises: determining a position of the tool relative to the object; and automatically selecting a viewing direction of the model of the object to be displayed on the operator display based on the determined position of the tool relative to the object.

[0006] According to a second aspect, there is provided a control device for controlling an operator display. The operator display is arranged to display a model of an object to be machined by a manually operated tool, the displayed model being part of operator instructions on how to use the tool to machine the object. The control device is configured to perform the method as defined in the first aspect.

[0007] By using information about the position of the tool relative to the object, the viewing angle of the model of the object can be automatically selected. For example, if the operator holding the tool is standing on a specific side of the object to be processed, the model displayed in the operator instructions can be displayed as viewed from this specific side. The operator then does not have to manually select the picture / viewing angle of the model. Instead, the selection is made automatically based on where the operator holding the tool is located relative to the object. Therefore, the operator is facilitated to use the operator instructions, and time-consuming manual processing can be reduced, which ultimately leads to a reduction in manufacturing time and costs. Further, positioning devices are usually already provided in the tool to determine the position of the processing point (e.g. joint) to be processed by the tool, thereby eliminating the need to add further hardware to the tool.

[0008] Operator instructions may alternatively be referred to as workflow instructions, operation sequences, work orders, work task orders, and the like.

[0009] In this specification, the term "manually operated tool" may include any tool that can be at least partially operated by a person. It may include any tool that can be moved by a person, optionally with the aid of some type of machinery. The tool may, for example, be completely hand-held, or may be manually operated with the aid of a torque reaction arm.

[0010] According to an embodiment, a viewing direction of the model of the object may be automatically selected so that a viewing angle on the model of the object to be displayed on the operator display reflects the determined position of the tool relative to the object.

[0011] In this context, the term "reflects" may (at least substantially) include corresponding to. Thus, the perspective from which the model on the display is actually viewed may substantially correspond to (or even be equal to) the determined position of the tool.

[0012] In different embodiments, the extent to which the selected viewing direction reflects the actual position determined for the tool may vary.

[0013] According to an embodiment, the method may further include storing at least two pictures depicting the model from at least two different viewing angles. Automatically selecting a viewing direction of the model of the object to be displayed on the operator display may then include selecting one of the at least two pictures to be displayed on the operator display based on the determined position of the tool relative to the object. The more pictures depicting the model from unique viewing angles are provided, the more accurately the selected viewing direction of the model reflects the actual detection position of the tool. An advantage of this embodiment is that it can reduce storage space because the pictures of the model that need to be stored can simply be two-dimensional images.

[0014] According to another embodiment, the method may further comprise storing a 3D (three-dimensional) model of the object, wherein automatically selecting the viewing direction of the model of the object on the operator display may comprise rotating the 3D model based on the determined position of the tool relative to the object, preferably such that the viewing angle of the 3D model reflects the determined position of the tool. Thus, the model of the object may be a 3D model. A view may be selected that more closely reflects the actual detected position of the tool, since the (preferably stepless) rotation of the 3D model allows the selection of a (at least almost) unlimited number of viewing angles. The 3D model may be, for example, a CAD (Computer Aided Design) model or any other type of 3D visualization of the object.

[0015] According to an embodiment, the position of the tool relative to the object can be determined by the same means as the means for determining the position of the processing point (e.g., joint) to be processed (e.g., tightened) by the tool. Therefore, there is no need to add additional hardware for positioning the tool to the tool / system.

[0016] For example, the positioning of the tool can be detected using camera-based techniques, such as a camera mounted on top above the processing area and / or a camera mounted on the tool that recognizes the image of the object, optionally including the use of tags on the tool and on the object detectable by at least one camera (e.g., with a QR code or other matrix code). Other technologies that can be envisioned are radio-based technologies, such as UWB (ultra-wideband radio), in which the flight time (or similar time) of the radio signal between the tool and the object is determined in order to calculate the position of the tool relative to the object. In the case where the tool is held by a torque reaction arm, positioning can be performed by detecting the angle of the joint in the arm. Other positioning techniques known in the art can also be envisioned.

[0017] According to an embodiment, the method may be performed in an industrial assembly plant, such as an automobile assembly plant.

[0018] According to an embodiment, the tool may be a fastening tool.The machining of the object may include tightening a joint (fastening device) such as a screw / bolt / nut at the object.

[0019] According to an embodiment, there is provided a computer program comprising instructions which, when executed by a computer, cause the computer to perform a method as defined in the first aspect or any one of the preceding embodiments.

[0020] According to an embodiment, a computer-readable storage medium is provided, the computer-readable storage medium comprising instructions, which when executed by a computer causes the computer to perform a method as defined in the first aspect or any one of the preceding embodiments.

[0021] According to an embodiment, a system is provided, comprising: a manually operable tool; an operator display arranged to display a model of an object to be processed by the manually operable tool, the displayed model being part of the operator's instructions on how to use the tool to process the object, and a control device defined according to the second aspect.

[0022] It should be noted that the embodiments of the present invention relate to all possible combinations of features recited in the claims. In addition, it should be understood that the various embodiments described for the method can be combined with the control device defined according to the second aspect of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] These and other aspects will now be described in more detail in the following illustrative and non-limiting detailed description of embodiments, with reference to the accompanying drawings.

[0024] Figure 1 A system according to an embodiment is shown.

[0025] Figure 2A Shows the operator is located Figure 1 At A in the figure, the display is included in the system.

[0026] Figure 2B Shows the operator is located Figure 1 At B in the figure, the display is included in the system.

[0027] Figure 3 A method according to an embodiment is shown.

[0028] All the drawings are schematic, not necessarily drawn to scale, and generally only show parts which are necessary in order to elucidate the embodiments, wherein other parts may be omitted. Throughout the specification, the same reference numerals refer to the same elements. DETAILED DESCRIPTION

[0029] Will refer to Figure 1 A system 1 according to an embodiment is described. The system 1 includes a tool 7 that can be operated by an operator 5. The tool 5 can be, for example, a power tool, such as a fastening tool, a drill, a grinder, etc. The system 1 can refer to an industrial manufacturing system 1. The operator 5 uses the tool 7 to process an object 6. For illustrative purposes and for simplicity, the object 6 in this example is simply depicted as a solid cylinder. However, it should be understood that the object 6 to be processed by the operator 5 can be virtually any product manufactured in an industrial plant, such as vehicle parts, consumer goods, to name just a few examples.

[0030] The system 1 further comprises an operator display 3 for displaying operator instructions for the operator 5. The operator instructions displayed on the display 3 comprise a model of the object 6 and preferably also symbols / marks / text information that help the operator 5 understand how to process the object 6.

[0031] The system 1 further comprises a control device 2, which is configured to control the display 3 and optionally also other components of the system 1, such as the tool 7. The control device 2 may alternatively refer to a computer, a control unit, a control system, etc. The control device 2 may include a memory and a processor for storing and executing computer program code. The control device 2 may be connected to the device it controls by wire or wirelessly for sending (and optionally also receiving) data to (or from) it.

[0032] The system 1 may further comprise means 4 for determining the position of the tool 7 relative to the object 6. The means 4 for positioning the tool 7 sends information to the control device 2 so that the control device 2 can calculate / determine the position of the tool 7 relative to the object 6. For example, the position of the tool 7 may be determined in one step, and the position of the object 6 may be determined in another step or may be predetermined (e.g. a preset fixed position). The mutual relationship between the tool 7 and the object 6 may be determined subsequently.

[0033] The means 4 for determining the position of the tool 7 may be the same as the means for determining the position of the tool 7 for determining the position of the machining point on the object 6. This is often done in industrial manufacturing to provide traceability at the various machining points. For example, the control device 2 may be configured to save information such as the tightening torque and angle used and whether the tightening was possible / not possible together with the positioning information of the various machining joints.

[0034] For example, the system 1 may include one or more camera devices 4 arranged on the top above the machining area. The camera device 4 may send image data to the control device 2. Based on the image data, the control device 2 is able to identify the tool 7 and the object 6, for example by identifying tags / markings located on the tool 7 and the object 6, and is able to calculate the relative position of the tool 7 and the object 6. Any other suitable device known in the art for tool positioning is also conceivable.

[0035] Now refer to Figure 1 , Figure 2A and Figure 2B The operation of the system 1 is described. When the operator 5 is to start machining the object 6, the position of the tool 7 relative to the object is determined by the control device 2, for example by analyzing image data from the camera device 4. According to a first example, the operator is located at position A (at Figure 1The control device 2 then automatically selects a viewing direction for the model 10 of the object 6 and instructs the display 3 to display the model 10 from this viewing direction, such as Figure 2A As shown. Therefore, the operator 5 will see the short end of the modeled object 10 on the display 3. For example, the display 3 can also show a mark 11 where the joint should be tightened at the object 6, so that the operator 5 knows how to process the object 6.

[0036] For example, the control device 2 can select one of at least two pictures of the model 10 that depict the object 6 from a unique angle based on the determined position of the tool 7, so that the selected picture depicts the model 10 from a side that matches the perspective of the operator 5 holding the tool 7.

[0037] Alternatively, the control device 2 may store a 3D model 10 of the object 6 , wherein the 3D model is rotated to a position based on the determined position of the tool 7 , so that the viewing angle on the model 10 on the display reflects the determined position of the tool 7 .

[0038] According to the second example, the operator 5 has moved to position B ( Figure 1 The control device 2 then automatically selects a viewing direction for the model 10 of the object 6 and instructs the display 3 to display the model 10 from this viewing direction, such as Figure 2B As shown. Therefore, the operator 5 will see the long side of the modeled object 10 on the display 3. For example, the display 3 can also show the mark 11 where the joint should be tightened at the object 6, so that the operator 5 knows how to process the object 6.

[0039] Determining the position of the tool 7 and automatically selecting the viewing direction of the model 10 to be displayed may be performed repeatedly, for example at predetermined time intervals, or in response to some type of trigger signal (for example in response to the operator 5 having completed a particular processing step).

[0040] Will refer to Figure 3 Describe the control operator display according to the embodiment (for example, refer to Figure 1 The method 100 of the display 3 of the system 1 described above. The method 100 can be performed, for example, by a control device (for example, referring to Figure 1The method may include storing a model of an object to be processed, for example storing at least two pictures depicting the model from at least two different perspectives (step 101a), and / or storing a 3D model of the object (step 101b). The method 100 further includes determining the position of the tool relative to the object (step 102), and automatically selecting a viewing direction of the model of the object to be displayed on the operator display based on the determined position of the tool relative to the object (step 103). Automatically selecting a viewing direction of the model of the object to be displayed on the operator display (step 103) may include selecting one of the at least two pictures (stored in step 101a) to be displayed on the operator display based on the determined position of the tool relative to the object (step 103a). Alternatively (or in addition), automatically selecting a viewing direction of the model of the object on the operator display (step 103) may include rotating the 3D model based on the determined position of the tool relative to the object (step 103b).

[0041] Those skilled in the art will appreciate that the present invention is by no means limited to the embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims.

[0042] Furthermore, variations to the disclosed embodiments may be understood and effected by those skilled in the art in practicing the claimed invention by studying the drawings, the disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. A method (100) of controlling an operator display arranged to display a model of an object to be machined by a manually operated tool, the displayed model being part of operator instructions for how to use the tool to machine the object, the method include: - determining the position of the tool relative to the object (102); as well as - Based on the determined position of the tool relative to the object, automatically selecting a viewing direction (103) of a model of the object to be displayed on an operator display.

2. The method according to claim 1, in, A viewing direction of the model of the object is automatically selected so that a viewing angle on the model of the object to be displayed on the operator display reflects the determined position of the tool relative to the object.

3. The method according to claim 1 or 2, further comprising: include: - storing at least two pictures (101a) depicting the model from at least two different viewing angles; Wherein, automatically selecting a viewing direction of a model of an object to be displayed on an operator display includes: - Based on the determined position of the tool relative to the object, selecting one of the at least two pictures to be displayed on the operator display (103a).

4. The method according to claim 1 or 2, further comprising: include: -Storing a 3D model of an object (101b); Wherein, automatically selecting a viewing direction of a model of an object on an operator display includes: - Rotating the 3D model based on the determined position of the tool relative to the object (103b).

5. The method according to any one of the preceding claims, in, The position of the tool relative to the object is determined by the same means as are used to determine the position of a processing point to be processed by the tool, such as a joint, such as tightening.

6. The method according to any one of the preceding claims, in, The method is performed in an industrial assembly plant.

7. The method according to any one of the preceding claims, in, The tool is a fastening tool.

8. A control device (2) for controlling an operator display (3), the operator display being arranged to display a model (10) of an object (6) to be machined by a manually operated tool (7), the displayed model being part of the operator's instructions on how to use the tool to machine the object, the control device being configured to carry out a method according to any one of the preceding claims.

9. A computer program comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 8.

10. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1-8.

11. A system (1), include: - a tool (7) which can be operated manually; - an operator display (3) arranged to display a model (10) of an object to be processed by the manually operable tool, the displayed model being part of the operator's instructions on how to use the tool to process the object; and - A control device (2) according to claim 8.

Images