DEVICE FOR MONITORING THE POSITION AND / OR LOCATION AND / OR MOVEMENT OF A TOOL

DE502022002035D1Active Publication Date: 2024-11-21SARISSA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
DE502022002035
Authority / Receiving Office
DE · DE
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-12-06
Filing Date
2022-09-05
Publication Date
2024-11-21
Estimated Expiration
2042-09-05

AI Technical Summary

Technical Problem

Inertial sensors used in positioning and movement detection of objects accumulate errors over time due to reliance on internal data, requiring frequent recalibration, which disrupts assembly processes and is time-consuming, especially in large work areas.

Method used

A device equipped with a gyroscope or inertial sensor, an image capture unit, and an ultrasonic locating device that allows for continuous calibration by comparing inertial sensor data with image processing data, enabling accurate position and movement tracking beyond the image capture range using transit time measurements.

Benefits of technology

This solution ensures continuous calibration of inertial sensors without interrupting assembly processes, allowing for precise position and movement tracking in larger work areas and improving the accuracy of location determination, reducing the need for frequent recalibration and enhancing workflow efficiency.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader
Need to check novelty before this filing date? Find Prior Art

Description

[0001] WO 2011 / 128766 A2 describes an image acquisition unit as a positioning device that can also be used to calibrate a gyroscope. Other types of positioning devices, e.g., for ultrasonic positioning, are mentioned as alternatives to image acquisition. DE 10 2013 018 703 A1 discloses a device for improving manufacturing processes. US 9,842,254 B1 describes a system or process by which an inertial measuring unit can be calibrated. US 2016 / 0214198 A1 shows a welding system that has a measuring system with an inertial measuring unit.

[0002] Inertial sensors such as a gyroscope sensor and / or an inertial sensor detect dynamic parameters of a moving object, such as a change in momentum, i.e., acceleration, change in angular momentum, or the like, and thereby allow the determination of the changed position and / or attitude and / or movement of the object. Since the position and / or attitude and / or movement is not determined via absolute detection by an external sensor, but rather based on the most recently determined data, errors in the determination of the position and / or attitude and / or movement accumulate over time. To maintain the desired accuracy, such a device must therefore be recalibrated regularly. This is usually done by bringing the object in question into a predetermined, standardized state with a known position and / or attitude and / or movement of the object.When using tools during the assembly of a workpiece, calibration interrupts the assembly process and causes a loss of time and disrupts the concentration of the worker.

[0003] The object of the invention is therefore to propose such a device according to the preamble of claim 1 with improved calibration.

[0004] This problem is solved by the characterizing features of claim 1.

[0005] Accordingly, a device with a tool for monitoring the position and / or attitude and / or movement of the tool is proposed, wherein an inertial sensor such as a gyroscope sensor and / or an inertial sensor is attached to the tool, wherein a control unit, at least one image acquisition unit and an image evaluation unit are provided on the device, wherein the control unit is designed to determine a position and / or attitude and / or movement of the tool starting from a predetermined initial situation using the acquired sensor data of the inertial sensor, wherein the image acquisition unit is provided for calibrating the inertial sensor, wherein the image acquisition unit is designed to acquire one or more images of the tool at least at specific times or time intervals,wherein the image evaluation unit is provided for determining the position and / or location and / or movement of the tool, and wherein the control unit for calibrating the sensor data of the inertial sensor is designed to process the data of the image acquisition unit, characterized in that a locating device is provided on the device, wherein the locating device is designed with an ultrasonic transmitter and an ultrasonic receiver, wherein the locating device is designed to locate the tool by determining the distance between the ultrasonic transmitter and the ultrasonic receiver via transit time measurements of ultrasonic signals, wherein the distance between corresponding transmitters and receivers can be determined by means of the ultrasonic signals,The working range of the locating device, which operates by means of time-of-flight measurement, extends beyond an image area of ​​the image acquisition unit, and the presence of the tool in the image area of ​​the image acquisition unit is determined by the locating device for locating the tool via time-of-flight measurements of the ultrasonic signals. This allows for regular comparison of the inertial sensor data with that of the image processing system, and thus also calibration of the inertial sensor, when the tool is in the image area of ​​the image acquisition unit. Thus, the inertial sensor is always sufficiently calibrated to provide the desired data on the position and / or orientation and / or movement of the tool, even outside the image acquisition area.

[0006] The image acquisition unit is preferably positioned in such a way that the tool is regularly moved into the image area during the intended workflow, thus enabling calibration without interruption during work.

[0007] However, the invention already offers advantages if the insertion of the tool into the image area is provided as a separate step in the workflow, since even then the calibration process is significantly shortened compared to positioning in a defined calibration position and / or location, e.g. in a template or the like.

[0008] The independent tracking device allows tool location in a workspace significantly larger than the field of view of the image acquisition unit. This tracking data can then be used in combination with data from the inertial sensor to precisely determine the position, orientation, and / or movement of the tool.

[0009] At the same time, the positioning via this independent positioning device can also be used to improve or facilitate image analysis by the image acquisition unit. For example, if the position of a tool or a sensor attached to the tool, such as an ultrasonic transmitter, is known, the orientation of the tool in space can be more easily determined from the image perspective.

[0010] Ultrasonic transmitters and receivers can be used to accurately locate objects using time-of-flight measurements. Electromagnetic signals from corresponding transmitters and receivers can also be used to locate objects, particularly using time-of-flight measurements.

[0011] A device according to the invention can be used, for example, in assistance or monitoring devices for quality assurance during the assembly of workpieces using one or more tools, in particular a hand tool such as a screwdriver, a drill, or the like. It can also be combined with visual or audible instructions for the various assembly steps and / or with automatic tool settings, e.g., a torque setting for a screwdriver or the like.

[0012] The invention is particularly advantageous when the workpiece to be machined is very large, as is the case, for example, in the assembly of aircraft, ships, trains, or the like. In such work, there are often areas that are not, or not completely, covered by locating devices such as ultrasonic devices. Therefore, the use of one or more inertial sensors significantly expands the working range of the tool monitoring and, when combined with another locating device, supplements it.

[0013] However, especially with large tool working areas, calibration by placing the tool in a predefined calibration position is very time-consuming due to the associated travel. The complete or extensive automation of calibration, as enabled by the invention, offers a major advantage.

[0014] It is advantageous if at least one data transmission unit for wirelessly or wired transmission of data / information of the inertial sensor and / or at least one locating transmitter of the independent locating unit is included.

[0015] With the aid of these measures according to the invention, various disadvantages of the prior art can be advantageously eliminated or reduced.

[0016] For example, a single positioning transmitter, particularly an ultrasonic transmitter or an LED / light source, can be provided, which is advantageously combined with the inertial sensor for positioning. The data transmission unit is advantageously used to transmit data from the inertial sensor and / or other data / information / parameters.

[0017] It has been shown that the combination of a single tracking transmitter, for example in the form of an ultrasonic transmitter, and an inertial sensor, preferably an inertial sensor unit, together with wireless data transmission in a very compact and space-saving design, offers a significant advantage. This allows the sensor unit on / in the tool to be smaller than previously possible. Accordingly, the tracking components or the sensor unit can be integrated into / in the tool and also protected against damage, etc.

[0018] In addition, the design with a single ultrasonic source enables greater accuracy in positioning, especially in determining coordinates or so-called "geodata", than if three ultrasonic transmitters were used or "calculated" as is currently commercially available.

[0019] The advantageous data transmission unit is furthermore advantageously designed or suitable for sending data with which, for example, preventive maintenance, etc., can be supported.

[0020] The data transmission unit for wireless transmission can be adapted to the environment and, for example, operate using electromagnetic waves / signals in the radio and / or infrared and / or visible frequency range.

[0021] However, the data transmission unit can also be designed for data transmission by means of ultrasound, in particular by modulating a location signal, and thus use an already existing transmitter.

[0022] If the data transmission unit is designed for electromagnetic transmission, it can also be used to transmit a trigger signal to start the time-of-flight measurement. Otherwise, a separate trigger unit for electromagnetic transmission, particularly in the infrared frequency range, of a trigger signal to start the time-of-flight measurement must be provided. Due to the significantly shorter propagation time of a trigger signal traveling at the speed of light, the start of the significantly slower ultrasonic signal for the time-of-flight measurement can be initiated without significant errors in the time-of-flight measurement.

[0023] By wirelessly transmitting the trigger signal, any spatially limited sources of interference can be avoided, even if the transmitters are located at a distance.

[0024] In a particular development of the invention, the inertial sensor is designed as an inertial sensor unit with at least one acceleration sensor for detecting a linear and / or rotational acceleration and / or with at least one yaw rate sensor for detecting a rotational speed or rotation. The inertial sensor unit or an inertial measuring unit (so-called IMU) advantageously comprises a spatial combination of several inertial sensors such as acceleration sensors and / or yaw rate sensors. To detect the six possible kinematic degrees of freedom, the inertial sensor unit or IMU advantageously comprises three acceleration sensors (translation sensors) that are arranged orthogonally to one another for detecting the translational movement in the x-, y- or z-axis and / or advantageously three yaw rate sensors or so-called"Gyroscopic sensors" for detecting rotating (circular) movements in the x-, y-, or z-axis.

[0025] The inertial sensor unit or IMU advantageously supplies / generates, for example, as signals or measured values / parameters, preferably three linear acceleration values ​​for the translational movement and / or three angular velocities for the rotation rates. For example, in a specific application of the sensor unit, a linear velocity can advantageously be determined from the measured values ​​of the IMU for the linear accelerations, if necessary after compensation for the acceleration due to gravity, by integration and, if necessary, further integration, the position in space relative to a reference point or coordinate system point / origin or the like. The advantageous integration of the three angular velocities supplies, relative to a reference point, the orientation of the sensor unit and / or the tool in space, which can advantageously be used for location or the production assistance device according to the invention.

[0026] The inertial sensor is preferably designed as a micro-electromechanical inertial sensor unit with at least one micro-electromechanical acceleration sensor for detecting linear and / or rotational acceleration and / or with at least one micro-electromechanical yaw rate sensor for detecting rotational speed / rotation. This allows for a particularly space-saving design, which improves integration capabilities and resilience against damage, etc.

[0027] In order to advantageously determine the integration constants, to improve the accuracy and / or to advantageously correct or adapt and / or calibrate a zero point and long-term drift of the above-mentioned inertial sensors or the position measuring system, additional sensors and / or measures can be advantageously used, e.g. magnetometers or magnetic field sensors and so-called GNSS sensors integrated in the inertial sensor unit.

[0028] Fundamentally, according to the invention, it is advantageous that location is determined by measuring the propagation time between the location sensor and the receiving unit. In general, an exact distance between a transmitter and a receiver can be determined using the propagation time of the location signal. Using multiple paths, location can thus be made more precise. With a total of three linearly independent paths, a maximum of two points remain as positions to be located, of which one can generally be excluded through a plausibility check. For example, a second theoretically possible point can systematically always lie outside the effective range of the location device. These considerations are based on the fact that for three measured distances to a location sensor, the common intersection points of three spherical surfaces in space must be determined.

[0029] This location option, using the existing location sensor(s), also allows the correct presence of the tool within the field of view of the image capture unit for calibration and / or verification. The image capture can then be used to determine a calibration and / or verification position.

[0030] The sensor unit can be implemented in various designs. A preferred design consists of only an ultrasonic transmitter and a so-called IMU unit.

[0031] With a clever choice of the inertial sensor unit, the following features are advantageously covered: accelerometer, gravity, linear acceleration, gyroscope, uncalibrated gyroscope, game rotation vector, pedometer, step detector, significant movement, tilt detector, pick up gesture, wake up gesture, look gesture, activity detection, etc.

[0032] Additional functionalities can also be implemented with an integrated or additional magnetometer, e.g.: geomagnetic field, uncalibrated magnetic field, orientation, rotation vector, geomagnetic rotation vector, etc.

[0033] With an advantageous combination of the inertial sensor unit with the ultrasound, the ultrasonic transmitter provides the location information, for example, by receiving the emitted sound waves from multiple receiver units, which allows the location of the transmitter to be determined. This location information is only a point in space (x, y, z). The information regarding the position (pose) and orientation of the object in space on which the sensor unit is mounted is advantageously obtained from the inertial sensor.

[0034] The acquired data is transmitted to the receiving unit, preferably via a wireless link, e.g., using 5G, BLE, 4G, Wi-Fi, a proprietary protocol in approved frequency bands, or others, and combined with the location information from the ultrasound. The ultrasound transmitter can also be replaced with one or more light sources, LEDs, if necessary.

[0035] The calibration of the inertial sensor unit, which must be performed at specific intervals, is carried out, for example, via a software command from the receiver to the transmitter via a bidirectional radio link, provided the transmitter is located at a defined location. The presence of the inertial sensor at the calibration location can be verified by the image acquisition unit and / or the independent positioning device.

[0036] A major advantage of the combination of an ultrasonic transmitter and an inertial sensor unit, along with a radio link, is its very compact design. Furthermore, the design with a single ultrasonic source will enable greater positioning accuracy than using three ultrasonic transmitters.

[0037] The integrated electromagnetic transmission unit is also suitable for advantageously sending data that can be used to support preventive maintenance.

[0038] An embodiment of the invention is shown in the drawing and is explained in more detail below with reference to the figures.

[0039] Show in detail Figure 1 a schematic representation of an assembly line with a device according to the invention, Figure 2 a block diagram of a device according to the invention and Figure 3a schematic representation of a locating device with a locating object.

[0040] In Figure 1 An assembly line 1 is schematically shown, which has an upper run 3 rotating around a deflection roller 2. On the assembly line 1, semi-finished automobiles 4, 5 are shown as workpieces to be machined according to the invention.

[0041] Each of the vehicles 4, 5 or workpieces is assigned a locating device 6, 7, which is moved along the assembly line 1 in a stationary manner to the corresponding workpiece 4, 5. Alternatively, the workpieces can also be arranged on self-propelled assembly islands.

[0042] Each locating device 6, 7 comprises three locating sensors 8, 9, 10 and a communication unit 11 for wireless communication by means of electromagnetic waves 12, for example as a data transmission unit as mentioned above.

[0043] Three schematically illustrated different locating objects in the form of a screwdriver 13, a glove 14 and an assembly part 15 are each also provided with locating sensors 16, inertial sensors 17, 18 and a communication unit 19.

[0044] In Figure 2 a central control unit 20 is shown, which includes a data connection to locating devices and locating objects by means of a communication unit 21.

[0045] It is also connected to a control unit 22, a display 23 and a signal generator 24.

[0046] In Figure 3The locating device 7 and the screwdriver 13 are depicted as the object to be located. The screwdriver 13 includes the locating sensors 25, 26, 27, a communication unit 28, and two inertial sensors 29, 30. One of the inertial sensors 29 can, for example, be an inertial sensor for measuring momentum changes, and the other inertial sensor 30 can be a gyroscope.

[0047] For calibrating the inertial sensors 29, 30, an image acquisition unit is provided which is connected via a wireless data channel 32 to an image evaluation unit 33, which in turn communicates with the central control unit 20.

[0048] As an example of the location method, a travel distance l is shown, over which the travel time of a location signal can be recorded. For example, the location sensor 10 can be designed as an ultrasonic transmitter and the location sensor 25 as an ultrasonic receiver. One possibility for measuring the travel time is, for example, for the central control unit 20 to send a trigger signal to the location device 7 and to the object 13 to be located via the communication unit 21. As a result, the location sensor 10 emits an ultrasonic signal, which is detected by the location sensor 25 after traveling the travel distance l. In one embodiment of the invention, the reception of the ultrasonic signal is confirmed via the communication unit 28, so that the central control unit 20 determines the travel time.In another embodiment, a clock can be started with the trigger signal in the location sensor 25, so that upon receipt of the location signal the running time is immediately measured and this is transmitted as information via the communication unit 28.

[0049] In principle, a location or a transit time measurement can also be carried out in a different direction, i.e. the location sensor 25 can be designed as a transmitter and the location sensor 10 as a receiver.

[0050] In the illustrated embodiments, three location sensors are attached to each object to be located. This allows not only the position of the respective object 13, 14, 15 to be determined, but also its orientation in space, which may be advantageous for the intended process monitoring. Locating and determining the orientation of an object 13, 14, 15 would also be possible with just one location sensor in conjunction with an inertial sensor.

[0051] As shown by Figure 1As can be seen, a locating device 6, 7 is provided on each workpiece 4, 5, which is fixedly mounted relative to the workpiece 4, 5. By locating a locating object 13, 14, 15 and, if applicable, its orientation, the relative position of the locating object with respect to the workpiece 4, 5 and its orientation are thus immediately known. Thus, the movements of the locating objects 13, 14, 15 with respect to the workpieces 4, 5 can be monitored by the central control unit 20.

[0052] Depending on the work step to be monitored, the process control computer can receive information about the type or nature of the corresponding object being located. However, if necessary, individual identification can also be performed via the separate data connection. In addition, further information, such as speed, torque, or the like, can also be transmitted during an assembly process. The settings of a tool that is recorded as a object being located can also be performed by the central control unit 20 via this data connection.

[0053] The movements of assembly personnel can be tracked using tracking devices that can be worn by an assembly worker, such as glove 14. This makes it possible to monitor assembly movements or guide assembly personnel through the process steps to be performed.

[0054] Assembly parts, i.e. parts that are to be mounted on the workpieces 4, 5, can also be monitored accordingly, as is schematically shown using a vehicle door 15.

[0055] In a special embodiment, the location sensors 8, 9, 10 of the location devices 6, 7 are equipped with transmitters for the time-of-flight signal, for example, ultrasonic transmitters. Accordingly, only receivers for the time-of-flight signal are required for the mobile location objects 13, 14, 15. This significantly reduces the energy consumption of these mobile objects. Thanks to the separate data connection via the communication units 19, the location objects can also be used with passive receiver sensors 16 for time-of-flight measurement.

[0056] The locating devices 6, 7, which are fixed to the workpieces, can be easily connected to larger energy storage units or to a power grid. In principle, the separate data connection on the locating devices 6, 7 side can also be wired to the process control computer or a similar unit with comparable functionality.

[0057] For the flexibility of the process support device, it is sufficient if the locating objects 13, 14, 15 are freely movable relative to the processing location of the workpieces 4, 5.

[0058] Via the display 23 and the signal generator 24, information about completed process steps can be provided to persons involved in the process being monitored. A warning signal in the event of a faulty process step can also be generated via these two units, for example. Process control is also possible via these units, for example, by displaying or highlighting the location of the next process step. A person in the process can also be guided to the location of the next process step acoustically.

[0059] The image acquisition unit 31 is positioned so that each object 13, 14, 15 to be monitored is regularly moved into the image acquisition area. The image acquisition data is then used to calibrate the inertial sensors 29, 30.

[0060] For this purpose, the image evaluation unit 33 is connected to the central control unit 20, which in turn communicates with the inertial sensors 29, 30. Data processing of an inertial sensor 29, 30 can be performed in the control unit or in the sensor itself. Accordingly, calibration also takes place either in the central control unit 20 or, after appropriate data transmission, in the corresponding inertial sensor 29, 30. Data processing in the periphery of a network, i.e., in the sensor(s), is also referred to as "edge computing."

[0061] The inertial sensors allow the use of locating objects connected to the process to be monitored, which do not necessarily have to be permanently connected to the respective locating device. Upon leaving the locating device's operating range, the position and / or orientation and / or movement of the tool can be determined from a given initial situation using the sensor data acquired by the inertial sensors 29, 30.

[0062] Preferably, the locating device is designed as an ultrasonic locating device for detecting the object(s) to be located by means of ultrasonic waves and / or an additional and / or separate data transmission unit for transmitting process-relevant actual and / or target data is designed as a radio data transmission unit for transmitting process-relevant actual and / or target data by means of electromagnetic waves.

[0063] For example, the different propagation times of ultrasonic waves on the one hand and the (much faster) electromagnetic radio transmission on the other, or even electrical data transmission via electrical cables or electrical wires and / or optical data lines, especially fiber optic cables, can be used to determine the distance or exact position of the object being located and, if necessary, further process it using an (electromagnetic) trigger signal or, when a trigger signal is sent, by detecting the propagation time of the ultrasonic location of the location sensor(s). The new 5G wireless standard can also be used to determine the position.

[0064] Instead of the assembly line 1, in another embodiment, a large workpiece such as an aircraft can be provided for assembly, with several ultrasonic transmitters and receivers of a positioning device arranged stationary around the aircraft. One or more image acquisition units are then distributed throughout the work area such that each tool to be monitored is regularly guided into an image acquisition area. The image acquisition data is then used to calibrate one or more inertial sensors on the tool(s).

[0065] For this purpose, an image analysis unit (not shown in detail) is connected to a control unit for the inertial sensor(s), which in turn communicates with the inertial sensor(s). Data processing for an inertial sensor can be performed in the control unit or within the sensor itself. Accordingly, calibration also takes place either in the control unit or, after appropriate data transmission, within the sensor.

[0066] In combination with the ultrasonic locating device shown, the position and / or orientation and / or movement of the tool can be determined by comparing this locating data. For this purpose, a control unit for the locating device is available in conjunction with the control unit for the inertial sensor(s) and / or the image evaluation unit. The image evaluation unit, as well as the control units for the locating device and / or the control unit for the inertial sensor(s), can also be combined into one or two control units or connected to a central control unit.

[0067] The data from the inertial sensors can be used in combination with the positioning data, for example, to reduce the number of positioning sensors or to perform a data comparison. However, the calibration of the inertial sensors according to the invention is not limited to use in combination with a positioning device according to the described embodiments. List of reference symbols:

[0068] 1 assembly line 32 wireless data channel 2 pulley 33 Image evaluation unit 3 Upper drum 4 automobile 5 automobile 6 Tracking device 7 Tracking device 8 Location sensor 9 Location sensor 10 Location sensor 11 Communication unit 12 electromagnetic waves 13 screwdriver 14 glove 15 Assembly part 16 Location sensor 17 Inertial sensor 18 Inertial sensor 19 Communication unit 20 central control unit 21 Communication unit 22 Control unit 23 Advertisement 24 Signal generator 25 Location sensor 26 Location sensor 27 Location sensor 28 Communication unit 29 Inertial sensor 30 Inertial sensor 31 Image acquisition unit

Claims

1. Device having a tool (13), wherein the device is used to monitor the position and / or attitude and / or movement of the tool, wherein the tool has an inertial sensor (17, 18) such as a gyroscope sensor and / or an inertia sensor mounted on it, wherein the device has provision for a supervisory unit (20), at least one image capture unit (31) and an image evaluation unit (33), wherein the supervisory unit is designed to determine a position and / or attitude and / or movement of the tool, starting from a specified initial situation, on the basis of the captured sensor data from the inertial sensor, wherein the image capture unit is intended to calibrate the inertial sensor, wherein the image capture unit is designed to capture one or more photographs of the tool at least at specific times or in specific time intervals, wherein the image evaluation unit is intended to determine the position and / or attitude and / or movement of the tool, and wherein the supervisory unit is designed to calibrate the sensor data from the inertial sensor in order to process the data from the image capture unit, characterized in that the device has provision for a locating device, wherein the locating device is designed to have an ultrasonic transmitter and an ultrasonic receiver, wherein the locating device is designed to locate the tool by determining the distance between the ultrasonic transmitter and the ultrasonic receiver by way of time-of-flight measurements on ultrasonic signals, wherein the ultrasonic signals can be used to determine the distance between applicable transmitters and receivers, wherein the operating range of the locating device operating by means of time-of-flight measurement extends beyond an image area of the image capture unit and the detection of the presence of the tool in the image area of the image capture unit by the locating device is intended to locate the tool by way of time-of-flight measurements on the ultrasonic signals.

2. Device according to Claim characterized in that the supervisory unit is designed so that the data from the inertial sensor are matched against those from the image processing, and / or the inertial sensor is calibrated, when the tool is in the image area of the image capture unit.

3. Device according to either of the preceding claims, characterized in that the supervisory unit is designed so that the data from the inertial sensor are routinely matched against those from the image processing, and / or the inertial sensor is calibrated, when the tool is in the image area of the image capture unit.

4. Device according to one of the preceding claims, characterized in that the locating device comprises electromagnetic sensors.

5. Device according to one of the preceding claims, characterized in that the device has a data transmission unit, wherein the data transmission unit is designed not only for the data transmission of data from the inertial sensor but also for the data transmission of other data / information / parameters, in particular for the transmission of a trigger signal from the locating transmitter.

6. Device according to one of the preceding claims, characterized in that the tool is a handheld tool such as a screwdriver, a drill or the like.