Device and method for processing at least one working area by means of a processing tool

Abstract

The invention relates to a device for processing at least one working area of ​​an object, the device having a processing tool. An image recording device for generating at least one image of the at least one working area and at least one sensor for generating sensor data during the processing of the at least one working area are provided at the processing tool. The device comprises an evaluation unit having a computing model, which is trained by means of machine learning based on the images of the working area and the sensor data, with the aid of which the at least one working area and / or the situation during the processing of the at least one working area can be identified. In addition, the invention relates to a method for processing at least one working area of ​​an object.

Description

Technical Field

[0001] The invention relates to a device for processing at least one working area with a processing tool, and to a method for processing at least one working area by means of a processing tool. Background Art

[0002] A device for logging screw connections is known from DE 10 2012 219 871 A1. Here, after the screw is screwed in and the installation tool is removed from the screwing position, an image is automatically recorded by a digital camera. The recorded image contains the screw and its immediate surroundings.

[0003] When processing objects, for example by a human user of a processing tool, human errors may occur. It may therefore be necessary that the device includes a system for machine and / or automatic quality assurance in order to prevent, avoid and / or correct human errors. Summary of the invention

[0004] The invention is therefore based on the object of avoiding errors in the processing of at least one working area and of improving the quality of the processing.

[0005] This object is achieved according to the invention by a device and a method for treating at least one working area.

[0006] The device for processing at least one working area of ​​an object comprises a processing tool. Such processing can be, for example, industrial installation and in particular screwing and riveting on an object such as a vehicle, but is not limited thereto. In one embodiment, the processing tool can be formed by an installation tool such as an industrial screwdriver, a manual installation device, a gun screwdriver, a rivet gun, a clamping device, a drill, an axial screwdriver or an angle screwdriver.

[0007] An image recording device, such as a camera module, is arranged on the processing tool for generating at least one image of the at least one working area, and at least one sensor, such as an inertial sensor, is arranged for generating sensor data during processing of the at least one working area.

[0008] At least one image and sensor data can be used to dynamically identify a working area or working point where a processing tool can be used to act on the working area. At least one image can show the working area, a part of the working area with a working point and optionally additionally show a section of the processing tool, such as a tool tip. At least one image can be, for example, a thermal image or a 2D or 3D photo or view. In particular, at least one image can be the topography of the surface of at least one working area. At least one image can also be generated by scanning the surface. In one embodiment, scanning can be performed optically. For scanning, for example, ultrasound or laser scanning can be used. In particular, a laser scanner or a time-of-flight camera can be used for scanning.

[0009] Obviously, at least one image and / or sensor data can be generated before and / or after processing at least one working area. In one embodiment, at least one image and / or sensor data can be generated when the processing tool approaches at least one working area. It is also conceivable and feasible to generate at least one image based on sensor data. For example, the generation of at least one image can be started or ended by generating specific sensor data or a specific pattern of sensor data. Conversely, the generation of sensor data can also be performed based on at least one image. For example, a specific image or a specific pattern on an image can start or end the generation of sensor data by at least one sensor.

[0010] In a design scheme with a processing tool formed by an industrial screw machine, a sensor configured as a screw control device can measure whether the correct rotation angle has been reached. Therefore, for example, an erroneous rotation angle can be measured. When processing multiple working areas, the rotation angle can be measured by the screw control device for each working area. In another design scheme, at least one sensor can be formed by an inertial measurement unit (IMU), an orientation sensor, an acoustic sensor, an ultrasonic sensor, an acceleration sensor, a vibration sensor, a spacing sensor, a time-of-flight (TOF) camera, a QR code scanner, a barcode scanner or a magnetic field sensor, and the vibration sensor can especially detect the gap of the tool part, especially when the object is processed by the processing tool. At least one magnetic field sensor can especially determine the position and / or orientation of the processing tool relative to the object and / or the working area according to the earth's magnetic field. Here, the magnetic field sensor can be used to modify the earth's magnetic field differently in different working areas, that is, to have a legend in the earth's magnetic field. Of course, multiple sensors are conceivable and feasible, and they are configured differently. Sensor data can, for example, include location orientation data. The location-specific data can be generated in particular by means of marker-based tracking or by utilizing an existing radio infrastructure, which comprises in particular wireless technologies such as WLAN, Bluetooth, Ultra Wideband or ZigBee.

[0011] In particular, the generation of at least one image can thus be started or ended by a distance measurement. For example, the recording of at least one image can be started by a preceding laser distance measurement.

[0012] The device further comprises an evaluation unit, for example a processing device, such as an industrial PC. The evaluation unit can include software, which includes modules for recording, processing and evaluating images. For the evaluation, the evaluation unit has a calculation model, which can be used to evaluate at least one image and sensor data. With the aid of the calculation model, at least one working area and / or a situation can be identified during the processing of at least one working area. Thus, with the aid of the evaluation by the calculation model, the processed or to be processed working area can be determined. The evaluation unit can therefore also be used to systematically and / or statistically analyze the surroundings based on at least one image and / or sensor data.

[0013] Furthermore, the processing point to be processed or processed of the at least one working area can be determined by means of the calculation model. In addition, the preconditions for processing the working point can be determined with the aid of the calculation model. The preconditions are, for example, the position or location of the tool part to be used, the component to be installed and / or the processing tool. With the aid of the calculation model, the relative position of the processing tool with respect to the working point or the at least one working area can also be determined with the aid of the at least one image and / or sensor data.

[0014] The computing model has been trained by means of machine learning on the basis of images and sensor data of the working area. The training can be carried out before the first processing of an object by the processing tool. During the training, in one embodiment, in a first step, an image is recorded as a reference image for each working area and / or working point. In particular, different situations and combinations of images and sensor data can be recorded during the processing of at least one working area. In addition, the (representative) position, orientation and / or angle of the processing tool relative to the working area and / or working point, in particular the tool position, can be recorded.

[0015] In one embodiment, the machine learning includes at least one neural network, at least one convolutional neural network, at least one bag-of-words and / or at least one random forest. Other methods of machine learning are of course conceivable and feasible.

[0016] In a second step, in one embodiment, the computational model can generate at least one classifier from at least one image and / or sensor data. At least one classifier can represent at least one main feature of at least one image and / or sensor data. The main feature can be image information and / or sensor information, which is more important than other image information and / or sensor information. The main feature can be, for example, a segment of at least one image and / or sensor data, which is more different from adjacent segments than other adjacent segments of at least one image and / or sensor data. Optionally, the main information can be used to identify a specific working area from different angles, orientations and / or positions of a processing tool. In particular, the main feature can also be used to identify, by an evaluation unit, when the processing tool moves relative to the processed or to-be-processed working area, that the working area is the same (feature tracking) as long as the main feature exists. The movement of the processing tool relative to the processed or to-be-processed working area can, for example, be a change in orientation or position, in particular angle and / or spacing from the working area to be processed.

[0017] In one embodiment, a dark circle on a bright surface is a primary feature of at least one image. Likewise, a series of images can contain primary features that can be represented by at least one classifier. Likewise, a combination of at least one image and sensor data can contain primary features that can be represented by at least one classifier.

[0018] In particular, the computational model can optically distinguish similar working areas in which the images are identical based on the sensor data. For example, at optically identical screwing locations (due to the symmetry of the object), the screwing locations can be distinguished based on the angle of the processing tool relative to the object.

[0019] In one embodiment, the computing model can recognize at least one working region based on at least one image and / or sensor data by means of at least one classifier. In one embodiment, the computing model can determine a probability level of the recognition. The probability level can indicate with what probability (confidence level) the at least one working region is a recognized working region. For example, the probability level of a correct recognition can be between 0.00 (low probability level) and 1.00 (high probability level).

[0020] In a variant, it is additionally possible to determine an alternative degree of probability that at least one working area is a working area other than the identified working area. The reliability of the identification can be estimated from the difference between the degree of probability of the identified working area and the degree of probability of the alternative. The reliability of the identification can hereby increase with the size of the difference. The reliability of the prediction can thus be quantified by the fractional difference between the best prediction and the second best prediction as to whether the working area is the identified or alternative working area. If the difference is large, then the prediction is very reliable. If the difference is slight, then the risk of confusion and hence also the unreliability increases.

[0021] Additionally, the fulfillment of a precondition can be determined by means of the calculation model using at least one classifier based on the at least one image and / or the sensor data.

[0022] In one embodiment, at least one classifier can provide a pixel-by-pixel evaluation, for example in the form of a heat map. The evaluation can indicate which part of the current image has a particularly high consistency and / or difference with already known images in the training data set. The computing model can therefore be configured to visually superimpose evaluation information on at least one image. The evaluation information can be based on the similarity of at least one segment of at least one image to at least one classifier and / or an image of the working area that has been used to train the computing model. For example, the evaluation information can indicate how similar at least one segment is to at least one classifier and / or an image of the working area that has been used to train the computing model.

[0023] At least one image with the visually superimposed evaluation information can be displayed to the user. The evaluation information can be used to determine anomalies in the at least one image. The anomaly can be, for example, an (unexpected, random) covering of a section of the at least one image. For example, the objective lens of a camera of the image recording device may be (randomly, unexpectedly) covered by an object or a body part of a user. Such an object may be, for example, a cable, a line and / or a rope. Likewise, irregularities of at least one working area or a processing tool, in particular a tool part, can be detected via the motion information.

[0024] In one embodiment variant, the evaluation unit is designed to identify static areas on at least one image, which do not change over a series of multiple images despite a movement of the processing tool relative to the object. The static areas of at least one image can be identified during a training process, i.e., during training of the computational model, via all training images. The static areas can be, for example, sections of tool parts, in particular processing tools. Based on the static areas, the error-free functioning of the image recording device can be controlled. If, for example, the static areas change with respect to imaging parameters such as brightness or sharpness, then there can be a camera malfunction, camera contamination, incorrect camera position and / or incorrect camera settings.

[0025] In one embodiment, the computational model extracts features from at least one image, weights and subsequently compresses the features in order to find the main features, and generates at least one classifier from the at least one main feature. The generation of at least one classifier can, for example, result in the computational model identifying at least one working area based on the shape and / or arrangement of holes in the at least one working area when identifying at least one working area based on the at least one image. Training the computational model can also include identifying the already trained working area by the computational model and using the result (correctly or incorrectly) to improve the computational model (supervised learning; back propagation). The weighting of the features can thereby be improved.

[0026] In an embodiment in which the computation model is initially untrained, the computation model can be trained as follows: In a first step, at least one image is generated from at least one working point by means of an image recording device. In a second step, the at least one image is checked to see whether the computation model is able to recognize at least one working point based on the at least one image. In the event of an erroneous recognition, in a third step, the at least one image is added to an image data set or a data pool for at least one classifier, with the aid of which the at least one classifier is retrained. The steps are repeated until at least one working point is correctly recognized by the computation model or with a desired degree of probability, for example greater than 0.75. In one variant, the at least one working point is designed as a screw point.

[0027] In one embodiment, at least one image is used to correct the sensor data by the evaluation unit. For example, the measurement of acceleration, angle, especially the orientation and / or position relative to at least one working area by at least one sensor, especially an inertial sensor, can be corrected with the aid of at least one image. For correcting the sensor data, for example, (main) features, such as holes or edges of an object, especially of at least one working area, which are identified on at least one image can be used. The correction can, for example, include correcting the measured acceleration according to the movement of the main features on at least three images. Similarly, the rotation direction of the processing tool can be corrected with the aid of at least one image by the measurement of at least one sensor.

[0028] In another design, tool parts, such as extensions, sprockets and / or drills can be determined by an evaluation unit, in particular by means of an image recording device and / or at least one sensor. The evaluation unit can, for example, identify, based on at least one image and / or sensor data, what size or type of drill is provided on the processing tool. In particular, tool parts can be determined based on characteristic patterns, labels and / or markings on the tool parts. For example, the marking can be optical. However, other, in particular magnetic barcodes or QR code markings are also conceivable and feasible. In addition, the wear of tool parts can be identified by an evaluation unit, in particular by an image recording device and / or at least one sensor. For example, the wear of tool parts can be identified based on the gap of tool parts, in particular screw machine / drill system. The evaluation unit can be configured to, for example, determine the use time of the tool parts based on the gap of the tool parts or based on the wear, and the use time indicates how long the tool parts have been used.

[0029] In one embodiment, the orientation or position of the processing tool, for example the angle relative to at least one working area or the position relative to the object, can be determined by the evaluation unit.

[0030] Furthermore, components, such as screws, can be determined by the evaluation unit. In particular, errors at the object, i.e., the absence of a component, an erroneous component, an erroneous position of a component at the object and / or an erroneous geometry of the component can be detected by the evaluation unit. Furthermore, the evaluation unit can be designed to detect other deviations, such as color deviations, of at least one working area. In a further embodiment, the evaluation unit is designed to identify an area of ​​the object that is not to be processed and / or to distinguish it from at least one working area.

[0031] In a variant, an indicator device is provided, which is designed to display to a user of the processing tool the necessary replacement of at least one tool part of the processing tool. For example, the indicator device can display to the user that a tool part should be replaced when the wear of the tool part exceeds a predetermined degree. Similarly, the indicator device can display that a tool part different from the tool part provided on the processing tool at that time is required to process a working area. In a variant, the indicator device can monitor the replacement of at least one tool part by means of evaluating at least one image and / or sensor data. For example, the indicator device can monitor the user when replacing a tool part and indicate errors during the replacement if necessary based on at least one image and / or sensor data.

[0032] In one embodiment, the computing model is configured to identify installation steps in the installation process based on a series of images and / or sensor data. A series of images can be formed, for example, by a plurality of images, which are generated at time intervals relative to each other. Similarly, a series of sensor data can be formed, for example, by a plurality of sections of sensor data, which are generated at time intervals relative to each other. The installation step can, for example, include changing the orientation and / or position of a processing tool and / or an action or a series of actions on at least one working area by a processing tool. The installation process can be a series of installation steps. In one design, the installation step includes tightening at a working point such as a screwing position.

[0033] In one embodiment, the evaluation unit is configured to select a key area of ​​at least one image based on a user input to train the computational model. The key area can be an image area (region of interest) that is used by the computational model to generate at least one classifier. The image area can contain at least one main feature in particular. The selection of the key area can be used to not observe the background of at least one image in order to train the computational model. The background can be, for example, an area that has fewer features than an area that is arranged closer to the working point than the background (along the axis from the processing tool to the object). For example, the selection of the key area can save time for training the computational model. In principle, the background can also be used to identify at least one working area by the computational model, for example by means of different depths of field in different working areas.

[0034] In an alternative embodiment, the background, in particular the far field, can be marked automatically by means of the evaluation unit, for example as a function of a threshold value for the depth of field. The marked areas are not taken into account for the evaluation.

[0035] In one embodiment, the distance between the processing tool and the object can be measured by means of a distance measuring module of the processing tool. In one embodiment, the distance measuring module comprises a laser beam aligned with the working point in combination with a camera. With the help of the distance measuring module, the installation depth to be reached can be determined. Therefore, it is possible to add a further quality statement, with the help of which the successful processing of at least one working area can be identified. For example, it can happen that in the case of an incorrect screwing, the predefined parameters have been reached and the qualified screwing is indicated by the evaluation unit. The measurement of the installation depth can be used in this case to determine that the component to be installed, for example the head of a screw, in particular the screw head, has not yet been laid flat and that an incorrect installation, for example an incorrect screwing, has thus occurred.

[0036] The distance measurement module can include, for example, a TOF camera, with which a time-of-flight measurement (TOF) can be performed. Optionally, the distance of the background in different working areas can be determined with the help of the distance measurement module. Here, the distance of the background can also be used to identify at least one working area by means of a calculation model.

[0037] In another embodiment, the evaluation unit is designed to select a critical region of the at least one image based on a user input for identifying the at least one working region. The user can select the critical region, for example, during processing of the working region by a processing tool.

[0038] In one embodiment, the image recording device is configured to generate a plurality of images before, during and / or after the processing tool acts on at least one working area. In principle, the image recording device can generate images at any time. In a variant, the image recording device generates at least one image when the processing tool acts on at least one working area for processing. The image recording device can be configured to automatically start and / or end the generation of images based on sensor data. For example, when the processing tool is removed by the user, the image recording device can end the generation of images. The removal of the processing tool can be measured, for example, by means of an (inertial) sensor. When the user approaches the processing tool to at least one working area, the image recording device can, for example, start the generation of images.

[0039] In one design, the number of images generated per unit time by the image recording device is variable. The image recording device can therefore change the rate of image recording. Similarly, the (wireless) transmission rate of images and / or sensor data from the image recording device and / or at least one sensor to the evaluation unit, i.e. the frequency of transmission, is variable. For example, when the processing tool is removed, the transmission rate can be small, and when at least one working area is processed by the processing tool, the transmission rate can be high. In one design, the time and path when the processing tool is periodically removed can be estimated with the aid of sensor data. In particular, a transmission with a high frequency can be started when processing at least one working area is started or when approaching at least one working area. In principle, images and / or sensor data can be transmitted from the image recording device and / or at least one sensor to the evaluation unit at a low frequency at any time when the device is running.

[0040] In one embodiment, the image recording device comprises at least one camera. The at least one camera can be used to record at least one image. In order to record the enlarged image, the at least one camera can include a zoom lens. The at least one camera can then enlarge the at least one working area. Obviously, the at least one camera can use the zoom lens to record the reduced image. The at least one camera can then be pulled out from the at least one working area.

[0041] The image recording device, in particular at least one camera, can be arranged on the processing tool by means of an adapter. The adapter can be designed specifically for the processing tool. Via the adapter, the image recording device, in particular at least one camera, can be arranged on different types of processing tools. Different types of processing tools can be distinguished by different shapes, for example round or angular cross sections, so that an adapter is required to mount the image recording device.

[0042] In one embodiment, at least one camera is adjustable relative to the processing tool. For example, at least one camera can be movably and / or rotatably supported on the processing tool relative to the processing tool. At least one camera can, for example, rotate around a tool axis and / or move along the tool axis. The tool axis can extend along the processing tool substantially from a handle for gripping by a user toward a tool tip for processing at least one working area.

[0043] In one configuration, at least one camera is designed as a barcode reader or can be used to read a barcode.

[0044] In another embodiment, the image recording device comprises at least two cameras to simultaneously generate at least two images of at least one working area. The images generated by the at least two cameras can show overlapping, identical or different areas of the at least one working area. The at least two images can also be stitched together to form a panorama (stitching), fused or merged with each other (in terms of content) and / or formed as a stereogram. The stereogram can be used in particular to calculate depth information. The at least two cameras can thus generate a 3D view. With the aid of the 3D view, the near field and the far field, i.e., for example, at least one working area and the background, can be distinguished.

[0045] In one embodiment, the image recording device comprises at least one objective lens. The at least one objective lens can include the end of at least one glass fiber. Through the at least one objective lens, the image recording device records at least one image. At least one optical axis of the at least one objective lens can intersect with a working point, in which the processing tool acts on at least one working area. Similarly, at least one detection axis of the at least one sensor can intersect with the working point. At least one detection axis of the at least one sensor can, for example, be a direction along which the at least one sensor is oriented.

[0046] In an alternative embodiment, at least one optical axis of at least one objective lens of the image recording device and / or at least one detection axis of at least one sensor extend parallel to the action axis, and the processing tool acts on at least one working area along the action axis. The parallel orientation of at least one optical axis and the action axis can realize the line of sight direction parallel to the axis. In a variant form, multiple objective lenses and / or multiple sensors of the image recording device are arranged annularly around the action axis. For example, the parallel orientation of the action axis and at least one optical axis and / or at least one detection axis can be integrated in an annular component, such as an adapter ring, by means of at least one camera and / or at least one sensor. The annular component can be arranged on the processing tool. The action axis can extend through the annular opening of the annular component. In addition, the annular component can include at least one objective lens.

[0047] In one embodiment, a connecting straight line between two objective lenses, in particular between the ends of two glass fibers, and / or between two sensors intersects the active axis. Thus, the two objective lenses and / or the two sensors are arranged opposite each other with respect to the active axis.

[0048] In one embodiment, the image recording device comprises at least one lighting device. With the aid of the lighting device, at least one working area can be illuminated. In a variant, only one section of at least one working area can be illuminated by the lighting device. Likewise, the lighting device can illuminate a processing tool or at least a part thereof. In particular, the lighting device can illuminate the following tool parts of the processing tool, with the aid of which the processing tool acts on at least one working area. The lighting device can, for example, comprise at least one light source. The spectrum of the at least one light source can in principle include any arbitrary wavelength or wavelength spectrum. In particular, the at least one light source can emit full-spectrum or infrared light. As a result, the device can be independent of external lighting, in particular lighting of the workshop.

[0049] The at least one light source can emit light synchronously with the generation of the at least one image, in particular with the recording of the at least one image by the at least one camera. Likewise, the depth information of the image recording device can be used to control the at least one light source. For example, the far field can be illuminated less than the near field. Far away areas can therefore be illuminated less.

[0050] In principle, at least one light source can of course emit light at any arbitrary time point and at any arbitrary time length and / or in any arbitrary time interval. In particular, at least one light source can emit light to mark the position of at least one working area, in particular to mark the working point to be processed. The light source can be configured as a projector, for example. Then, with the help of the light source, at least one pattern can be projected onto at least one working area. At least one pattern can, for example, contain a sign or direction indication of a section of at least one working area. At least one pattern can be projected onto at least one working area by a processing tool in any direction. Therefore, the risk of incorrectly placing the processing tool by the user can be reduced. In addition, the learning time for new staff can be reduced.

[0051] In one embodiment, the lighting device comprises a projection unit, by means of which the working point to be processed can already be identified and marked when the processing tool approaches the working area.

[0052] In one embodiment, the device includes a simulation system, by means of which at least one working area can be simulated to generate at least one image, sensor data and / or at least one classifier. For example, by means of the simulation system, a working area for training can be simulated for an evaluation unit. To this end, the simulation system can provide at least one image and / or sensor data to the evaluation unit. The evaluation unit can then generate at least one classifier from at least one image and / or sensor data. In a variant, the simulation system derives at least one classifier from at least one simulated image and / or sensor data. At least one classifier can then be provided for identifying at least one working area. For example, at least one classifier can be provided to the evaluation unit and / or transmitted to a classifier memory (classificator repository). In particular, the simulation system can simulate a 3D model of at least one working area and provide it to the evaluation unit for training. To this end, the simulation system can include a system for computer-aided design (CAD system). The 3D model of at least one working area can be associated with a computational model. In particular, the 3D model of at least one working area can be selected according to a machine learning method.

[0053] By means of the simulation system, the working area of ​​the object can be (virtually) trained, i.e. optionally before the object physically exists. Likewise, multiple variants of the working area can be virtually trained. In one embodiment, the simulation system can simulate different stereograms of the image recording device. The different stereograms of the image recording device can be, for example, different viewing angles of at least one camera of the image recording device. Stereograms that deviate from the training may occur, for example, due to changes in tool parts, such as the use of extensions. By means of the simulation system, at least one classifier can be regenerated for different stereograms.

[0054] In one embodiment, at least one working region is simulated by means of a simulation system while the processing tool acts on the at least one working region, ie in real time. The real-time simulation enables the calculation model to be adjusted during the processing of the working region, in particular to generate additional classifiers.

[0055] In a variant, at least one classifier is called up by the simulation system, in particular for selected working areas and / or working points, in particular important screwing locations (A- and B-screws). For example, at least one classifier can be called up by means of an application of the evaluation unit.

[0056] In one embodiment, the image recording device includes a mobile terminal device in order to generate an image of at least one working area. In particular, a mobile device, a mobile phone or a smart phone. The mobile terminal device can be arranged on the processing tool. Of course, it is not excluded that the image recording device includes at least one other camera to generate an image of at least one working area. Additionally or alternatively, the mobile terminal device can include at least one sensor. At least one sensor of the mobile terminal device can, for example, determine the position, orientation and / or location of the processing tool to generate sensor data. In order to transmit images and / or sensor data, the mobile terminal device can be connected to the evaluation unit.

[0057] In a variant, the mobile terminal generates sensor data, which shows the status and screwing data of the processing tool, especially the screw machine. The status of the processing tool can include, for example, the power used, the switching-on and switching-off data, the torque, the screwing time, the screwing curve, the rotation speed, the angle relative to the object and / or other settings of the processing tool. The screwing curve can be, for example, the ratio of the torque of the processing tool at the screw to the rotation angle of the screw or to the screwing time. The mobile terminal can be connected to the processing tool wirelessly, especially by means of WLAN, for example, to generate the sensor data.

[0058] In another embodiment, the mobile terminal device can receive the classifier from a classifier memory and use it to identify at least one working area based on the generated image and sensor data.

[0059] In one embodiment, the generated images and sensor data can be stored together with the identification of the processed object and / or the processed work area. The object or the work area can be identified one-to-one based on the identification. The images and sensor data can thus be used for quality assessment over a long period of time. In particular, a statistical analysis of the service branch and the service personnel can be carried out with the aid of the quality assessment.

[0060] In one embodiment, the evaluation unit is configured to recognize at least one unknown image and / or at least one unknown segment of sensor data in a series of multiple known images and / or known sensor data, which contain at least one unknown image and / or at least one segment of unknown sensor data. A series of known images can be, for example, a sequence of images during the processing of an object, within which the evaluation unit recognizes all images again. Re-recognition can mean that the probability degree of recognition is higher than a certain threshold value. The threshold value can be, for example, 0.75 or 0.95. The evaluation unit can recognize the image again in particular when the image has been trained before in training. At least one unknown image can be, for example, an image or a sequence of images, which has not been trained before in training. The same applies to a series of known sensor data. The duration of the segment of the unknown sensor data can be less than the duration of the preceding and succeeding known sensor data.

[0061] In one embodiment, the evaluation unit can also be designed to recognize at least one known working area based on at least one unknown image and / or at least one segment of unknown sensor data by means of machine learning training. In particular, at least one unknown image and / or at least one unknown segment of sensor data can be used to generate at least one classifier. It can thereby be achieved that at least one unknown image and / or at least one unknown segment of sensor data is received in a data pool for the next classifier cycle. At least one classifier can be generated or improved from the data pool in a classifier cycle. This can increase the probability of recognizing the working area.

[0062] In other words, at least one classifier can be continuously improved during operation of the device. To this end, the device can collect images during operation for later training.

[0063] In a first variant, the computational model is designed to be trained by itself. This is done by unsupervised learning, i.e. without the need for experts. At least one image can be used for post-training. A prerequisite for this is that at least one working area is identified by the computational model with a high degree of probability, for example a degree of probability greater than 0.75, based on at least one image. Thus, in principle, every image that a classifier can predict with high reliability can be used for post-training. It is assumed here that the prediction is correct. By the computational model being trained by itself, the overall degree of probability for correctly identifying a plurality of working areas can be increased. Thus, the overall reliability of the prediction can continue to increase over time.

[0064] In a second variant, the computational model is designed to be trained by a user. This takes place by supervised learning, i.e. with the aid of an expert. In this case, at least one image can be used for post-training. A prerequisite for this is that at least one working area is identified by the computational model with a low degree of probability, for example with a degree of probability of less than 0.45, based on at least one image. Thus, in contrast to the first variant, images are collected in which the classifier has a high degree of uncertainty in the prediction. The user can train the computational model by pre-setting the identification of at least one working area, and the at least one working area is identified based on at least one image. The at least one image is thus post-calibrated by the expert, i.e. associated with a correct identification, and can then be used for post-training.

[0065] Of course, in both variants, sensor data can be used for post-training instead of or also in addition to the at least one image.

[0066] In principle, the evaluation unit can also be designed to recognize at least one unknown combination of at least one image and at least one section of the sensor data. It is conceivable and feasible that the at least one image and at least one section of the sensor data are respectively known. Furthermore, the evaluation unit can then be designed to recognize at least one known working range from the at least one unknown combination by means of machine learning training.

[0067] In one embodiment, a monitoring unit is provided, which is designed to document a series of working areas processed by the processing tool. The monitoring unit can thus document which working areas have been processed and in what order the working areas have been processed. In addition, the monitoring unit can document the tool parts used, the components used, such as screws, at least one image and / or sensor data. The use of the components can be documented by the monitoring unit, for example, for planning the logistics of the components or as a proof of use. In addition to the documentation of the components used, the position and / or environment of the components can be documented.

[0068] In one variant, the monitoring unit is designed to predetermine a series of working areas and / or working points for processing by the processing tool. The monitoring unit can be designed in particular to group at least two working areas of the series of working areas for processing by the processing tool. For example, the monitoring unit can group working areas or, for a working area, working points that are to be processed with the same state of the tool part and / or the processing tool, such as an angle or torque relative to the object.

[0069] In one embodiment, the monitoring unit can associate the incorrectly processed working area during the processing of the working area or after the processing of a plurality of working areas or a group of a plurality of working areas. If necessary, the monitoring unit can specify the incorrectly processed working area for reprocessing. Likewise, the monitoring unit can detect an incorrect sequence when processing the working areas.

[0070] In order to document the processed working area and / or working point, the monitoring unit can be connected to the processing tool, the image recording device, the at least one sensor and / or the evaluation unit for exchanging data, for example via a radio or cable connection.

[0071] In one embodiment, a control unit is provided, which is connected to the processing tool, the monitoring unit and / or the evaluation unit and by means of which the processing tool can be controlled. The state of the processing tool, such as the rotational speed or the torque, can be adjusted by means of the control unit. Obviously, the state of the processing tool can be adjusted automatically according to the identified working area by means of the device. By means of the dynamic identification of the working point actually to be processed, the state of the processing tool can be adjusted in real time. In particular, it is also possible to group the working points and / or working areas with the same requirements for the state of the processing tool by means of the monitoring unit.

[0072] In another embodiment, a signaling device is provided for outputting a signal to a user of the processing tool. By means of the signal, it is possible to signal to the user that at least one working area has not been identified, at least one working area has been processed incorrectly and / or at least one working area of ​​a series of working areas has not been processed. The signaling device can, if necessary, signal the user of the processing tool that trimming is necessary at at least one working area. The signaling device can also signal the user that the evaluation unit was not able to identify the working area. In a variant, the signaling device signals the user a result determined by the evaluation unit and / or the monitoring unit. The signaling device can therefore in principle be used to output a signal of the monitoring unit and / or the evaluation unit to the user. In an embodiment in which the evaluation unit has determined that a tool part has been used for longer than a predefined usage period, the signaling device can, for example, output a signal for replacing the tool part.

[0073] The signal can be output, for example, at the processing tool. For example, the signal can be sound, light or vibration. Of course, any signal is conceivable and feasible.

[0074] In one embodiment, the device comprises a test work area for calibrating the device. A user can, for example, calibrate the image recording device and in particular at least one camera with the aid of the test work area. The evaluation unit can be designed to automatically recognize that calibration is required. The test work area can also be designed to recognize damage and / or contamination at the processing tool. In order to recognize damage and / or contamination, a standard processing procedure can be performed at the test work area. A deviation of at least one image and / or sensor data from the expected at least one image and / or expected sensor data can be an indication of damage and / or contamination. A possible process can include one or more of the following steps: 1. Move closer to the test work area. 2. Check the function of the processing tool. 3. Calibrate the processing tool. 4. Align the image recording device with the test work area. 5. Identify contamination and / or damage. 6. Eliminate contamination and / or damage.

[0075] In one embodiment, the device comprises at least one interface, which is configured to be connected to another system. Information can be exchanged between the device and the other system via the interface. For example, the interface can be designed as an application programming interface (API). In particular, the device can be reached remotely, for example by a server or a mobile device, by means of the interface. Likewise, the device can be managed or controlled centrally, for example by a server, by means of the interface.

[0076] In a further embodiment, the processing tool comprises a display for displaying textual information, for example at least one image, sensor data and / or parameters, settings or states of the processing tool to a user, in particular after identification of at least one working area.

[0077] In one embodiment, the use of the device can include the following optional steps: In a first step, a person searches for an object, such as a motor vehicle, an authorized workshop. The person needs to perform repairs on the object (warranty or critical component). In a second step, the authorized workshop registers the repair with the aid of the device at a server, such as a server of an OEM network. In a third step, the authorized workshop may obtain the required components, such as spare parts for the object. In addition, the authorized workshop obtains permission for the repair. In a fourth step, the device obtains at least one classifier from the server via an interface, which is optionally loaded into a classifier memory. In principle, the device can obtain data for monitoring and / or ensuring orderly processing.

[0078] In the fifth step of this embodiment, the processing of the object, such as installation or maintenance, is started. A processing tool is provided for this. The processing tool is known to the OEM, connected to the server via an interface and / or obtains OEM permission. In the sixth step, at least one classifier is loaded. At least one classifier is provided to the evaluation unit. At least one classifier can be specific to an individual's motor vehicle. In the seventh step, the user sets a device, such as a factory-MA, an image recording device on the processing tool. The image recording device can be a mobile camera or a mobile phone, for example. In the eighth step, the user performs processing on the object with a preset value, i.e., a preset state and / or a preset situation by means of the processing tool. The processing is optionally archived via a monitoring unit by means of a processing tool, in particular an image recording device and / or at least one sensor (identifiable in the environment by a classifier). In the ninth step, data for monitoring and / or ensuring orderly processing are compared with at least one image and / or sensor data by an evaluation unit and / or a monitoring unit. The consistency of the data and the image confirms successful processing.

[0079] In one variant, at least one classifier is provided for carrying out a diagnosis of the object. By means of the at least one classifier, for example, faulty locations on the object can be identified.

[0080] In another variant, at least one classifier can be optimized by means of current diagnostic values. For example, when erroneous or unlearned locations in at least one working region of the object are determined, the at least one working region can be relearned.

[0081] In an additional variant, the OEM can provide at least one currently valid classifier as a function of a one-to-one identification code of an object, in particular a motor vehicle, via an interface, for example via an intranet connection.

[0082] In principle, the system can also be used for conventional diagnostics. A trained work area with at least one structure, for example at a motor vehicle, can be used to detect problems, such as missing, displaced and / or defective components. For example, during a test run with a motor vehicle suspected of having a problem, the image recording device is directed to the area that may have a problem. This area can be visually checked with the aid of the image recording device during the test run under driving conditions. In one embodiment, the device can also detect components that are deformed under wind loads and / or accelerations and other external influences, in particular during the test run. For example, inadmissible deformations due to damage can be detected with the aid of the device. In addition, the deformations can be registered and documented. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The method according to the invention can of course be used here using the device proposed according to the invention, so that the advantages and features of the embodiment variants of this device mentioned above and below also apply to the embodiment variant of the proposed method, and vice versa.

[0084] An exemplary embodiment is shown below. Here, it is shown:

[0085] Figure 1 An apparatus for processing at least one working area of ​​an object is shown;

[0086] Figure 2A Demonstrating the training of a computational model and the application of a computational model;

[0087] Figure 2B shows the generation of classifiers;

[0088] Figure 2C showing the confidence with which the working area can be identified in the angle of the processing tool;

[0089] Figure 3A A device having a classifier memory is shown;

[0090] Figure 3B A device with a simulated system is shown;

[0091] Figure 4 A device having a control unit is shown;

[0092] Figure 5 A device is shown with a monitoring unit, a signaling device and a simulation system;

[0093] Fig. 6A A processing tool having an image recording device with an illumination device is shown;

[0094] Figure 6B A device having a lighting device is shown;

[0095] Figure 7 A device having an optical axis and a detection axis is shown;

[0096] Figure 8 A device is shown having a detection axis and an optical axis parallel to the action axis;

[0097] Fig. 9 An apparatus for determining a component is shown;

[0098] Fig.10 An apparatus having a tool component is shown;

[0099] Fig.11 shows a cross-sectional view of a device having an image recording device with two objective lenses;

[0100] Fig.12 A perspective view showing a processing tool in an embodiment as an industrial screw machine;

[0101] Fig.13 A side view showing a processing tool in an embodiment as an industrial screw machine;

[0102] Fig.14 A perspective view showing a processing tool in an embodiment as an industrial screw machine for processing vehicles;

[0103] Fig.15A A perspective view showing a processing tool with tool components in an embodiment as an industrial screw machine;

[0104] Fig. 15B A perspective view showing a processing tool in an embodiment of an industrial screw machine for processing a vehicle seat;

[0105] Fig.16A The working area at an angle of 0° is shown with the main features;

[0106] Fig. 16B The working area at an angle of 3° is shown with the main features;

[0107] Fig. 16C The working area at an angle of 8° is shown with the main features;

[0108] Fig.17A showing components that are missing from the motor vehicle; and

[0109] Fig. 17B The subsequently installed components in the motor vehicle are shown. DETAILED DESCRIPTION

[0110] Figure 1 A device with a processing tool 1 is shown for processing a working area A of an object B. The working area A extends over the surface of the object B. In the embodiment variant shown, the working area A is rectangular. In principle, any shape of the working area A is conceivable and feasible. A working point P is provided within the working area A, with which the processing tool 1 interacts. The processing tool 1 can be, for example, a screwdriver, a torque wrench or other tool. The processing of the object B can be carried out, for example, for installation or repair.

[0111] An image recording device 12 is provided on the processing tool 1 for generating at least one image 121 of a working area A of the object B. Furthermore, a sensor 13 is provided on the processing tool 1 for generating sensor data 131. The device further comprises an evaluation unit 2 having a calculation model 21.

[0112] The computing model 21 is trained by means of machine learning based on the images 120 and sensor data 130 of the working area A, as in Figure 2A As shown in . Based on the image 121 of the working area A, the working area A can be identified during the processing of the working area A with the help of the calculation model 21. Of course, the working area A can be identified with the help of the calculation model 21 before processing, for example, when the processing tool 1 is close to the working area A. Based on the sensor data 131, the working area A can also be identified. In particular, the working area A can be identified via the position, acceleration or orientation of the processing tool 1 in space. For this purpose, the sensor 13 can be configured as a position sensor, an acceleration sensor or an orientation sensor, for example. The working area A can also be identified via an acoustic profile, that is, a change curve of the sound signal with respect to time. In particular, the working point P can be identified based on the sound that occurs when the processing tool 1 is placed on the working point P. For this purpose, the sensor 13 can be configured as an acoustic sensor. In addition, the working area A can also be identified based on a combination of at least one image 121 and sensor data 131.

[0113] The calculation model 21 can also be used to identify situations during the processing of at least one working area A. The situation can be characterized during the processing of at least one working area A by features of the processing tool 1, such as the state, in particular the torque, the sensor data 131 and the image 121 of the working area A, individually, in combination or in summary.

[0114] During training, computer model 21 forms classifiers 221 , 222 , 231 , 232 based on sensor data 130 , images 120 and / or features of processing tool 1 , such as states. Figure 2B The formation of a classifier 221 based on the image 120 is shown. The image 120 shows a working area A, which is processed by the processing tool 1, wherein the tool part 11 interacts with the working area A. In a first step S1, the computing model 21 extracts features and forms feature associations. The features are weighted and compressed in a second step S2. From the results of the weighting and compression, the computing model 21 forms a feature vector in a third step S3. The feature vector is used to generate a classifier.

[0115] When applying, Figure 2AAs shown in , the computing model 21 utilizes the images 120 and sensor data 130 trained during training, in particular the classifier 221, in order to recognize the working area A and / or the situation based on the images 121 and the sensor data 131. In a variant, the computing model 21 additionally takes into account the combination of the images 121 and the sensor data 131. The recognition of the working area A based on at least one image 121 can then be related to the sensor data 131. For example, the position or orientation of the processing tool 1 determined during training is necessary in order to recognize at least one working area A based on at least one image 121. In this way, in particular, an erroneous or wrong position and / or orientation of the processing tool 1 can be determined. In a variant, the evaluation unit 2 can preset the permissible position and / or orientation of the processing tool 1 for the user, for example via the signaling device 5. In addition, the computing model 21 can relearn the erroneous position and / or orientation, for example as a reaction to a user input.

[0116] The computing model 21 can also utilize classifiers formed or input in another way, for example, on another evaluation unit, in order to recognize the working area A and / or the situation based on the image 121 and the sensor data 131. In addition, the computing model 21 can be designed to relearn the working area A or the situation in the context from the unknown image 121' and / or the unknown sensor data 131'. The context can be, for example, a series of known images 121 and / or known sensor data 131, which contain unknown images 121 and / or sensor data 131. After the unknown images 121' and / or the unknown sensor data 131' have been relearned from the working area A or the situation by the computing model 21, the working area A or the situation can be recognized by the computing model 21.

[0117] Figure 2C The angles of the processing tool 1 between 0° and 360° relative to the working area A are shown as being related to a probability level, which indicates with what degree of confidence the working area A was identified by the calculation model 21 based on at least one image 121. The angle represents an arbitrary parameter here and in particular any arbitrary state and / or any arbitrary situation is considered. Of course, it is also conceivable and feasible to carry out such a correlation for the sensor data 131. The following embodiments therefore also apply to the sensor data 131 and to the combination of the sensor data 131 and the image 121.

[0118] The inner area 251 indicates what degree of probability the images detected so far have reached. The middle area 252 indicates what degree of probability may be achieved for at least one image 121. The outer area 253 indicates what degree of probability may be achieved theoretically at most. At least one image 121 is recorded at a specific angle. If the images detected so far at an angle adjacent to the specific angle of the processing tool 1 achieve a higher degree of probability than at least one image 121, then at least one image 121 is used to train the computational model 21. In particular, at least one image 121 can be received in a data pool (reference image pool) for the next classifier cycle. For example, a difference threshold can be set, which indicates how much higher the degree of probability the images detected so far must have compared to at least one image 121 in order to use at least one image 121 for training the computational model 21. In one embodiment, the difference threshold can be 0.25. Similarly, the degree of probability of at least one image 121 can be based on the degree of probability compared to the maximum degree of probability that may or theoretically be achieved, so as to determine whether at least one image 121 can be used for training the computational model 21. The difference threshold value then corresponds to the possible or theoretically maximally achievable degree of probability.

[0119] exist Figure 3A In the image recording device 12 shown in FIG. 1 , the generated image 121 shows the working area A when being processed by the processing tool 1. The circular member T can then be identified. Similarly, the image 121 can show any other part of the object B. In particular, the image 121 can show a part of the object B that may be processed by the processing tool 1 or has been processed by the processing tool 1.

[0120] In addition, a sensor 13 for generating sensor data 131 is provided on the processing tool 1. The sensor data 131 shows the superposition of two waveforms with different frequencies. Of course, the sensor 13 can generate the sensor data 131 at any time. In particular, the sensor 13 can generate the sensor data 131 before the processing area A is processed by the processing tool 1 or after the processing area A is processed by the processing tool 1.

[0121] The device further comprises an evaluation unit 2. The evaluation unit 2 receives the image 121 and the sensor data 131 from the image recording device 12 and the sensor 13, respectively, for evaluation. For this purpose, the evaluation unit 2 is connected to the image recording device 12 and the sensor 13. Of course, the evaluation unit 2 can receive the image and the sensor data 131 via radio or via other transmission paths. In a variant, the evaluation unit 2 is arranged on the processing tool 1.

[0122] The evaluation unit 2 has a calculation model 21, which is designed to generate a classifier 221, 231 from the image 121 and / or the sensor data 131. The classifiers 221, 231 represent at least one essential feature of the image 121 and / or the sensor data 131. For illustration purposes, the evaluation unit 2 generates a slice as a classifier 221 for the image 121. For the sensor data 121, the superposition of the two waveforms, Figure 3A The evaluation unit 2 in selects the waveform with the smaller frequency as the classifier 231. In principle, the generation of the classifier is dependent on the calculation model 21. The example shown is only for illustration.

[0123] The classifiers 221, 231 generated by the evaluation unit 2 can be stored in a classifier memory 24. In order to identify at least one working area A and / or a situation during the processing of at least one working area A (application), the classifiers 221, 231 can be called up from the classifier memory 24 by the evaluation unit 2 and / or the processing tool 1. For this purpose, the classifier memory 24 can be connected to the processing tool 1 wirelessly, for example.

[0124] exist Figure 3B In the embodiment of the present invention, the simulation system 6 is connected to the evaluation unit 2. The simulation system 6 simulates the working area A. The simulation system can be used to generate classifiers 222, 232. Similarly, the simulation system 6 can be used to provide the simulated image 120' or the sensor data 130' to the evaluation unit 2. The simulated image 120' or the sensor data 120' can be used to generate the classifiers 222, 232 by the calculation model 21. The simulation system 6 can be connected to the evaluation unit 2 in a detachable manner. In a variant, the simulation system 6 can be connected to the evaluation unit 2 for training and can be abandoned during use.

[0125] Figure 3B The simulation system 6 in the embodiment further generates a classifier 222, which is used to identify a working area A having a rectangular component T. Another classifier 232 generated by the simulation system 6 can be used to identify sensor data having an angular waveform. The classifiers 222, 232 generated by the simulation system 6 can be stored in the classifier memory 24. Of course, the simulation system 6 can generate classifiers 222, 232 for any working area A and / or situation. In particular, the classifiers 222, 232 can be generated with the help of the simulation system 6 in real time, that is, for example, during the processing of the working area A by the processing tool 1. By means of the simulation system 6, it can be ensured, for example, that variations of the working area A or the object B can be recognized by the evaluation unit 2, which has not been trained based on the image 120 and the sensor data 130 of the working area A.

[0126] exist Figure 4In the embodiment shown in , the device comprises a control unit 4 for controlling the processing tool 1. The control unit 4 is connected to the processing tool 1 by means of a cable 123. Of course, the control unit 4 can be connected to the processing tool 1 wirelessly, integrally or in another way. In particular, the control unit 4 can also be connected to the processing tool 1 via the processing unit 2. Similarly, the control unit 4 can be connected to the processing tool 1 and the evaluation unit 2.

[0127] Figure 5 The device in the embodiment comprises, in addition to the simulation system 6, a monitoring unit 3, which is designed to document and / or record which working areas A have been processed by the processing tool 1. For this purpose, the monitoring unit 3 can store a series of working areas A that have been processed by the processing tool 1. The monitoring unit 3 is connected to the evaluation unit 2. The evaluation unit 2 thus forwards information about the processed working areas A to the monitoring unit 3. Likewise, a control unit 4 can be connected to the monitoring unit 3. The control unit 4 thus can forward information about the processed working areas A to the control unit 3.

[0128] Furthermore, a signaling device 5 is connected to the evaluation unit 2. The signaling device 5 can be arranged, for example, on the processing tool 1. The signaling device 5 can also be arranged on other components of the device or be designed separately as an independent unit. Likewise, the signaling device 5 can be integrated into a component of the device, for example, the evaluation unit 2. With the aid of the signaling device 5, a signal can be output to a user of the processing tool 1. The signal is used, for example, to signal the user that the evaluation unit 2 has determined an error. The error can be, for example, an incorrect working area A, an incorrect processing of the processing area A and / or the processing of the working areas A in an incorrect order.

[0129] The test working area 7 is used to calibrate the device. For example, calibration of the device may be necessary to increase the probability of identifying the working area A. On the test working area 7, it can be checked, for example, whether the image recording device 12 functions as intended. In particular, the image recording device 12 can be calibrated with the aid of the test working area 7.

[0130] The image recording device 17 can generate images by means of at least one camera. For this purpose, the image recording device 12 can be arranged on the side of the processing tool 1 that faces the working area A when the working area A is processed by the processing tool 1, such as in Fig. 6A As shown in . Of course, the image recording device 12 can be arranged at any position of the processing tool 1. In particular, the image recording device 12 can include a plurality of cameras, which are arranged at different parts of the processing tool 1. With the help of a plurality of cameras, an image of the surroundings of the processing tool 1, an image of one or more working areas A and / or images of different stereograms of the working area A and / or an image of the environment of the working area A can be generated by the image recording device 12.

[0131] The image recording device 12 comprises an objective lens 122, the optical axis O of which intersects with a working point P, at which the processing tool 1 processes the working area A. The working point P can thus be arranged in the center of the image generated by the image recording device 12. Of course, it is also conceivable that the working point P is arranged at any position of the image. In order to adjust the part of the object B shown in the image, the image recording device 12 can be adjusted relative to the processing tool 1. It is also conceivable and feasible that the image recording device 12 is configured to zoom toward and away from the working area A. For this purpose, the image recording device 12 can include at least one camera with a zoom lens.

[0132] In principle, the optical axis O can intersect the action axis E along which the processing tool 1 acts on the working area A, such as in Figure 7 As shown in Figure 8 In the exemplary embodiment shown in FIG. 8 , the optical axis O extends parallel to the effective axis E. In the exemplary embodiment shown in FIG.

[0133] An illumination device 14 is arranged on the image recording device 12, which illuminates a part of the processing tool 1 and the working area A. To this end, the illumination device 14 emits light substantially along the optical axis of the image recording device 12 in the direction of the working area A. The light of the illumination device 14 particularly illuminates the working point P. Of course, the illumination device 14 can emit light of any arbitrary wavelength in any arbitrary direction. It is also conceivable and feasible to arrange the illumination device 14 at a position close to the processing tool 1. Figure 6B 1 and in particular at the at least one sensor 13. The illumination device 14 can include a plurality of light sources which are arranged at different locations of the processing tool 1 and can emit light in any arbitrary direction. The illumination device 14 can emit light permanently, emit light when the working area A is processed by the processing tool 1 or emit light synchronously with the recording of at least one image by the image recording device 12 and / or synchronously with the generation of sensor data by the at least one sensor 13.

[0134] The sensor 13 is arranged at the tip of the treatment tool 1 facing away from the user, in particular at the tip of the treatment tool 1. In principle, the sensor 13 can be arranged at any part of the treatment tool 1 and in particular within the treatment tool 1, i.e. integrated. The detection axis D of the sensor 13 can intersect with the working point P on the working area A, such as at Figure 7 The detection axis D can in principle extend in the direction in which the sensor 13 generates sensor data 131 with maximum accuracy. For example, in the case of a sensor 13 designed as a microphone, the detection axis D extends perpendicularly to the opening of the microphone.

[0135] In particular, the detection axis D can also extend parallel to the action axis E, as in Figure 8 In principle, the detection axis D can extend in any direction. Of course, it is also conceivable and feasible that at least one sensor 13 generates sensor data 131 with maximum accuracy in multiple directions.

[0136] Fig. 9 A working area A with a pattern T is shown. Such a component T can be determined by the evaluation unit 2. In particular, the position of the component T can be determined by the evaluation unit 2. In principle, any arbitrary component T can be determined by the evaluation unit 2. In addition, missing, ie, non-existent or faulty components T can be detected by the evaluation unit 2.

[0137] Fig.10 A processing tool 1 is shown with an extension. In this way, a tool part 11 can be imaged on at least one image 121. The evaluation unit 2 can then determine the presence of the tool part 11 based on the image. Likewise, the evaluation unit 2 can identify an erroneous tool part 11 that is not suitable for processing the working area A. For identification by the evaluation unit 2 via the image recording device 12, the tool part 11 can, for example, include a marking. Of course, the tool part 11 can also be identified by at least one sensor 13.

[0138] Fig.11 An image recording device 12 is shown, whose objective lenses 122a, 122b are arranged in a ring around the action axis E. The connecting straight line between the two objective lenses 122a, 122b intersects the action axis E. A plurality of objective lenses can be arranged, for example, in a star shape around the action axis E. A plurality of images that can be recorded by a plurality of objective lenses can provide a plurality of stereograms of the working area A. Other arrangements of the objective lenses, for example distributed on the surface of the processing tool 1 or in the form of clusters, are of course conceivable and feasible.

[0139] Fig.12 An embodiment with an industrial screwdriver as processing tool 1 is shown. Processing tool 1 comprises an extension of tool part 11. Image recording device 12 comprises a camera with objective lens 122 and is screwed to processing tool 1 on the tool part 11 side by means of adapter 15. Cable 123 is provided for connecting image recording device 12 to evaluation unit 12. Of course, a wireless connection between image recording device 12 and evaluation unit 2 is also conceivable and feasible, as in Fig.13 As shown in .

[0140] The image recording device 12 generates at least one image 121 of at least one working area A in the area within the image recording cone 124. The image recording cone 124 intersects the tool part 11. The tool part 11 is therefore also imaged on the at least one image 121. Of course, the image recording cone 124 can intersect other sections of the processing tool or not any section of the processing tool. In particular, the image recording cone 124 can be adjusted by means of an adjustable image recording device.

[0141] exist Fig.14 1 shows an image 121 on which a tool part 11 is imaged. The working point P is arranged in the center of the image 121. The tool part 11 formed by the extension and the drill head extends from the edge of the image 121 to the working point P to the center. Features of the working area A, which are used by the evaluation unit 2, optionally in combination with sensor data, for identifying the working area A, are shown by closed shapes and lines. Object B is formed by the vehicle.

[0142] exist Fig.15A Another embodiment is shown in FIG. The tool part 11 is formed by a drill bit. With the aid of the tool part 11, Fig. 15B In the embodiment, the component T formed by the screw is tightened at the object B formed by the vehicle seat at the working point P. The working area A is formed by one end of the seat rail. The image recording device 12 generates at least one image 121 of the seat rail, the drill bit and the screw for identifying the working area.

[0143] Figures 16A-16C An image 121 of a working area A at a motor vehicle door is respectively shown. Fig.16A The angle of the processing tool 1 in FIG. 1 is defined as 0° in the image plane, which is arranged substantially parallel to the working area A. A plurality of main features 121a, 121b, 121c, 121d, 121f, 121g are visible on the working area A. The main features 121a, 121b, 121c, 121d, 121f, 121g are formed by holes, screws and covers. Fig. 16B In relative to Fig.16A The processing tool 1 in FIG. 1 is rotated 3° in the image plane. By the rotation, a larger section of the main feature 121g represented by the rectangle is visible. Fig. 16C Processing tool 1 in relative to Fig. 16C A further rotation of 5° to 8° is performed in the image plane. By the rotation, the main features 121a, 121b, 121c, 121d, 121f, 121g are moved further into the image 121. Another main feature 121e, which was previously obscured by the tool part 11, is visible. Fig.16A- The images shown in C form a series of images that follow one another. In principle, the images have a series of images that are independent of one another. With the aid of the evaluation unit 2, the images 121 can be linked via the contained main features 121a, 121b, 121c, 121d, 121e, 121f, 121g in order to establish a connection (feature recognition) between the images of the series of images. The linking of the images can increase the degree of probability of recognizing the processed working area.

[0144] Fig.17A A working area A of a motor vehicle is shown, at which a component T is missing. The missing component can be detected by means of the device. For this purpose, the evaluation unit 2 determines the missing component T. The evaluation unit 2 thus recognizes that the component T is missing on the motor vehicle, ie, the component T is not present. Fig. 17B A working area A of a motor vehicle is shown, on which a screw T is mounted.

[0145] List of reference numerals:

[0146] 1 Processing tools

[0147] 11 Working parts

[0148] 12 Image recording equipment

[0149] 120 images (training)

[0150] 120' Image (Analog)

[0151] 121 images

[0152] 121a, 121b, 121c, 121d, 121e, 121f, 121g Main features 121' Unknown Image

[0153] 122, 122a, 122b Objective lens

[0154] 123 Cable

[0155] 124 Image Recording Cone

[0156] 13 Sensors

[0157] 130 Sensor data (training)

[0158] 130' Sensor data (analog)

[0159] 131 Sensor data

[0160] 131' Unknown sensor data

[0161] 14 Lighting Equipment

[0162] 15 Adapter

[0163] 2 Evaluation Units

[0164] 21 Computational Model

[0165] 221, 222, 231, 232 Classifiers

[0166] 24 Classifier memory

[0167] 3 Monitoring unit

[0168] 4 Control unit

[0169] 5 Signaling equipment

[0170] 6. Simulation system

[0171] 7 Test work area

[0172] 8 Indicator Devices

[0173] 9 Interface

[0174] A. Work Area

[0175] B Object

[0176] D Detection axis

[0177] E Action axis

[0178] O Optical axis

[0179] P working point

[0180] S1 First Step

[0181] S2 Step 2

[0182] S3 Step 3

[0183] T Component

Claims

1. A device for processing at least one working area (A) of an object (B), the device comprising a processing tool (1) and an evaluation unit (2), wherein the processing tool (1) is provided with: an image recording device (12) for generating at least one image (120, 121) of the at least one working area (A); and at least one sensor (13) for generating sensor data (130, 131) during the processing of the at least one working area (A), It is characterized in that The evaluation unit (2) has a computing model (21) which is trained by means of machine learning based on images (120, 120', 121) and sensor data (130, 130', 131) of the working area (A), wherein the computing model (21) and the at least one image (120, 121) and / or the sensor data (130, 131) are used to identify the relative position of the processing tool (1) relative to the working point or relative to the at least one working area (A) and / or to identify the situation during the processing of the at least one working area (A), wherein at least one optical axis (O) of at least one objective (122, 122a, 122b) of the image recording device (12) and / or at least one detection axis (D) of at least one sensor (13) intersects with a working point (P), at which the processing tool (1) acts on the at least one working area (A), and / or at least one optical axis (O) of at least one objective lens (122, 122a, 122b) of the image recording device (12) and / or at least one detection axis (D) of the at least one sensor (13) extends parallel to an action axis (E), along which the processing tool (1) acts on the at least one working area (A), and wherein the image recording device (12) and a tool component (11) interacting with the working area (A) are arranged at the processing tool (1) for movement.

2. The device according to claim 1, It is characterized in that The calculation model (21) is designed to generate at least one classifier (221, 231) from at least one image (120, 120', 121) and / or the sensor data (130, 130', 131), the at least one classifier (221, 231) representing at least one essential feature of the at least one image (120, 120', 121) and / or the sensor data (130, 130', 131).

3. The device according to claim 1, It is characterized in that The evaluation unit (2) is designed to select critical areas of the at least one image (120, 120', 121) based on a user input for training the calculation model (21) and / or for identifying the at least one working area (A).

4. The device according to claim 1, It is characterized in that The calculation model (21) is designed to recognize an installation step in an installation process based on a series of images (121) and / or sensor data (131).

5. The device according to claim 2, It is characterized in that A simulation system (6) is provided, by means of which the at least one working area (A) can be simulated in order to generate the at least one image (120'), the sensor data (130') and / or the at least one classifier (222, 232).

6. The device according to claim 5, It is characterized in that The at least one working area (A) can be simulated by means of the simulation system (6) while the processing tool (1) acts on the at least one working area (A).

7. The device according to claim 1, It is characterized in that The evaluation unit (2) is designed to identify, in a series of a plurality of known images (121) and / or known sensor data (131) containing at least one segment of at least one unknown image (121') and / or unknown sensor data (131'), at least one unknown segment of the at least one unknown image (121') and / or sensor data (131').

8. The device according to claim 7, It is characterized in that The computing model (21) is designed to identify the at least one known working area (A) based on the at least one section of the at least one unknown image (121') and / or unknown sensor data (131') by means of machine learning training.

9. The device according to claim 1, It is characterized in that The evaluation unit (2) is designed to identify a region of the object (B) that is not to be processed and / or to distinguish the region that is not to be processed from the at least one working region (A).

10. The device according to claim 1, It is characterized in that The machine learning includes neural networks, convolutional neural networks, bag of words and / or uncorrelated decision trees.

11. The device according to claim 1, It is characterized in that A monitoring unit (3) is provided, which is designed to document a series of working areas (A) that have been processed by the processing tool (1).

12. The device according to claim 11, It is characterized in that The monitoring unit (3) is designed to predetermine a series of working areas (A) to be processed by the processing tool (1).

13. The device according to claim 11, It is characterized in that The monitoring unit (3) is designed to group at least two working areas (A) from a series of working areas (A) for processing by the processing tool (1).

14. The device according to claim 11, It is characterized in that The monitoring unit (3) is connected to the processing tool (1), the image recording device (12), the at least one sensor (13) and / or the evaluation unit (2) for exchanging data.

15. The device according to claim 1, It is characterized in that A control unit (4) is provided, which is connected to the processing tool (1) and / or the evaluation unit (2) and by means of which the processing tool (1) can be controlled.

16. The device according to claim 11, It is characterized in that A control unit (4) is provided, which is connected to the processing tool (1) and / or the evaluation unit (2) and by means of which the processing tool (1) can be controlled, wherein the control unit (4) is connected to the monitoring unit (3).

17. The device according to claim 1, It is characterized in that A signaling device (5) is provided for outputting a signal to a user of the processing tool (1), by means of which the user can be informed that at least one working area (A) has not been identified, at least one working area (A) has been processed incorrectly and / or at least one working area (A) in a series of working areas (A) has not been processed.

18. The device according to claim 1, It is characterized in that A test work area (7) is provided for calibrating the device.

19. The device according to claim 1, It is characterized in that The image recording device (12) comprises at least one lighting device (14), which illuminates the at least one working area (A) or at least a part thereof and / or the processing tool (1) or at least a part thereof and / or projects at least one pattern onto the at least one working area (A).

20. The device according to claim 19, It is characterized in that The at least one lighting device (14) is designed to emit light synchronously with the recording of at least one image (120, 121) by the image recording device (12).

21. The device according to claim 1, It is characterized in that The image recording device (12) comprises at least one camera.

22. The apparatus according to claim 1, It is characterized in that The image recording device (12) comprises at least one camera which is adjustable relative to the processing tool (1).

23. The apparatus according to claim 1, It is characterized in that The image recording device (12) comprises at least two cameras for simultaneously generating at least two images of the at least one working area (A).

24. The apparatus according to claim 1, It is characterized in that The image recording device (12) is designed to generate a plurality of images (120, 121) before, during and / or after the processing tool (1) acts on the at least one working area (A).

25. The apparatus according to claim 1, It is characterized in that The number of images (120, 121) generated per unit time by the image recording device (12) is variable.

26. The apparatus according to claim 1, It is characterized in that The image recording device (12) is designed to automatically start and / or end the generation of an image (120, 121) based on sensor data (130, 131).

27. The apparatus according to claim 1, It is characterized in that The image recording device (12) comprises a mobile terminal device.

28. The device according to claim 27, It is characterized in that The at least one sensor (13) or a part of the sensor (13) is integrated into the mobile terminal.

29. The apparatus according to claim 1, It is characterized in that A connecting straight line between the two objective lenses (122a, 122b) of the image recording device (12) and / or between the two sensors (13) intersects an action axis (E), along which the processing tool (1) acts on the at least one working area (A).

30. The apparatus according to claim 1, It is characterized in that The plurality of objective lenses (122a, 122b) and / or the plurality of sensors (13) of the image recording device (12) are arranged in a ring around an action axis (E), along which the processing tool (1) acts on the at least one working area (A).

31. The apparatus according to claim 1, It is characterized in that The evaluation unit (2) is designed to recognize at least one tool part (11) of the processing tool (1) based on the at least one image (121).

32. The apparatus according to claim 31, It is characterized in that An indication device (8) is provided, which is designed to display the necessary replacement of at least one tool part (11) of the processing tool (1) to a user of the processing tool (1) and / or to monitor the replacement of the at least one tool part (11) by means of an evaluation of at least one image (121) and / or sensor data (131).

33. The apparatus according to claim 31, It is characterized in that The evaluation unit (2) is designed to determine a degree of wear of the at least one tool part (11).

34. The apparatus according to claim 1, It is characterized in that The evaluation unit (2) is designed to detect at least one component (T) of an object (B) arranged in the at least one working area (A).

35. The apparatus according to claim 34, It is characterized in that The evaluation unit (2) is designed to determine, based on the at least one image (121) and / or the sensor data (131), whether a component (T) is defective.

36. The apparatus according to claim 1, It is characterized in that The image recording device (12) is arranged on the processing tool (1) by means of an adapter (15).

37. The apparatus according to claim 1, It is characterized in that The image recording device (12) is connected to the evaluation unit (2) for exchanging data.

38. The apparatus according to claim 1, It is characterized in that At least one interface (9) is provided, which is configured to be connected to another system.

39. A method for processing at least one working area (A) of an object (B) by means of a processing tool (1) and an evaluation unit (2), wherein the processing tool (1) is provided with: an image recording device (12) for generating at least one image (120, 121) of the at least one working area (A); and at least one sensor (13) for generating sensor data (130, 131) during the processing of the at least one working area (A), It is characterized in that After the computational model (21) of the evaluation unit (2) has been trained by means of machine learning based on images (120, 120', 121) of the working area (A) and sensor data (130, 130', 131), the relative position of the processing tool (1) relative to the working point or relative to the at least one working area (A) can be identified using the computational model (21) and with the aid of the at least one image (120, 121) and / or the sensor data (130, 131), and / or the situation during the processing of the at least one working area (A) can be identified by the evaluation unit (2) with the aid of the computational model (21), wherein at least one optical axis of at least one objective lens (122, 122a, 122b) of the image recording device (12) The line (O) and / or at least one detection axis (D) of at least one sensor (13) intersects a working point (P), in which the processing tool (1) acts on the at least one working area (A), and / or at least one optical axis (O) of at least one objective lens (122, 122a, 122b) of the image recording device (12) and / or at least one detection axis (D) of the at least one sensor (13) extend parallel to an action axis (E), along which the processing tool (1) acts on the at least one working area (A), and wherein the image recording device (12) and a tool component (11) interacting with the working area (A) are arranged at the processing tool (1) for movement.

40. The method according to claim 39, It is characterized in that The calculation model (21) generates at least one classifier (221, 331) from at least one image (120, 120', 121) and / or the sensor data (130, 130', 131), wherein the at least one classifier (221, 331) represents at least one main feature of the at least one image (120) and / or the sensor data (130).

41. The method according to claim 40, It is characterized in that The computational model (21) identifies installation steps in an installation process based on a series of images (121) and / or sensor data (131).

42. The method according to claim 41, It is characterized in that The evaluation unit (2) identifies at least one section of at least one unknown image (121') and / or unknown sensor data (131') in a series of a plurality of known images (121) and / or known sensor data (131) which contain at least one section of at least one unknown image (121') and / or unknown sensor data (131').

43. The method according to claim 42, It is characterized in that The computing model (21) is trained by means of machine learning to identify the at least one known working area (A) based on at least one section of the at least one unknown image (121') and / or unknown sensor data (131').

Images