Method for masking an unwanted object from the field of view of a camera, system for executing the method, laser processing machine and computer program product

Abstract

The invention relates to a method for masking an unwanted object in the field of view of a camera, particularly in laser processing machines, comprising: capturing an image by a first camera (1) with a field of view, wherein the field of view is partially obscured by an unwanted object (2); capturing an image by at least a second camera (3) with a field of view that at least partially overlaps with the field of view of the first camera (1); determining the position and extent of the unwanted object (2) in the field of view of the first camera (1), wherein the obscured image areas in the image of the first camera (1) are replaced by perspective-corrected image information from the second camera (3), the perspective correction being based on the known positions and orientations of the cameras (1, 3).

Description

[0001] The present invention relates to a method for masking an unwanted object in the field of view of a camera, comprising: capturing an image with a first camera having a field of view, wherein the field of view is partially obscured by an unwanted object; capturing an image with at least one second camera having a field of view that at least partially overlaps with the field of view of the first camera; and determining the position and extent of the unwanted object in the field of view of the first camera. The invention further relates to a system for carrying out the method, a laser processing machine, and a computer program.

[0002] In industrial applications, particularly when monitoring machining processes such as laser cutting, unwanted objects can significantly impair the camera's field of view. A typical example is the cutting head of a laser cutting machine, which, depending on its position, can obscure substantial portions of the work area. To ensure an uninterrupted and complete view of the material being processed, such as a sheet of metal, it is necessary to selectively block out such interfering objects. This significantly simplifies and optimizes the monitoring and control of the machining process.

[0003] While the state of the art in methods for masking unwanted objects from the field of view of cameras, particularly in the monitoring of laser processing machines, shows some promising approaches, significant technical and practical limitations remain. These methods are often based on combining image data from multiple cameras or on perspective correction of wide-angle images. Examples can be found in CN112896047A and DE102022107250A1, which describe the use of multiple cameras with overlapping fields of view to compensate for obscured image areas. However, these methods are limited in their efficiency and precision.

[0004] A common problem is the lack of accuracy in perspective correction. Existing methods mostly use generic correction algorithms that cannot fully compensate for distortions from wide-angle lenses or transitions between camera fields of view. This leads to unsatisfactory results, especially in applications with high precision requirements, such as those prevalent in laser processing. The quality of the reconstructed image data is often insufficient to ensure reliable and precise monitoring of the work area.

[0005] Another significant disadvantage of current solutions is their limited real-time processing capability. The methods described in DE102022107250A1, for example, require considerable computing resources for combining and processing the image data streams. In dynamic environments with moving objects such as cutting heads or workpieces, this can lead to delays that compromise the efficiency and safety of the entire machining process. This is a serious limitation, especially in industrial applications where rapid response times are crucial.

[0006] Furthermore, it is evident that existing methods often exhibit only limited levels of automation in object recognition. Patents such as WO2022117914A1 and CN112896047A utilize simple image processing algorithms that prove inadequate in dynamic scenarios. Changes in lighting, rapid movements, or complex geometries of the unwanted objects often lead to false detections or insufficient adjustments. Consequently, the robustness of these methods to changing operating conditions remains limited.

[0007] Economic aspects such as costs and implementation complexity also pose problems. Some methods rely on cloud computing solutions to meet the high computing demands, as described, for example, in CN112896047A. However, this leads to additional infrastructure costs and a dependence on stable network connections, which are not always guaranteed in industrial environments. The implementation of such methods thus becomes expensive and sometimes unreliable.

[0008] In summary, the current state of the art does not offer a satisfactory solution for the specific requirements of laser processing machines. Existing methods exhibit weaknesses in terms of precision, real-time capability, automation, and cost structure. In particular, a robust and efficient method for seamlessly eliminating unwanted objects, one that operates reliably under the changing conditions of industrial scenarios, is lacking.

[0009] Against this background, the purpose of the invention is to overcome these shortcomings and to offer a precise, robust and economical solution for monitoring laser processing processes.

[0010] This task is solved by a method for masking an unwanted object in the field of view of a camera, particularly in laser processing machines, comprising: • Capturing an image with a first camera with a field of view, where the field of view is partially obscured by an unwanted object, • Capturing an image by at least one second camera with a field of view that overlaps at least partially with the field of view of the first camera, • Determining the position and extent of the unwanted object in the field of view of the first camera, whereby the obscured image areas in the image of the first camera are replaced by perspectively corrected image information from the second camera, the perspective correction being based on the known positions and orientations of the cameras.

[0011] This method offers the advantage that the field of view of a camera, which is partially obscured by an unwanted object, can be fully reconstructed by incorporating a second camera. Using perspective-corrected image information from the second camera enables a precise and detailed representation of the obscured areas. This leads to a significant improvement in the visual monitoring of work areas, especially in complex or dynamic processes, without requiring any physical modifications to the camera installation.

[0012] By using multiple cameras, it is possible to supplement information obscured by an unwanted object, such as a laser cutting head in a laser processing machine, with data from the fields of view of other cameras. The obscured area in the image of one camera is replaced with corresponding image data from another camera whose field of view covers that area. This enables a complete reconstruction of the obscured image information and ensures a comprehensive representation of the monitored area.

[0013] In a laser processing machine, such as a laser cutting machine, the field of view of one or more cameras is used to monitor the processing process. Since the positions of the cameras within the machine are known, the overlaps of their fields of view can be precisely calculated. If an object, such as the laser cutting head, located within the field of view of one camera is identified as unwanted, the information hidden behind this object can be displayed using image data from another camera whose field of view covers that area. This makes it possible to filter out a large number of unwanted objects within the cameras' fields of view while still ensuring continuous and complete visual monitoring of the processing process. This method significantly improves process control and guarantees precise monitoring even during dynamic changes in the work area.

[0014] To ensure complete and seamless monitoring, the obscured pixels in the image are replaced with image information from another camera. This image information is adapted using a predefined transformation so that it can be inserted precisely into the obscured areas, thus enabling a consistent and distortion-free representation of the work area.

[0015] Applying the described method in a laser processing machine, particularly for monitoring a laser cutting head, enables a significant improvement in image information. By detecting and masking unwanted objects, such as the laser cutting head itself, obscured image areas can be replaced with precisely corrected and reconstructed image data. This results in a more comprehensive and detailed representation of the laser processing machine's work area. Especially during remote operations, where the machine is controlled and monitored from a distance, the operator has access to more visual information thanks to the complete image reconstruction. This facilitates decision-making, increases process reliability, and improves the efficiency of monitoring the processing process.

[0016] According to an advantageous embodiment of the invention, the unwanted object can be automatically detected using image processing algorithms by analyzing contrasts, edges, or movements in the image. Advantageously, the automatic detection of the unwanted object using image processing algorithms enables efficient and reliable recognition without manual intervention. The analysis of contrasts, edges, or movements ensures that detection occurs regardless of the object's position or movement. This increases the system's robustness to changes in the environment and its flexibility in application.

[0017] According to a further preferred embodiment of the invention, perspective correction can also be performed using a transformation matrix calculated from the geometric parameters of the cameras. Using a transformation matrix for perspective correction offers the advantage that the calculations can be performed precisely and reproducibly. The transformation matrix, derived from the geometric parameters of the cameras, ensures an exact superimposition of the image areas and avoids distortions. This results in high accuracy when replacing obscured image sections and reduces the computational effort through standardized mathematical procedures.

[0018] Furthermore, according to another advantageous embodiment of the invention, the obscured image areas can be replaced by a pixel-by-pixel mapping between the camera images. This pixel-by-pixel mapping enables a highly detailed and visually seamless replacement of the obscured image areas. This offers the advantage that even small image details are preserved, which is particularly important in applications with high image quality requirements, such as in laser technology.

[0019] According to a further particularly preferred embodiment of the invention, the detection of the unwanted object can be carried out by comparing successive images from the first camera in order to identify moving objects. Detecting the unwanted object by comparing successive images enables the identification of moving objects with high reliability. This offers the advantage that even dynamic disturbances, such as moving cutting heads or robot arms, can be effectively detected and taken into account. This significantly expands the functionality of the system in environments with continuous movement.

[0020] Furthermore, the object of the invention can also be achieved by a system for carrying out the method according to any one of claims 1 to 5, comprising: a first camera and at least one second camera; a control unit configured to perform the detection of the unwanted object, the perspective correction and the replacement of the obscured image areas; wherein the control unit uses stored transformation data from the cameras to calculate the perspective correction.

[0021] The system, consisting of multiple cameras and a control unit, offers the advantage of forming an integrated and highly functional monitoring system. The stored transformation data enables fast and precise calculation of perspective correction. This results in high system efficiency and allows for seamless integration into existing machine designs without requiring additional hardware modifications.

[0022] In a preferred embodiment of the invention, the cameras can also be mounted at fixed positions within a machine tool. The fixed mounting of the cameras at defined positions within a machine tool, such as a laser processing machine, ensures consistent and reliable acquisition of the fields of view. This offers the advantage that the calculations for perspective correction remain constant and do not require repeated adjustments.

[0023] It can also be advantageous to further develop the invention such that the control unit performs real-time processing of the image information in order to immediately replace the obscured image areas, thus enabling continuous and uninterrupted monitoring of the process. This is particularly advantageous in applications where fast response times are required, such as the real-time monitoring of laser processing.

[0024] The problem of the invention can also be solved by a laser processing machine comprising: a camera system, in particular according to one of claims 6-8 comprising a first camera and at least one second camera, for monitoring the laser processing process, wherein the camera system is configured to detect unwanted objects in the field of view of one camera, to replace obscured image areas with perspective-corrected image information from the other camera and thus to enable complete monitoring of the processing process.

[0025] Combining a laser processing machine with a camera system for monitoring the laser processing offers the advantage that unwanted objects in the field of view are detected and the obscured areas are reliably replaced by the perspective-corrected image information from another camera. This ensures complete monitoring of the processing and enables precise control and monitoring of the machine, which is particularly crucial for demanding or safety-critical processing operations.

[0026] Finally, the problem of the invention can also be solved by a computer program product configured to execute the method according to any one of claims 1 to 5, wherein the computer program is installed on a control unit and controls the detection of the unwanted object, the perspective correction, and the replacement of the obscured image areas. The computer program product offers the advantage of implementing the method for detecting and replacing obscured image areas in software, thus providing a flexible and scalable solution for various camera systems. Installing the program on a control unit enables centralized control and easy integration into existing machine or monitoring systems, thereby reducing the cost of additional hardware and expanding the range of applications.

[0027] The invention will now be explained in more detail with reference to a figure, without limiting the general inventive concept.

[0028] It shows: Fig. 1 System for executing a method for hiding an unwanted object from the field of view of a camera in a schematic representation.

[0029] Based on the Fig. In this document, a method and system for masking an unwanted object 2 in the field of view of a camera, which is particularly useful for monitoring laser processing machines, is described in more detail. The method comprises capturing an image with a first camera 1 whose field of view is partially obscured by an unwanted object 2, and capturing an image with at least one second camera 3 whose field of view at least partially overlaps with that of the first camera 1. The position and extent of the unwanted object 2 are determined in the field of view of the first camera 1, whereby obscured image areas are replaced by perspective-corrected image information from the second camera 3. The perspective correction is based on the known positions and orientations of cameras 1 and 3.

[0030] The unwanted object 2 is automatically detected using image processing algorithms that analyze contrasts, edges, or movements in the image. This enables precise and efficient detection even with dynamic movements or varying lighting conditions. Perspective correction is performed using a transformation matrix calculated from the geometric parameters of cameras 1 and 3. This ensures a seamless and detailed transfer of the obscured image areas.

[0031] The obscured image areas are then replaced by a pixel-by-pixel mapping between the camera images, ensuring high accuracy and detail in image reconstruction. This is particularly crucial in precision applications, such as monitoring laser processing.

[0032] The detection of the unwanted object 2 can also be achieved by comparing successive images from the first camera 1. This enables the identification of moving objects, such as a moving cutting head, and ensures robust and dynamic adaptation to the conditions of the laser processing machine.

[0033] The procedure is installed in the form of a computer program on a control unit 4, which controls the detection of the unwanted object 2, the perspective correction and the replacement of the hidden image areas.

[0034] The system shown for carrying out the procedure comprises, according to the description in the Fig.The system consists of a first camera 1, at least one second camera 3, and a control unit 4. The control unit 4 is configured to perform the detection of the unwanted object 2, perspective correction, and replacement of the obscured image areas. The control unit 4 uses stored transformation data from cameras 1 and 3 to calculate the perspective correction precisely and efficiently.

[0035] Cameras 1 and 3 are mounted at fixed positions in a machine tool, specifically a laser processing machine, enabling stable and reproducible image processing. The figure also shows that the control unit 4 performs real-time processing of the image information, allowing obscured image areas to be replaced immediately. This significantly contributes to the efficiency and reliability of the monitoring system.

[0036] The described camera system is preferably integrated into a laser processing machine. It comprises a first camera 1 and at least one second camera 3 and serves for the complete monitoring of the laser processing process. The system is configured to detect unwanted objects 2 in the field of view of one camera, replace obscured image areas with perspective-corrected image information from the other camera, and thus enable continuous and precise monitoring of the processing process. This technical solution ensures improved process control and increases the safety and efficiency of the processing tasks.

[0037] The invention is not limited to the embodiments illustrated in the figures. The foregoing description is therefore not to be considered limiting, but rather explanatory. The following claims are to be understood as meaning that a named feature is present in at least one embodiment of the invention. This does not preclude the presence of further features. Insofar as the claims and the foregoing description define 'first' and 'second' features, this designation serves to distinguish between two similar features without establishing any hierarchy. Reference symbol list 1 camera 2 objects 3 cameras 4 Control unit QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] CN 112896047A [0003, 0006, 0007] DE 102022107250A1 [0003, 0005] WO 2022117914A1

[0006]

Claims

A method for masking an unwanted object in the field of view of a camera, particularly in laser processing machines, comprising: • capturing an image by a first camera (1) with a field of view, wherein the field of view is partially obscured by an unwanted object (2), • capturing an image by at least a second camera (3) with a field of view that overlaps at least partially with the field of view of the first camera (1), • determining the position and extent of the unwanted object (2) in the field of view of the first camera (1), characterized in that the obscured image areas in the image of the first camera (1) are replaced by perspectively corrected image information from the second camera (3), wherein the perspective correction is based on the known positions and orientations of the cameras (1, 3). Method according to claim 1, characterized in that the unwanted object (2) is automatically detected using image processing algorithms by analyzing contrasts, edges or movements in the image. Method according to one of the preceding claims characterized in that the perspective correction is carried out by a transformation matrix calculated from the geometric parameters of the cameras (1, 3). Method according to one of the preceding claims, characterized in that the hidden image areas are replaced by a pixel-by-pixel mapping between the camera images. Method according to one of the preceding claims, characterized in that the detection of the unwanted object (2) is carried out by comparing successive images of the first camera (1) to identify moving objects. System for carrying out the method according to one of claims 1 to 5, comprising: • a first camera (1) and at least one second camera (3), • a control unit (4) configured to perform the detection of the unwanted object (2), the perspective correction and the replacement of the obscured image areas, • characterized in that the control unit (4) uses stored transformation data of the cameras (1, 3) to calculate the perspective correction. System according to claim 6, characterized in that the cameras (1, 3) are mounted at fixed positions in a machine tool. System according to claim 6 or 7, characterized in that the control unit (4) performs real-time processing of the image information in order to immediately replace the obscured image areas. Laser processing machine, comprising: • a camera system, in particular according to one of claims 6 - 8 comprising a first camera (1) and at least one second camera (3), for monitoring the laser processing process, • wherein the camera system is configured to detect unwanted objects (2) in the field of view of one camera, to replace obscured image areas with perspective-corrected image information from the other camera and thus to enable complete monitoring of the processing process. Computer program product configured to execute the method according to any one of claims 1 to 5, wherein the computer program is installed on a control unit (4) and controls the detection of the unwanted object (2), the perspective correction and the replacement of the hidden image areas.

Images