Camera system and method for setting a camera system for detecting an object

Abstract

The invention relates to a method (200) for setting a camera system (10) for detecting an object (12). The method (200) comprises (a) receiving (202) an image, captured by means of a camera (14) of the camera system (10), of an object (12) located, as a reference object, at least partially in an image field (16) of the camera (14), and (b) initiating the execution (204) of object detection based on the captured image. The method (200) further comprises (c) initiating a determination (206) of a detection reliability for the executed object detection, and (d) assessing (208) the suitability of an orientation and / or focussing of the camera (14) for the executed object detection based on the determined detection reliability.

Description

[0001] CAMERA SYSTEM AND METHOD FOR SETTING A CAMERA SYSTEM FOR OBJECT RECOGNITION

[0002] Provided are a computer-implemented method for adjusting a camera system for object detection, a use of the detection reliability of an object detection performed by a camera system as a controlled variable for controlling the orientation and / or focusing of the camera system, a control unit for a camera system for object detection, a computer program, a computer-readable medium, a camera system, and a robot. The embodiments thus lie in particular in the field of object detection and optionally in the field of automation technology.

[0003] Camera systems that can be used for the optical capture of objects are known in the art. This requires the camera to be adapted to the objects to be captured. The sequential adaptation of the camera usually takes place in a sequential series of independent steps, which initially comprise an adjustment of the camera's image field, which can be achieved by positioning the camera relative to the object to be captured or by adjusting the focal length of the camera lens. The latter is only possible if the camera has a lens with an adjustable focal length or allows the lens to be replaced. In a further step, the focus position of the imaging optics, i.e. the camera lens, is usually adjusted in order to focus the image on one or more objects to be captured.When using the camera to detect the optically captured object, after completing the adjustment and setting of the camera, a further step can be taken, including capturing one or more images and analyzing the image(s). The detection of the object(s) can be performed using artificial intelligence. US8181878B2 describes a method and apparatus for indicating a focus or blur state for a two-dimensional imaging device in an optical reading and decoding device for receiving an encoded image of a target area on an object or packaging.

[0004] JP2006173979A presents a digital camera with a rotatable camera lens, with an LED lamp installed behind the camera lens to confirm to the operator that the camera lens is in focus in spatial relation to an object.

[0005] W02009 / 089142A2 describes a technique for focusing a digital camera, comprising the following steps: 1. Pre-storing at least one specific image object, such as a known face or landmark. 2. Activating the camera to acquire an image. 3. Analyzing objects in the image. 4. Comparing the objects in the image with the at least one specific image object. 5. Determining whether there is a match between at least one object in the image and at least one specific image object. If a match is found, the camera focuses on the matching object, and the image is captured.

[0006] EP2693363B1 discloses a camera system (or a camera-based code reading system) for detecting a stream of objects moving relative to the camera system, wherein the camera system comprises a plurality of detection units, each having an image sensor and a focusing unit, as well as at least one control unit.

[0007] CN 109 559 353 A describes a calibration method for a first camera module, a second camera module and a depth camera module.

[0008] US 2015 / 022713 describes an imaging device and an imaging method in which an imaging device adjusts a focus based on the result of facial recognition. The methods and devices known in the prior art have the common disadvantage that an inappropriate setting or adjustment of the camera to the object to be recognized can lead to a complete failure of object recognition. However, since the inappropriate setting or adjustment of the camera may only be discovered upon incorrect object recognition, this results in the need to repeat an installation or setup procedure for adjusting or adjusting the camera. This usually requires time-consuming repetition of manual processes by the operating personnel. This disadvantage can be particularly pronounced when many different objects or error states must be recognized.This can also be made more difficult if fast-moving objects need to be detected.

[0009] Against the background of this prior art, an object of the present disclosure can be seen in specifying a device and / or a method which are each suitable for enriching the prior art.

[0010] The problem is solved by the features of the independent claims. The subordinate claims and the dependent claims each contain optional developments of the disclosure.

[0011] A method is provided for adjusting a camera system for object detection. The method comprises (a) receiving an image, captured by a camera of the camera system, of an object located as a reference object at least partially within a field of view of the camera, and (b) initiating object detection based on the captured image. The method further comprises (c) determining a detection reliability during the performed object detection, and (d) assessing the suitability of an alignment and / or focusing of the camera for the performed object detection based on the determined detection reliability.

[0012] Furthermore, a use of a detection reliability of an object detection performed by means of a camera system as a control variable for controlling the orientation and / or focusing of the camera system is provided.

[0013] In addition, a control unit for a camera system for detecting an object is provided, wherein the control unit is configured to carry out a method according to the disclosure.

[0014] In addition, a computer program is provided which comprises instructions which, when executed by a computer, cause the computer to carry out a method according to the disclosure.

[0015] Furthermore, a computer-readable medium is provided which comprises instructions which, when executed by a computer, cause the computer to perform a method according to the disclosure.

[0016] Furthermore, a camera system for object detection is provided. The camera system comprises a camera with variable orientation and focus, wherein the camera can be aligned with an object to be detected such that the object is at least partially located within a field of view of the camera, as well as a control unit. The control unit is connected to the camera system and configured to trigger the capture of an image of the object located in the field of view by the camera and to trigger the implementation of object detection based on the captured image. The control unit can also be a component of the camera system or the camera.The control unit is further configured to initiate a determination of a detection reliability during the object detection performed, and to provide information about the determined detection reliability, which enables an assessment of the suitability of the orientation and / or focusing of the camera for the object detection performed based on the determined detection reliability.

[0017] Furthermore, a robot comprising a camera system according to the disclosure is provided.

[0018] The method can be embodied as a computer-implemented method. The fact that the method can be a computer-implemented method means that at least some method steps are performed by a computer. Optionally, all method steps can be performed by a computer. Optionally, the computer can be embodied as one of the devices listed below as examples: a personal computer, a notebook computer, a tablet computer, a smartphone, a control unit, a microcontroller, and a processor. The control unit can optionally have one or more processors and one or more memory elements.

[0019] The camera system can comprise one or more cameras. Optionally, the camera system can comprise additional components, such as a control unit and / or a communications module. The optional multiple components of the control unit can together form a structural unit and can optionally be arranged in a common housing. Alternatively, at least some components of the camera system can be arranged separately from one another.

[0020] The detection of an object can also be referred to as object recognition. The detection of the object can serve to determine whether one or more optically detected objects correspond to a predetermined object type or not. Optionally, the object recognition can also include the extraction of further information, for example, whether there are deviations between a detected object and a predetermined object type or not. Optionally, the extraction of further information can also include the reading of information stored in or on the object, such as the

[0021] Reading a barcode and / or a QR code and / or conventional, human-readable writing and / or other types of symbols.

[0022] The field of view is an area in an object plane imaged by the camera, which is spanned by the angle of view. In other words, the field of view represents the area in the object plane that can be imaged by the camera with the selected settings. The orientation of the camera can be determined by the direction in which the optical axis of the camera or the optical axis of an imaging element of the camera, such as an objective lens of the camera, is pointed. Optionally, the orientation of the camera can also be determined by a focal length of the imaging element of the camera. Optionally, the orientation of the camera can also be determined by a distance between the camera and the object to be optically recorded.Thus, changing the orientation of the camera can be done by changing the orientation of the optical axis of the camera and / or by changing the position of the camera along the optical axis, i.e. by changing the distance between the camera and the object to be optically captured. Changing the orientation of the camera can optionally also be done by changing the focal length of the imaging element of the camera, for example a zoom lens. Focusing the camera can involve focusing the camera, or the imaging element of the camera, on the object so that a sharp image of the object is created on a sensor surface of the camera. In other words, focusing can represent a precise adjustment of the distance of the imaging element between the object and the sensor surface of the camera. The distance setting can optionally be done using a camera lens.

[0023] Recognition reliability of object recognition can refer to a reliability with which, given the prevailing quality of the received image, object recognition can be successfully carried out using a reference object, i.e., the reliability with which the process can be carried out. This means that the recognition reliability can depend primarily or exclusively on the received image and its suitability for object recognition. Optionally, the relevant quality can depend at least partially and / or exclusively on the alignment and / or focusing of the camera on the reference object. In other words, the recognition reliability within the meaning of the disclosure can primarily or exclusively be a measure of the suitability of the one or more received images captured by the camera of the camera system.On the other hand, the recognition reliability can be selected as an assessment criterion in such a way that a quality and / or suitability of the selected reference object and / or a quality and / or suitability of the training data and / or a training method of an artificial intelligence for training object recognition and / or other reference information for the ability to recognize objects do not influence the recognition reliability or influence it only to a small extent.

[0024] Recognition reliability can be influenced by the degree to which an object detection leads to a correct result. Recognition reliability can optionally be divided into several levels, such as high, medium, and low. Recognition reliability can optionally be expressed as a percentage, with a recognition reliability of 100% indicating completely reliable and optionally error-free object detection, while 0% can indicate a complete lack of object detection reliability.

[0025] Optionally, the recognition reliability can comprise two levels, with a first level of recognition reliability being present when an object corresponding to a reference object was detected during the initiated object recognition based on the captured image, and a second level of recognition reliability being present when an object corresponding to a reference object was not detected during the initiated object recognition based on the captured image. Optionally, the recognition reliability can be expanded to include additional levels in order to be able to appropriately present, for example, ambiguous object recognition results that cannot be assigned to either the first or second level.

[0026] Optionally, the recognition reliability can be a measure of whether or not an object corresponding to a reference object was recognized during an object recognition process based on the captured image. This can optionally be advantageous if initiating the provision of information about the determined recognition reliability includes initiating the provision of information about whether or not an object corresponding to a reference object was recognized during an object recognition process based on the captured image. This can be advantageous in that the method for adjusting the camera system can be carried out in a targeted and efficient manner using reference objects, ensuring that objects that correspond to a reference object are presented to the camera system for adjustment.If the subsequently provided information on the determined recognition reliability indicates that the object recognition system failed to detect an object corresponding to a reference object based on the captured image data, this may lead to the assessment that the camera's orientation and / or focus during image capture is not sufficiently suitable for the object recognition performed. However, if the subsequently provided information on the determined recognition reliability indicates that the object recognition system failed to detect an object corresponding to a reference object based on the captured image data, this may lead to the assessment that the camera's orientation and / or focus during image capture is sufficiently suitable for the object recognition performed.

[0027] Optionally, the detection reliability can be a measure of whether or with what reliability edges were detected during edge detection based on the captured image of an object corresponding to a reference object, and / or how large any deviation in the number and / or characteristics of the detected edges is from a predetermined number and / or characteristics of the edges of the reference object. For example, the object corresponding to a reference object can have a predetermined number and / or characteristics of edges. A value for the current detection reliability can be derived based on the number and / or characteristics of the edges actually detected.

[0028] The fact that the computer-implemented method and / or the control unit are configured to initiate the execution of certain steps can mean that the computer-implemented method and / or the control unit carries out the steps itself and / or transmits them to another device for execution. Initiating certain steps can also mean that a human-perceivable signal is output, based on which a human operator can recognize necessary steps and then (manually) carry them out and / or initiate them. Optionally, the other device can be designed as another computer. Optionally, the other device can be in the form of a server and / or a central control unit with which the control unit can have a communication connection. Optionally, the server and / or the central control unit can have other functions in addition to executing the steps.Optionally, the control unit can be configured to transmit the instructions and / or information to the other computer via a network connection, such as Ethernet and / or USB, and / or via a wireless transmission method. The control unit can be optimized to execute the steps using artificial intelligence. Optionally, the control unit can have a hardware architecture that is particularly suitable for the operation of artificial intelligence and / or is resource-efficient.

[0029] The computer program may be in the form of program code, in particular code suitable for controlling camera systems. The computer program does not necessarily have to be stored on a computer-readable storage medium to be made available to the control unit, but can also be obtained externally via the Internet or otherwise.

[0030] A robot can refer to a machine that can be programmed to perform specific tasks. The robot can be designed as a stationary robot that is permanently connected to its location. Alternatively, the robot can be designed as a mobile robot that allows the robot to change its location. Optionally, the robot can be designed to recognize objects using a camera system and optionally process them.

[0031] A reference object can be an object that corresponds to an object to be recognized according to the method. Optionally, several identical and / or different reference objects can be used.

[0032] The fact that the assessment of the suitability of the alignment and / or focus of the camera for the object detection carried out can be understood in such a way that the assessment of the suitability of the alignment and / or focus of the camera is carried out alternatively or additionally for object detections to be carried out in the future, which are optionally carried out during regular operation of the camera system after setup has been completed.

[0033] The disclosure offers the advantage that detection reliability can be used as a criterion for the suitability of the selected orientation and / or setting of the focal length and / or focusing of the camera of the camera system for detecting an object. In other words, the disclosure offers the advantage that the actual use of the camera system for object detection can be used as a decision criterion regarding the orientation and / or focusing of the camera, instead of first optimizing the orientation and / or focusing of the camera itself, detached from detection reliability. The invention thus offers the possibility of simplifying the adjustment of the camera system for object detection and / or reducing the time and / or effort required for this.In particular, the disclosure may offer the advantage that a number of necessary steps for setting up the camera system for object recognition can be reduced and / or the setting up of the camera system can be partially or completely automated.

[0034] Furthermore, the disclosure offers the advantage that conventional methods for focus evaluation and focus adjustment, such as phase comparison and / or edge contrast measurement, can optionally be dispensed with, since the suitability of the selected focus of the camera system for the intended object detection can be determined based on the detection reliability of the object detection. This can reduce the technical requirements for the camera system and, optionally, lower the manufacturing costs for the camera system. Thus, the disclosure offers the advantage of providing a camera system for object detection with low manufacturing costs.

[0035] The assessment of the suitability of the camera's alignment and / or focus for the object detection performed based on the determined detection reliability can optionally be carried out by the control device itself and / or by a computing unit located remotely from the camera system as part of a computer-implemented process. This can optionally be partially or fully automated. Alternatively or additionally, the assessment of the suitability of the camera's alignment and / or focus for the object detection performed based on the determined detection reliability can be carried out by a human user. Optionally, the assessment of the suitability of the camera's alignment and / or focus for the object detection performed can be limited to whether or not a reference object is detected with sufficient reliability using the prevailing alignment and / or focus of the camera.Alternatively, the assessment of suitability may include a graded evaluation of suitability, such as the extent to which the reference object is detected given the prevailing orientation and / or focus.

[0036] The suitability of the camera's alignment and / or focus for the object detection performed can optionally be assessed solely based on the determined detection reliability. This can facilitate intuitive adjustment of the camera's alignment and / or focus. Optionally, in an attempt to improve the camera's alignment and / or focus, the camera's alignment and / or focus can be adjusted and the resulting impact on the detection reliability when detecting a reference object can be assessed.

[0037] Assessing the suitability of the camera's alignment and / or focus for the object detection being performed may include assessing whether the detection reliability is below a predetermined threshold. This can offer the advantage of ensuring reliable detection of the objects to be detected while still allowing a wide adjustment range for the camera's alignment and / or focus. This can simplify camera setup.

[0038] Furthermore, the method can optionally comprise initiating a change in the orientation and / or focus of the camera such that the recognition reliability is improved by the changed image field and / or the changed focus, provided that the assessed recognition reliability is below the predetermined threshold. In other words, if the recognition reliability is insufficient, the camera can be adjusted to improve the recognition reliability. This can offer the advantage that the recognition reliability can be used as a decision criterion when determining whether or not a change in the camera settings is necessary. This can avoid unnecessary time expenditure on optimizing the orientation and / or focus of the camera, for example in a case where the recognition reliability is already sufficient without such effort.The change in orientation and / or focus can be carried out manually by a user if necessary, for example after the camera system issues a notification to the user, and / or partially or fully automated by the camera system itself.

[0039] Steps a) to d) and optionally e) can be performed repeatedly. This offers the advantage of allowing regular and / or even continuous monitoring of detection reliability. This also offers the advantage that any insufficient detection reliability can be quickly identified and appropriate countermeasures initiated. Thus, this can offer the advantage of reliably maintaining sufficient detection reliability during operation of the camera system.

[0040] The method may further comprise outputting a signal for controlling the alignment and / or focusing of the camera system by repeatedly performing steps a) to e). In other words, the regular and optionally continuous determination of the detection reliability may allow for control to maintain the detection reliability at a predetermined level and / or change it according to a predetermined profile. This offers the possibility of automatically maintaining the detection reliability at a predetermined level and / or within a predetermined range.

[0041] Determining recognition reliability can include determining a quality and / or rate and / or certainty of the performed object recognition based on a comparison with predefined comparison information. The predefined comparison information can include image information representing objects that correspond to the objects to be recognized. Object recognition can therefore include a comparison of the captured images of the at least one reference object with the predefined comparison information, so that a similarity and / or a degree of correspondence between the object to be recognized in the captured image and the comparison information can be determined. The recognition reliability can optionally be determined based on the greatest of all values ​​for the correspondence with the comparison information.The specified comparison information can optionally be used to train artificial intelligence-based software of the camera system in order to then perform object recognition at least partially using artificial intelligence.

[0042] Initiating object recognition based on the captured image can optionally include edge detection in the captured image. Edge detection can extend to the entire captured image or to a definable or predefined portion of the captured image. Edge detection can optionally be designed to detect rectilinear textures in the image and interpret them as edges. The reliability of edge detection can optionally depend on the camera focusing on the object. Reliable edge detection is possible if the camera is sufficiently focused, while reliable edge detection is not possible if the camera is insufficiently focused due to the blurriness of the captured image.

[0043] Optionally, a captured image of a given object or of an object with a given design that has a predetermined number and / or characteristics of edges can be used for this purpose. The number and / or characteristics of edges actually detected during edge detection can then be compared with the predetermined number and / or characteristics of the edges when determining the detection reliability. If the number and / or characteristics of the actually detected edges match the number and / or characteristics of the predetermined number and / or characteristics of the edges, this can be interpreted as a high detection reliability. If, on the other hand, there is a deviation between the number and / or characteristics of the actually detected edges and the number and / or characteristics of the predetermined number and / or characteristics of the edges, a measure of this deviation can be interpreted as a measure of the loss in detection reliability.For example, a low level of edge detection capability can be inferred from a low level of detection reliability, which in turn may indicate insufficient alignment and / or focusing of the camera system.

[0044] Providing information about the determined detection reliability and / or the assessment of the orientation and / or focus of the camera system to a user of the camera system can optionally include outputting an optical signal, wherein a color and / or a flashing frequency can optionally be dependent on the determined detection reliability and thus optionally dependent on the degree of reliability of the edge detection. For example, the flashing frequency can be faster the greater the determined detection reliability, and vice versa.

[0045] A predefined graphic structure can optionally serve as the reference object. The predefined graphic structure can optionally be in the form of a reproduction of a predetermined logo, such as a logo of the camera system manufacturer, such as a TURCK logo. Optionally, object recognition can first be used to identify a sub-area of ​​the captured image in which a predefined graphic structure is recognized or suspected. Edge detection can then optionally be limited to the identified sub-area. Optionally, packaging of the camera system with a reproduction of a manufacturer's logo can serve as the object for image acquisition. This can offer the advantage that the packaging is delivered with the camera system anyway, eliminating the need to provide separate reference objects. This can keep the costs for manufacturing, delivery and / or setting up the camera system low.Optionally, the number and / or shape and / or density of edges in the specified graphical structure can be determined in advance and optionally stored in the camera system. This can make it possible to determine the detection reliability internally within the camera.

[0046] The method may further comprise providing information about the determined recognition reliability and / or the assessment of the orientation and / or focus of the camera system to a user of the camera system and / or a device connected to the camera system. Optionally, providing the information may comprise issuing a notification about any insufficient recognition reliability to enable the user and / or the connected device, such as a server, to initiate countermeasures. Optionally, the method may comprise providing information about the presence of sufficient recognition reliability to signal to the user and / or the connected device that no action is required in this regard.

[0047] Optionally, initiating the provision of information about the determined recognition reliability and / or the assessment of the alignment and / or focus of the camera system to a user of the camera system and / or a device connected to the camera system can comprise initiating the provision of information about a recommended action to the user of the camera system and / or a device connected to the camera system. Thus, optionally, the recommended action can be designed such that the user and / or a device connected to the camera system receives an indication as to whether and, if so, which setting needs to be changed with regard to the image to be captured. Optionally, this can include an indication as to whether the alignment and / or focus of the camera needs to be changed with the intention of improving object recognition.Providing information about the determined recognition reliability and / or the assessment of the alignment and / or focus of the camera system to a user can optionally be done by the camera system outputting a signal that can be perceived by the user. The camera system can have one or more output elements for this purpose. Optionally, a signal that can be perceived by the user can comprise an optical signal and / or an acoustic signal and / or a haptic signal. Optionally, the signal can be provided by an output element designed as a lighting element and / or display element, for example as a light signal by a lighting element arranged on and / or in the camera system. Optionally, the signal can be output as an acoustic signal by an output element of the camera system designed as a loudspeaker.Optionally, the signal can be output as a haptic signal using an output element configured as a vibration element. This can offer the advantage of providing the user with simple and / or efficient feedback on the success of object detection and / or the current detection reliability.

[0048] Initiating the provision of information about the determined recognition reliability and / or the assessment of the alignment and / or focus of the camera system to a device connected to the camera system may include initiating the provision of a machine-interpretable signal, such as an electronic and / or optical signal. The signal may optionally be provided via a suitable data interface and communication connection between the camera system and the device, such as via a wired and / or wireless network and / or an optical network with fiber optics.

[0049] Performing object detection based on the captured image and / or determining the detection reliability of the performed object detection and / or assessing the orientation and / or focus of the camera system can be done using artificial intelligence. Optionally, the comparison information can be used to train the artificial intelligence.

[0050] The initiation of object recognition based on the captured image can be carried out by a control unit of the camera system. As a result of the initiation, the control unit and / or a computing unit located remotely from the camera system can carry out the object recognition. In other words, the control unit can initiate the object recognition and carry out the process itself and / or have the object recognition carried out elsewhere, for example by a server located remotely from the camera system. The latter can offer the advantage that the hardware requirements for the control unit can be kept low, since any complex image analysis and / or object recognition does not have to be carried out by the control unit itself. Carrying out the process by the control unit itself can offer the advantage that data transmission to a remote computing unit is not absolutely necessary.Accordingly, no communication connection needs to be provided.

[0051] Optionally, the determination of the recognition reliability during the object detection process can be initiated by a control unit of the camera system. Optionally, the control unit and / or a computing unit located remotely from the camera system can then carry out the determination of the recognition reliability. The latter can offer the advantage that the hardware requirements for the control unit can be kept low, since any complex image analysis and / or object detection does not have to be carried out by the control unit itself. Carrying out the process by the control unit itself can offer the advantage that data transmission to a remote computing unit is not absolutely necessary. Accordingly, no communication connection needs to be provided. The assessment of the orientation and / or focus of the camera can optionally be carried out by a control unit of the camera system.As a result of the initiation, the control unit and / or a computing unit located remotely from the camera system can assess the alignment and / or focus of the camera. To do this, the control unit and / or the computing unit located remotely from the camera system can output a corresponding control signal to the camera system or the camera. Implementation by the remote computing unit can offer the advantage that the hardware requirements for the control unit can be kept low, since any complex image analysis and / or object recognition does not have to be performed by the control unit itself. Implementation by the control unit itself can offer the advantage that data transmission to a remote computing unit is not absolutely necessary. Accordingly, no communication connection needs to be provided.Optionally, a control unit can initiate a change in the orientation and / or focus of the camera. Following this action, the camera of the camera system can perform the change in orientation and / or focus. Optionally, following this action, a human user can manually adjust the orientation and / or focus of the camera. The control unit can optionally actively issue a notification to the user as a trigger. Optionally, the trigger can simply be the user's recognition of a need for improvement in detection reliability.

[0052] The camera system can optionally be designed to detect objects moving relative to the camera or objects stationary relative to the camera. Optionally, the camera system can be configured to detect objects moving relative to the camera system on a conveyor belt. Optionally, the camera system can be integrated into a mobile robot and detect objects in the environment in which the robot is moving. Optionally, before performing object detection, the camera system can be initially aligned such that the object is at least partially located within the camera system's field of view. This can define an area in which objects are to be detected.

[0053] An example of an optional embodiment of the disclosure is described below, which, however, does not limit the disclosure content.

[0054] A first step in setting up the camera system for object recognition can involve learning the objects to be recognized later, or the states of these objects—that is, training an artificial intelligence designed for object recognition. This step can optionally be performed under "laboratory conditions," for example, in a location with ideal lighting conditions, a contrast-free background (a solid-color screen), and / or with stationary objects.

[0055] If different states of a single object are to be recorded, e.g., the ideal state and one or more typical defect states, several sample images can be taken for each and assigned, for example, the quality criteria "good" or "poor." Defect states of a packaging unit as an object can optionally include an unsealed or improperly sealed box, damage such as holes or dents, missing labels, and / or incorrect markings.

[0056] A collection of these comparison images of the objects to be detected during subsequent operation can optionally be stored in the camera system itself and / or in an external storage device, such as a server, and / or in a database. This allows trained images to be easily transferred to multiple cameras that are to be used, for example, at different locations within a production facility or similar. Setting up the camera system at the actual site of use can begin with the mechanical installation of the camera system. In a subsequent adjustment phase, some or all of the variable parameters can be adjusted during the actual operation of the camera system and / or special setup scenarios can be run through.

[0057] First, the largest possible field of view can be set by – depending on the given situation / equipment – ​​maximizing the distance of the camera (e.g., the front lens) from the object plane and / or selecting the smallest possible focal length ("wide angle") of the lens. This allows the smallest possible magnification of the camera's optical image to be set. This, in turn, can lead to the largest possible extension of the field of focus between the front lens and the background ("depth of field"). Optionally, a camera aperture can be set alternatively or additionally.

[0058] For objects that move through the field of view relative to the camera - e.g. on a conveyor belt - the largest possible field of view can also represent the setting with the longest residence time of the objects to be detected in the field of view of the camera.

[0059] This first parameter, “field of view,” can – like the other parameters – be adjusted automatically, electromechanically and / or by manual intervention by an operator.

[0060] Based on this initial setting, an initial check of the camera system's feedback regarding the detection reliability of the objects captured in the image field can be performed. This feedback can optionally be provided via optical signaling.

[0061] In addition to or alternatively to a binary feedback, “recognized” (e.g. green

[0062] For detection of a "clear" (e.g., "lights") or "not detected" (e.g., no light), a graduated signal can optionally be provided, which corresponds to a detection rate or certainty rate of the image recognition algorithm used. This rate (e.g., a real number between 0 = "no detection" and 1 = "certainly detected") can be mapped to a color-coded feedback signal or a flashing frequency of a signal lamp. For example, a rate of Q=0 can be signaled by no light; Q=0.5 by a flashing frequency of 3 Hz; Q=0.7 by a flashing frequency of 10 Hz; and Q>0.9 by continuous light.

[0063] The feedback can optionally be provided in the form of an analog and / or digital signal (any or according to a suitable standard) to a control device, which evaluates or further processes this signal. This control device can optionally be implemented as an internal circuit and / or as an internal module of the camera system.

[0064] In some cases, it may be possible to improve detection reliability during the adjustment phase by adjusting the lens's focusing distance (focus position). If an optical signal is provided to an operator, the operator can manually operate a corresponding lens adjustment ring, thus improving detection reliability. This can be achieved more easily the faster the continuous feedback—that is, the feedback on detection reliability corrected after an adjustment—arrives. This delay can be less than one second, or less than 1 / 10 of a second.

[0065] In the case of automatic operation (“autofocus”), the feedback signal from the detection module optionally leads to automatic adjustment of the lens focus, e.g., by a servo motor and / or with the aid of an ultrasonic process.

[0066] Optionally, a magnification of the objects to be detected can be used as an additional parameter to improve the detection rate. As described above, the minimum magnification is optionally set initially. By moving the camera closer to the object plane and / or by setting a longer focal length, an object can be imaged larger and thus potentially detected more reliably. If several different objects are to be detected, this setting can optionally be tested for all of these objects.

[0067] The method can be particularly advantageous when the difference between the smallest and largest object to be detected is not too great. The largest of all objects can determine the largest possible image scale, since it should optionally fit completely within the image field. In the case of moving objects, the dwell time of the largest object in the image field must optionally be sufficiently long. With this application-specific largest possible image scale, even the smallest of all objects should optionally be sufficiently detected. If this is not possible, it can be advantageous to accordingly restrict the selection of a set of all objects to be detected with a single camera setting.

[0068] Depending on the optical properties of the camera system—especially the depth of field of the camera or the lens system—the correct setting of the focal plane (focusing distance) can also be assessed based on the recognition reliability of the captured object. This is all the more important the larger the image scale—and thus the shallower the depth of field—of the optical image.

[0069] Optionally, a user can specify a threshold for detection reliability. The threshold can optionally be between 0 and 1. This informs the camera that this threshold is "sufficient" for reliable object detection. If the current detection reliability reaches or exceeds this threshold, this can affect the external signaling (e.g., a continuous light or green signal). Furthermore, an additional signal can be output in this case, e.g., via a signal line. It is also possible for the camera to start a new step of an automated setup procedure or to automatically end the current setup procedure when the threshold is reached.

[0070] Optionally, an algorithm based on artificial intelligence can be used to detect errors in object recognition, such as an image scale that is too small, a resolution that is too low, and / or an insufficient dwell time of the (moving) object in the image field.

[0071] Optionally, the process can adjust the image focus. If the captured image is not sufficiently sharp, the direction in which the focus should be moved can be signaled by flashing and / or coloring an illuminated indicator, e.g., by rotating a lens ring, readjusting a mounting bracket, or any necessary camera movement if the camera's focus range is insufficient. When the focus point is reached, this can optionally be signaled by a continuous light or a color change. Optionally, an automated change in the camera's position along the optical axis can occur or be initiated. Optionally, a robot can receive a control signal to change the camera's position.

[0072] Optionally, for a multi-object mode of the setting procedure, individual images or different objects from the database of comparison information can be assigned individual weightings, which can be taken into account when calculating the overall recognition reliability.

[0073] All disclosures presented for the computer-implemented method are also to be considered as disclosed for the camera system, the control unit, the computer program, the computer-readable medium and the robot and vice versa.

[0074] The features and embodiments mentioned above and explained below are to be considered disclosed not only in the explicitly stated combinations, but are also encompassed by the disclosure in other technically reasonable combinations and embodiments. Further details and advantages will now be explained in more detail using the following examples and optional embodiments with reference to the figures.

[0075] They show:

[0076] Fig. 1 A and 1 B show a camera system 10 according to an optional embodiment;

[0077] Fig. 2 to 4 show methods according to optional embodiments for setting and / or operating a camera system for detecting an object;

[0078] Figure 5 shows a robot according to an optional embodiment in a schematic representation.

[0079] In the following figures, identical or similar elements in the various embodiments are designated by identical reference numerals for the sake of simplicity.

[0080] Figure 1A shows a schematic side view of a camera system 10 according to an optional embodiment for detecting an object 12. The camera system 10 comprises a camera 14 with a lens 14a with variable orientation and focus, which can be aligned with an object 12 to be detected such that the object 12 is at least partially located within an image field 16 of the camera 14. The image field 16 is spanned by the image angle, which is symbolically represented by the lines 16a.

[0081] In addition, the camera system 10 has a control unit 18 which is connected to the camera system 10 or integrated into the camera system 10 and is configured to cause an image of the object 12 located in the image field 16 to be captured by means of the camera 14 and to cause an object recognition to be carried out on the basis of the captured image.

[0082] The control unit 18 is further configured to initiate a determination of the recognition reliability during the performed object recognition and to provide information about the determined recognition reliability, which enables an assessment of the orientation and / or focus of the camera 14 for the performed object recognition based on the determined recognition reliability. The control unit 18 can be optimized for executing the steps using artificial intelligence.

[0083] The camera system 10 can change the orientation of the camera 14 by changing an orientation of the optical axis 100 of the camera 14 and / or by changing a focal length of the lens 14a.

[0084] The camera system 10 can be configured to carry out the object recognition and / or the determination of the recognition reliability and / or the assessment of the orientation and / or the focus of the camera 14 by means of the control unit 18 itself and / or by means of a remote computing unit 20 connected to the control unit.

[0085] The camera system may further comprise an output element 22 for outputting a notification to a user. Optionally, the output element 22 may comprise a light-emitting element, such as an LED, for outputting a light signal to the user. The notification may optionally provide information about whether or not there is sufficient recognition reliability.

[0086] The object can optionally be arranged on a support 24 provided for this purpose. This support can optionally be designed as a conveyor belt to move the object 12 or multiple objects relative to the camera 14. The camera 14 can optionally offer an adjustment option, e.g., a focusing ring 26, for the camera. This allows the image captured by the camera to be manually and / or automatically focused on the object or a predefined object plane.

[0087] The orientation of the camera 14 relative to the object 12 can optionally be changed based on the distance of the camera 14 from the object 12 along the optical axis 100, as indicated by the arrow 102. The image area 16 can optionally be varied.

[0088] Figure 1 B shows the camera system 10 in a schematic representation from the perspective of the object 12.

[0089] The camera system 10 and optionally the control unit 18 can be configured to execute a method 200 explained below with reference to Figure 2. The method 200 can optionally be configured as a computer-implemented method 200 and executed by the control unit 18 of the camera system 10.

[0090] The method 200 comprises, in a step 202, receiving an image of an object 12, captured by a camera 14 of the camera system 10, which is located as a reference object at least partially in an image field 16 of the camera 14.

[0091] In a step 204, the method 200 includes initiating an object recognition process based on the captured image. The initiation of an object recognition process can be performed based on the captured image by the control unit 18 of the camera system 10, wherein, as a result of the initiation process, the control unit 18 and / or a computing unit 20 arranged remotely from the camera system can perform the object recognition.

[0092] In a step 206, the method 200 further comprises initiating a determination of a recognition reliability in the performed object recognition. Determining the recognition reliability can comprise determining a quality and / or rate and / or certainty of the performed object recognition based on a comparison with predetermined comparison information. The initiating the determination of the recognition reliability in the performed object recognition can be performed by the control unit 18 of the camera system, wherein, as a result of the initiation, the control unit 18 and / or the computing unit 20 arranged remotely from the camera system or another computing unit can perform the determination of the recognition reliability.

[0093] In a step 208, the method 200 comprises assessing the suitability of an orientation and / or focus of the camera 14 for the object detection performed based on the determined detection reliability. The assessment of the orientation and / or focus of the camera can be initiated by a control unit of the camera system, wherein, as a result of the initiation, the control unit 18 and / or the computing unit 20 arranged remotely from the camera system or another computing unit can carry out the assessment of the orientation and / or focus of the camera 14. Alternatively or additionally, the assessment of the suitability of the orientation and / or focus of the camera (14) for the object detection performed can be carried out by a human user of the camera system based on the determined detection reliability.Optionally, assessing the suitability of an alignment and / or a focus of the camera 14 for the object detection performed may include assessing whether the determined detection reliability is below a predetermined threshold.

[0094] In an optional step 210, the method 200 may include initiating a change in the orientation and / or focus of the camera 14 such that the recognition reliability is improved by the changed image field and / or the changed focus, provided the assessed recognition reliability is below the predetermined threshold. The change in the orientation and / or focus of the camera may be initiated by the control unit 18, and as a result of the initiation, the camera 14 of the camera system 10 performs the change in orientation and / or focus.

[0095] The method may optionally further comprise, in a further step 212, outputting a signal for controlling the alignment and / or focusing of the camera system 10 by repeatedly performing steps 202 to 210.

[0096] Furthermore, in a step 214, the method 200 can optionally include providing information about the determined recognition reliability and / or the assessment of the orientation and / or focus of the camera system to a user of the camera system and / or a device connected to the camera system 14. Providing the information can optionally include outputting a hint via the output element 22. This can enable a user to assess the suitability of the orientation and / or focus of the camera system for object recognition.

[0097] Performing object recognition based on the captured image and / or determining recognition reliability in the performed object recognition and / or assessing the orientation and / or focusing of the camera system 14 can be done using artificial intelligence.

[0098] Steps 202 to 208 and optionally 210 to 214 can be performed repeatedly.

[0099] With reference to Figure 3, an exemplary method 300 for operating a camera system for detecting an object is provided, without the disclosure being limited to the example. First, in step 302, the camera system 10 is initially assembled and installed. In step 304, a check is performed to determine whether the camera system 14 is ready for use. If this is not the case, a warning is output in step 304a, for example, by a red light element that illuminates and / or flashes at short intervals.

[0100] If the camera system 10 is ready for use, image recognition within the adjustable focus range occurs in step 306. If this is unsuccessful, a warning is issued in step 306a, for example, by a yellow light element and / or a light element that flashes briefly three times.

[0101] If image recognition was successful in step 306, a check is performed in step 308 to determine whether the image recognition is sharp. If so, an indication is output in step 308a, for example, by means of a green and / or permanently illuminated light element.

[0102] In the case of blurred image recognition, in case 310, in which the focus area is too close for the detected object, a warning is output in step 310a, for example, by a blue illuminated or briefly flashing light element. In case 312, in which the focus area is too far away for the detected object, a warning is output in step 312a, for example, by a violet illuminated and / or long-flashing light element.

[0103] After adjusting the settings of camera system 10, a further check 314 is performed to determine whether the image recognition is sharp. If this is not the case, steps 310 and 310a, or 312 and 312a, are repeated. If, however, the image recognition is sharp, an indication is output in step 316, for example, by means of a green illuminated light element. In step 318, the setup of camera system 10 is thus completed.

[0104] With reference to Figure 4, an exemplary method 400 for operating a camera system 10 for detecting an object 12 is provided, without the disclosure being limited to the example. In step 402, an artificial intelligence of the camera system 10 is first trained for object detection, and the camera system 10 is installed at the intended location.

[0105] In step 404, a minimum possible image scale is set.

[0106] In step 406, the focus is set to a minimum possible distance.

[0107] In step 408, the first or next object 12 is presented and object recognition is performed.

[0108] In step 410, it is determined whether the recognition reliability has reached a predefined threshold for object recognition. If so, in step 412, it is checked whether another object and / or scenario is intended for object recognition. If so, step 408 is repeated. If not, in step 414, object recognition is terminated with an indication that object recognition has been successfully completed.

[0109] If it is determined in step 410 that the detection reliability has not reached a predetermined threshold during object detection, a check is performed in step 416 to determine whether the maximum distance is present during focusing. If not, the distance setting of the lens 14a is increased in step 418, and a new object detection is subsequently performed; then step 410 is repeated.

[0110] If the maximum distance is present during focusing, a check is performed in step 420 to determine whether the maximum magnification has been reached. If not, the magnification is increased in step 422, the focusing distance is reduced or minimized, and then the object is detected again; then step 410 is repeated. If step 420 results in the maximum magnification being reached, the object detection is terminated unsuccessfully in step 424, and a corresponding warning is output, indicating that the object is not sufficiently detectable. Figure 5 shows a schematic representation of a robot 28 with a camera system 10 according to an optional embodiment.

[0111] List of reference symbols

[0112] 10 Camera system

[0113] 12 objects

[0114] 14 Camera

[0115] 14a lens

[0116] 16 image fields

[0117] 16a angle of view

[0118] 18 Control unit

[0119] 20 computing units

[0120] 22 Output element

[0121] 24th edition

[0122] 26 Adjustment option or distance ring

[0123] 28 robots

[0124] 100 optical axis of the camera

[0125] 200 Methods for setting up a camera system for detecting an object

[0126] 202-214 Procedural steps

[0127] 300 Method for operating a camera system for detecting an object

[0128] 302-318 Procedural steps

[0129] 400 Procedures for operating a camera system

[0130] 402-424 Procedural steps

Claims

Patent claims 1 . Method (200) for setting a camera system (10) for the detection of an object (12), the method (200) comprising: a) receiving (202) an image, captured by a camera (14) of the camera system (10), of an object (12) located as a reference object at least partially in an image field (16) of the camera (14); b) initiating an implementation (204) of an object detection based on the captured image; characterized in that the method (200) further comprises: c) initiating a determination (206) of a detection reliability in the performed object detection;and d) assessing (208) a suitability of an alignment and / or a focus of the camera (14) for the object recognition carried out on the basis of the determined recognition reliability e) arranging for the provision (212) of information about the determined recognition reliability and / or the assessment of the alignment and / or focus of the camera system (10) to a user of the camera system (10) and / or a device connected to the camera system (10); 2. The method (200) according to claim 1, wherein the recognition reliability is a measure of whether or not an object corresponding to a reference object was recognized during an object recognition carried out on the basis of the captured image.

3. The method according to claim 1 or 2, wherein the recognition reliability comprises a first stage and a second stage, wherein - the first level of recognition reliability is achieved if, when the object recognition is carried out on the basis of the captured image, an object corresponding to a reference object is detected, and The second level of recognition reliability exists when, when the object recognition was carried out on the basis of the captured image, an object corresponding to a reference object was not recognized.

4. The method (200) according to any one of the preceding claims, wherein causing a provision (212) of information about the determined recognition reliability and / or the assessment of the alignment and / or focus of the camera system comprises causing a signal to be output by means of an output element of the camera system.

5. The method according to claim 4, wherein the output element comprises one or more of the following elements: - a display element and / or lighting element for outputting an optical signal; - a loudspeaker for emitting an acoustic signal; - a vibration element for emitting a haptic signal; and - a data interface for outputting an electronic and / or optical, machine-interpretable signal.

6. Method (200) according to one of the preceding claims, wherein the assessment of the suitability of the alignment and / or the focusing of the camera (14) for the object recognition carried out is carried out exclusively on the basis of the determined recognition reliability.

7. The method (200) according to any one of the preceding claims, wherein the assessment (208) of a suitability of an alignment and / or a focus of the camera (14) for the object recognition carried out comprises assessing whether the determined recognition reliability is below a predetermined threshold value.

8. The method (200) of claim 7, further comprising: f) initiating a change (210) in the alignment and / or focusing of the camera (14) such that the detection reliability is improved by the changed image field (16) and / or the changed focusing, provided that the assessed detection reliability is below the predetermined threshold value.

9. The method (200) according to any one of the preceding claims, wherein steps a) to e) and optionally f) are performed repeatedly.

10. The method (200) according to claim 8 or according to claim 9 when dependent on claim 8, further comprising outputting a signal for controlling the alignment and / or focusing of the camera system (10) by repeatedly performing steps a) to f).

11. Method (200) according to one of the preceding claims, wherein determining (206) a recognition reliability comprises determining a quality and / or rate and / or a certainty of the object recognition performed based on a comparison with predetermined comparison information.

12. The method (200) according to any one of the preceding claims, wherein performing (204) the object recognition based on the captured image and / or determining (206) a recognition reliability in the performed object recognition and / or assessing (208) the alignment and / or focusing of the camera system (10) are carried out using artificial intelligence.

13. Method (200) according to one of the preceding claims, wherein the initiation of the implementation (204) of an object recognition on the basis of the captured image is carried out by a control unit (18) of the camera system (10), and wherein, as a result of the initiation, the control unit (18) and / or a computing unit (20) arranged remotely from the camera system (10) carries out the object recognition.

14. The method (200) according to any one of the preceding claims, wherein the determination (206) of the recognition reliability during the performed object recognition is initiated by a control unit (18) of the camera system (10); and wherein, as a result of the initiation, the control unit (18) and / or a computing unit (20) arranged remotely from the camera system (10) carries out the determination (206) of the recognition reliability.

15. The method (200) according to any one of the preceding claims, wherein the initiation of the assessment (208) of the alignment and / or the focus of the camera (14) is carried out by a control unit (18) of the camera system (10), and wherein, as a result of the initiation, the control unit (18) and / or a computing unit (20) arranged remotely from the camera system (10) carries out the assessment of the alignment and / or the focus of the camera (14).

16. The method (200) according to any one of the preceding claims, wherein a change (210) in the orientation and / or focus of the camera is initiated by a control unit (18), and wherein, as a result of the initiation, the camera (14) of the camera system (10) carries out the change in the orientation and / or focus.

17. Use of a recognition reliability of an object recognition carried out by means of a camera system (10) as a controlled variable for controlling the alignment and / or focusing of the camera system (10).

18. Control unit (18) for a camera system (10) for the detection of an object, characterized in that the control unit (18) is configured to carry out a method according to one of claims 1 to 16.

19. A computer program, characterized in that the computer program comprises instructions which, when the program is executed by a computer, cause the computer to carry out a method (200) according to one of claims 1 to 16.

20. A computer-readable medium, characterized in that the computer-readable medium comprises instructions which, when executed by a computer, cause the computer to carry out a method (200) according to any one of claims 1 to 16.

21. Camera system (10) for detecting an object (12), the camera system (10) comprising: - a camera (14) with variable orientation and focus, which can be aligned with an object (12) to be recognized in such a way that the object (12) is at least partially located in an image field (16) of the camera (14); - a control unit (18) which is connected to the camera system (10) and is configured to: - to cause an image of the object (12) located in the image field (16) to be captured by means of the camera (14); and - to initiate an object recognition based on the captured image; characterized in that the control unit (18) is further configured to: - to determine the reliability of the object detection carried out; and - to provide information about the determined recognition reliability, which enables an assessment of the suitability of the alignment and / or focusing of the camera (14) for the object recognition carried out on the basis of the determined recognition reliability; and - to arrange for information about the determined recognition reliability and / or the assessment of the alignment and / or focus of the camera system (10) to be provided to a user of the camera system (10) and / or a device connected to the camera system (10).

22. Camera system (10) according to claim 21, wherein changing the orientation of the camera (14) is carried out by changing an orientation of the optical axis (100) of the camera (14) and / or by changing the position of the camera along the optical axis (100) and / or by changing a focal length of the camera (14).

23. Camera system (10) according to claim 21 or 22, wherein the control unit (18) is optimized to execute the steps using artificial intelligence.

24. Camera system (10) according to one of claims 21 to 23, wherein the camera system (10) is configured to carry out the object recognition and / or the determination of the recognition reliability and / or the assessment of the orientation and / or the focus of the camera (14) by means of the control unit (18) and / or by means of a remote computing unit (20) connected to the control unit (18).

25. Camera system (10) according to one of claims 21 to 24, further comprising an output element for providing information about the determined recognition reliability and / or the assessment of the orientation and / or focus of the camera system.

26. Camera system (10) according to claim 25, wherein the output element comprises one or more of the following elements: - a display element and / or lighting element for outputting an optical signal; - a loudspeaker for emitting an acoustic signal; - a vibration element for emitting a haptic signal; and - a data interface for outputting an electronic and / or optical, machine-interpretable signal.

27. Robot (28) comprising a camera system (10) according to one of claims 21 to 26.

Images