Camera device and method for detecting moving object flow

Abstract

The present application relates to the detection of a flow of moving objects. A camera device (10) for detecting a flow of objects (54) moving relative to the camera device (10) is provided, the camera device having an image sensor (20) for recording image data of the object (54), a geometry detection sensor for measuring the object (54), and a control and evaluation unit (42) designed to determine at least one region of interest (74) based on the measurement data of the geometry detection sensor in order to limit the evaluation of the image data to the region of interest (74). The image sensor (20) has a configuration unit (22) in order to be able to read only an adjustable portion of the corresponding recorded image data, and the control and evaluation unit (42) is designed to read only the portion of the image data determined based on the region of interest (74) from the image sensor (20).

Description

[0001] The present invention relates to a camera device and a method for detecting a moving object flow.

[0002] In industrial applications, cameras are used in various ways to automatically detect object properties, for example for inspecting or measuring objects. Here, an image of the object is recorded and, depending on the task, evaluated by image processing methods. Another application of cameras is reading codes. An object with a code located on it is recorded with the aid of an image sensor, the code area is identified in the image and subsequently decoded. Camera-based code reading can also easily cope with other code types besides one-dimensional barcodes, which, like matrix codes, are also constructed two-dimensionally and provide more information. In principle, automatic text detection of printed addresses (OCR, Optical Character Recognition) or automatic text detection of handwritten fonts is also code reading. Typical areas of application for code readers are supermarket checkouts, automatic package identification, mail sorting, airport baggage handling and other logistics applications.

[0003] A common inspection scenario is to mount the camera above a conveyor belt. During the relative movement of the object stream on the conveyor belt, the camera records images and initiates further processing steps based on the object properties obtained. These processing steps include, for example, further processing adapted to the specific objects on the machine acting on the conveyed objects or changing the object stream by excluding certain objects from the object stream or sorting the object stream into a number of sub-object streams within the framework of quality control. If the camera is a camera-based code reader, the objects are identified based on the attached code for correct sorting or similar processing steps.

[0004] More powerful image sensors are increasingly becoming available, so that large areas, such as the width of a conveyor belt, can be covered with few cameras. However, in the case of objects of varying heights, there will always be situations where the depth of field is not wide enough to record all objects in the field of view with sufficient clarity. In principle, focus adjustment offers a way out here. However, for this purpose, fast recording sequences are required, since otherwise the objects to be recorded will no longer be recorded or at least no longer at the correct perspective. In the case of large image sensors, the required high recording frequencies cannot be achieved, since more data is generated than can be processed between two recordings. Smaller image sensors would help in this regard, but in doing so would sacrifice the intended advantage of covering large areas with a few cameras or, ideally, with a single camera.

[0005] In principle, the depth of field range can be extended with a small aperture and additional lighting to compensate for the light loss and thus the poorer signal-to-noise ratio. However, this would make the already powerful lighting even more expensive. Furthermore, this would not reduce the data load. Furthermore, one could consider using image sensors with an extra-wide aspect ratio, with which the conveyor belt could be covered without excessively increasing the amount of data. However, such exotic image sensors are not available or would at least significantly limit the choice with regard to other factors that are important for the image sensor. Furthermore, a large visible range has its advantages even in the conveying direction, which are lost with an extra-wide image sensor.

[0006] EP 1 645 838 B1 and EP 1 645 839 B1 each disclose a device for monitoring moving objects on a conveyor belt. A rangefinder or a laser scanner is located upstream of the camera and its data are used to determine a region of interest (ROI), to which the subsequent evaluation of the image data is limited. Here, a line scan camera is used, whose image lines are combined to form an image. Therefore, all image data are first generated and only then reduced. Matrix cameras are only mentioned in passing, without explaining how their image data are processed. In this form, matrix cameras only exacerbate the problem of the amount of data compared to line scan cameras, since the image lines are often even detected redundantly.

[0007] In DE 20 2006 020 599 U1, previously measured geometric data are used in order to determine the region of interest, which is then detected with a higher resolution than the region of no interest. The difference in one embodiment is that the image data is evaluated more completely in the region of interest than in the region of no interest, which then leads only to another variant of EP 1 645 838 B1 and EP 1 645 839 B1. Alternatively, in the region of no interest, for example only every third or every tenth line is evaluated. Therefore, less data can only be obtained if the entire field of view of the line camera is uniformly a region of no interest. Furthermore, the method is not transferable to a matrix camera at all.

[0008] EP 2 693 363 A1 describes a camera system that uses previously detected geometric data to complementarily focus multiple cameras. This is based on redundant detection of objects in multiple cameras and is therefore not suitable for systems where only one camera covers the entire width of the object stream to be detected. In addition, due to the redundancy, the total data volume is even doubled again.

[0009] A camera is known from EP 3 537 339 A2 in which an optoelectronic distance sensor according to the principle of the light flight method is integrated. Its data are used for a variety of functions, among which is also the determination of a region of interest based on the height profile obtained from the distance values. However, the region of interest is regarded as additional information or the image is cropped to the region of interest in a possible processing step. As a result, the data to be processed is either not reduced at all or is only reduced in a later processing step.

[0010] US 9 237 286 B2 proposes an image sensor which allows energy-efficient reading of partial images. The term region of interest (ROI) is mentioned in the introduction, but the document does not deal with applications, but rather describes image sensors at the hardware level.

[0011] It is therefore an object of the present invention to further improve the detection of flows of moving objects.

[0012] This task is achieved by a camera device and a method for detecting a stream of moving objects. The camera device uses an image sensor to record image data of objects moving relative to it. Preferably, the image sensor covers most or the entire width when viewed from a top view, or covers the height of the stream and a certain length in the direction of movement when viewed from a side view, and therefore preferably has a plurality of light receiving elements or pixels in a matrix arrangement with a resolution of typically several million pixels. The geometric detection sensor measures the geometry of the object. Distance sensors that measure the corresponding distance to the object are particularly suitable for this. Thereby, a height profile can be generated in the direction of movement and preferably also transversely to the direction of movement, knowing the position and posture of the geometric detection sensor during the relative movement.

[0013] The control and evaluation unit uses the measurement data of the geometry detection sensor to determine at least one region of interest (e.g. with an object), an object surface or other structure associated with the camera device. The further evaluation of the image data is then limited to the region of interest ("Cropping"). If the measurement data contain intensity information or color information, non-pure geometric information (e.g. brightness, color or contrast) can alternatively or supplementarily be used to determine the region of interest. In order to determine the region of interest in the plane of the image sensor based on the geometry data, the image sensor and the geometry detection sensor are preferably calibrated to each other and a corresponding transformation is performed.

[0014] The invention is based on the basic idea of ​​limiting the image data at the source in the image sensor directly to at least one region of interest. The image sensor has a configuration unit and thus provides a configuration possibility in order to be able to read only an adjustable part or sub-region of the image data. The control and evaluation unit uses this configuration possibility in order to adapt the part of the image data to be read to the at least one region of interest. In particular, exactly those pixels or pixel rows that form the region of interest are read. However, this is only preferably consistent with pixel accuracy, buffers or cropped sub-regions of the region of interest can be considered, as well as restricted configuration possibilities that only allow or restrict access to certain groups of pixels together. Image data that are not relevant to the region of interest do not even have to be read.

[0015] The advantage of the invention is that the image data are reduced to the essential elements right from the start. This has great advantages at many points in the image processing chain. Bandwidth for data transmission, storage and computing resources as well as processing time are saved. This does not involve quality losses, since non-interesting areas do not contain any application-relevant information anyway. The disadvantages described at the beginning of covering the entire width or height of the object stream with fewer or only one camera are thus overcome. In combination with a focus adjuster, it can be ensured that all areas of interest are imaged sharply in at least one recording.

[0016] Preferably, the control and evaluation unit is designed to adjust at least one region of interest and / or the portion of the image data that is read out between recordings. The object with the structure of interest is in relative motion with respect to the camera device, so that the position of the region of interest in the plane of the image sensor constantly changes. Furthermore, objects leave the visible range and new objects enter. The adjustment will take this into account. Therefore, the image sensor or its configuration unit preferably offers the possibility of dynamic reconfiguration, preferably with a short response time that is lower than the recording cycle. The adjustment can, but does not necessarily have to, be carried out after each individual recording. Furthermore, the control and evaluation unit can conclude during the adjustment that the previous configuration is still suitable for the next recording, in particular by determining and reading out the region of interest using a certain buffer that is sufficient for the short time period between two or more recordings.

[0017] Preferably, the control and evaluation unit preferably has a preprocessing unit in order to read and preprocess the image data from the image sensor, wherein the preprocessing unit is designed such that a complete preprocessing time is required for reading and preprocessing the recorded complete image data of the image sensor, and wherein the image sensor is operated with a (particularly flexible) recording frequency that allows a shorter time between two recordings than the complete preprocessing time. Preferably, the preprocessing unit comprises at least one FPGA (Field Programmable Gate Array), at least one DSP (Digital Signal Processor), at least one ASIC (Application Specific Integrated Circuit), at least one VPU (Video Processing Unit) or at least one neural processor and performs preprocessing steps such as equalization, brightness adjustment, binarization, segmentation, finding code areas, etc. In order to read and preprocess the complete image of the image sensor, the preprocessing unit requires a complete processing time, 25 ms being a purely numerical example for ease of understanding. This is expressed in a virtual form, because according to the invention, the complete image data is not read. Nevertheless, this describes the hardware resources of the preprocessing unit, such as bandwidth for data transmission, computing power or storage capacity.

[0018] Preferably, the recording period of the image sensor is set to be shorter than the complete preprocessing time. In other words, the recording frequency is so high that the preprocessing unit can no longer process the complete image data. In the example where the complete preprocessing time is 25ms, the maximum recording frequency that can still be tolerated is 40Hz, but the image sensor operates at a higher frequency of, for example, 50-100Hz or higher. After the preprocessing, further image processing steps, such as code reading, can be performed, which image processing steps also require more time even in the case of a pipeline structure. This also increases the advantage of specifically reading image data associated with the area of ​​interest. The recording frequency can be flexible. Once the configured part of the image data has been read, the next image can be recorded, and the time required for this is related to the size of the currently determined area of ​​interest. Here, the flexible recording frequency remains higher than the recording frequency corresponding to the complete preprocessing time.

[0019] Preferably, the configuration unit is designed for selecting image rows. As a result, the image data read out are not adapted completely precisely to the region of interest, which utilizes only a portion of the width of the row. For this reason, the hardware of the image sensor and its controller for reading out are simplified. The processing time in the entire image processing chain starting from the reading out decreases linearly with the unselected image rows. Here, without limiting the generality, it is assumed that the row direction of the image sensor is arranged transversely to the transport direction. Another alternative is also conceivable, in which the columns play the role of the rows and vice versa, which should not be distinguished separately in language.

[0020] Preferably, the configuration unit is designed to select rectangular sub-regions. Thus, not only can entire rows be excluded, but also the region of interest can be cropped directly at the source in the image sensor in the row direction. The hardware structure and control of the image sensor are slightly more complex, but the image data to be read and processed are reduced more precisely.

[0021] Preferably, the control and evaluation unit is designed to read from the image sensor only a portion of the image data recorded according to the region of interest within the depth of field of the image sensor. Only if the image data are clearly recorded can the further image processing chain fulfill its function. There may be at least one further region of interest, which is located outside the depth of field range and whose image data are not even read. Here, further processing steps, in particular attempts to decode the code, will most likely only waste time without achieving the desired result. Preferably, the focus adjuster ensures that the region of interest is located in the depth of field range. However, this alone does not solve the problem, because there may be multiple objects in the visible range, the height differences of which force a part of the field of view to exceed each depth of field range. Then, despite the focus adjustment, a region of interest that is not clearly recorded still occurs.

[0022] Preferably, the control and evaluation unit is designed to determine a suitable depth of field range for the region of interest outside the depth of field range and to refocus on the suitable depth of field range for subsequent recording. Therefore, if the image data of the region of interest has to be ignored or not read at all because it will be blurred, there is a possibility of rapid compensation. To this end, with one of the subsequent recordings, preferably the recording immediately following it, after refocusing, the previously ignored region of interest is now re-recorded in the depth of field range and then also read. If even two different depth of field ranges are not sufficient to clearly record all regions of interest, the method can be iterated. Due to the image data reduced by targeted reading of only the regions of interest, in particular only the regions of interest that are clearly recorded, a much higher recording frequency can be achieved. Therefore, further recording is carried out in a timely manner when the scene is almost unchanged, at least before the object leaves the visible range. In addition, based on the measurement data of the geometric detection sensor, the sequence of focus positions can be planned in a timely manner so that each region of interest is clearly recorded frequently enough. For example, if there is still enough time, such as in the case of two objects at different heights that do not follow each other very closely, at least one further recording can be taken into account in the current focus position before the focus position is switched to another object and the region of interest associated therewith.

[0023] Preferably, the control and evaluation unit is designed to determine at least one region of interest based on the depth of field range of the image sensor. This is in some ways the opposite of the method explained last. Instead of making a selection based on the focus position among the regions of interest found according to other criteria, the focus position itself defines the region of interest in a very simple way. Everything that is recorded sharply enough can be read out, so that available image data is never discarded prematurely. A possibly more specific determination of the region of interest is then made based on more complex criteria such as edges, contrast or object surfaces. If the depth of field range is preferably also varied, as in the case of an oscillating focus position, it is always ensured that each structure is recorded and read out sharply at least once in a short period.

[0024] Preferably, the control and evaluation unit is designed to recognize the code area in the image data and to read the code content of this code area. The camera device thus becomes a camera-based code reader for barcodes and / or 2D codes according to various standards and, if necessary, for text recognition (OCR, Optical Character Reading). In code reading applications, it is particularly important that all code areas are recorded sufficiently clearly. At the same time, the code area only occupies a small part of the total area. Therefore, it is particularly effective to crop the image data in advance to the code area or at least the object that may carry the code.

[0025] Preferably, the geometry detection sensor is designed as a distance sensor, in particular as a photoelectric distance sensor according to the principle of the light time-of-flight method. The distance sensor first measures the distance of the object to the geometry sensor, but this can be converted into the height of the object, for example above a conveyor belt, knowing the position and posture of the distance sensor. Thus, a height profile is generated at least in the direction of movement. If the distance sensor has spatial resolution, a height profile transverse to the direction of movement is also detected. The term "height profile" is based on a top view of the image sensor, and the corresponding profile is meaningfully detected from another perspective. Optical principles, in particular the light time-of-flight method, are suitable for measuring distances in camera systems.

[0026] Preferably, the geometry detection sensor is integrated into the camera together with the image sensor. This results in a particularly compact system and the measurement data of the geometry detection sensor are detected directly from a viewing angle comparable to the image data. Alternatively, the geometry detection sensor is arranged externally and upstream of the image sensor in countercurrent to the flow in order to measure the object before the image data are recorded. In this case, the geometry detection sensor is, for example, a laser scanner that measures distances.

[0027] Preferably, the camera device has a speed sensor for determining the speed of the flow. For example, the speed sensor is an encoder at the conveyor belt. Preferably, the control and evaluation unit is designed to determine the speed of the flow based on the measurement data and / or image data of the geometry detection sensor. For this purpose, for example, specific structures, such as object edges, are tracked in time. The displacement vector can be correlated with the time difference to estimate the speed. The optical flow can be determined more accurately by additional correlation of the entire height profile or the entire image area. Then, no additional speed sensor or the addition of an additional speed sensor is required. The speed information can be used to convert the position in the direction of movement of the flow, in particular the measurement data of the upstream geometry detection sensor can be correlated with the position of the image sensor.

[0028] Preferably, the camera is fixedly mounted at a conveyor which conveys the objects in a conveying direction. This is a very common industrial application of cameras. The frame data of the stream of objects are known and simplify the image processing, such as the conveying direction and, at least in predictable intervals, also the speed and usually also the type and rough geometry of the object to be detected.

[0029] Preferably, the camera device has at least one image sensor for recording the flow from above and / or at least one image sensor for recording the flow from the side. In this description, detection from above is generally the guiding concept, but there are also similar situations for detection from the side, where the object spacing now changes not due to the object height but due to the lateral positioning of the object. Therefore, the above-mentioned perspective description can be understood from different perspectives in any case. Particularly preferably, the object is detected from multiple perspectives by multiple image sensors, especially in code reading, where it cannot always be guaranteed that the code is located on the upper side or completely on a specific side. Here, the terms top reading, side reading and bottom reading are sometimes used. But the latter does not cause problems in terms of depth of field range. On the contrary, for example, a window or gap must be created in the conveyor belt in order to be able to fully perceive the object. In addition, the perspective is usually in a mixed form, that is, there is no direct top view or side view, but there is an inclined component, such as side front, side back, front top or back top. Preferably, only one image sensor is set for each perspective, which covers the height or width of the object flow separately accordingly.

[0030] The method according to the invention can be further developed in a similar manner and at the same time exhibits similar advantages. Such advantageous features are described exemplarily but not exhaustively in the dependent claims which are dependent on the independent claim. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Other features and advantages of the present invention will be described in more detail below based on exemplary embodiments and with reference to the accompanying drawings. In the accompanying drawings:

[0032] Figure 1 shows a schematic cross-sectional view of a camera with an integrated distance sensor;

[0033] Figure 2 A three-dimensional view showing an exemplary application of a camera mounted at a conveyor belt;

[0034] Figure 3 A three-dimensional view showing an alternative embodiment of a camera and external distance sensor mounted at a conveyor belt;

[0035] Figure 4 A schematic cross-sectional view showing the visual range of the distance sensor and camera;

[0036] Figure 5 shows a schematic diagram of an image sensor having image rows corresponding to regions of interest configured for reading;

[0037] Figure 6 Shows something like Figure 5 Schematic diagram of a method of manufacturing a computer program product having an additional configuration of pixels to be read within an image row;

[0038] Figure 7 A schematic cross-sectional view of detecting two objects with a limited depth of field is shown;

[0039] Figure 8 A schematic diagram of an image sensor having a sub-region configured for reading, the sub-region corresponding to Figure 7 The area of ​​interest within the depth of field;

[0040] Fig. 9 Shows relative to Figure 7 A schematic cross-sectional view of detecting two objects with a changed depth of field range; and

[0041] Fig.10 Shows something like Figure 8 , but now with a corresponding Fig. 9 A configured sub-region of the region of interest within the depth of field range.

[0042] Figure 1 A schematic cross-sectional view of a camera 10 is shown. The received light 12 from the detection area 14 is incident on a receiving optical device 16 with a focus adjuster 18, which guides the received light 12 to an image sensor 20. Preferably, the optical element of the receiving optical device 16 is configured as an objective lens consisting of a plurality of lenses and other optical elements (e.g., apertures, prisms, etc.), but for simplicity, only one lens is represented here. The focus adjuster 18 is only schematically shown and can, for example, be a moving deflection mirror or liquid lens by mechanically moving an element of the receiving optical device 16 or the image sensor 20. The actuator is, for example, based on a motor, an active coil or a piezoelectric element. Preferably, the image sensor 20 has a matrix arrangement of pixel elements with a high resolution in the order of megapixels, for example 12 million pixels. The configuration unit 22 allows the readout logic of the image sensor 20 to be configured, thereby allowing a dynamically adjustable selection of pixel rows or pixel areas to be read from the image sensor 20.

[0043] In order to illuminate the detection area 14 with the emitted light 24 during recording by the camera 10, the camera 10 comprises an optional lighting unit 26, which Figure 1 2 is shown in the form of a simple light source and without emission optics. In other embodiments, a plurality of light sources (e.g., LEDs or laser diodes) are arranged, for example, in a ring around the receiving path, which light sources may also be polychromatic and may be controlled in groups or individually to adapt the parameters of the lighting unit 26, such as the color, intensity and direction of the lighting unit 26.

[0044] In addition to the actual image sensor 20 for detecting image data, the camera 10 also has an optoelectronic distance sensor 28, which uses the light time of flight method (ToF) to measure the distance to objects in the detection area 14. The distance sensor 28 includes a TOF light transmitter 30 with a TOF transmitting optics 32 and a TOF light receiver 34 with a TOF receiving optics 36. Thus, a TOF light signal 38 is emitted and received again. An optical flight time measurement unit 40 determines the flight time of the TOF light signal 38 and thereby determines the distance to the object on which the TOF light signal 38 is reflected back.

[0045] Preferably, the TOF light receiver 34 has a plurality of light receiving elements 34a or pixels and is spatially resolved. Therefore, not only a single distance value is detected, but also a spatially resolved height profile (depth map, 3D image). Here, preferably, only a relatively small number of light receiving elements 34a are provided and thus a small lateral resolution of the height profile is provided. 2×2 pixels or even 1×2 pixels are sufficient. Of course, a higher lateral resolution with n×m pixels (n,m>2) enables a more complex and more accurate evaluation. However, the number of pixels of the TOF light receiver 34 remains relatively low, for example, tens, hundreds or thousands of pixels or n,m≤10, n,m≤20, n,m≤50 or n,m≤100, which is far from the conventional megapixel resolution of the image sensor 20.

[0046] The structure and technology of the distance sensor 28 are purely exemplary. In the further description, the distance sensor 28 is regarded as a packaged module for geometric measurement, which periodically provides measurement data, such as distance values ​​or height profiles, when detecting an object or waiting for an inquiry. Here, other measurement data, especially intensity measurements, can be envisioned. Photoelectric distance measurement by means of the light flight time method is well known and will not be explained in detail. Two exemplary measurement methods are light mixing detection (Photomischdetektion) using a periodically modulated TOF light signal 38 and pulse flight time measurement using a pulse modulated TOF light signal 38. Here, there is also a highly integrated solution in which the TOF optical receiver 34 and the optical flight time measurement unit 40 or at least a part thereof (for example, a TDC (Time-to-Digital-Converter) for flight time measurement) are arranged on a common chip. A TOF optical receiver 34 configured as a matrix of SPAD (Singel-Photon Avalanche Diode) light receiving elements 34a is particularly suitable for this. The TOF optical devices 32, 36 are only symbolically shown as corresponding single lenses representing any optical device (such as a microlens array).

[0047] The control and evaluation unit 42 is connected to the focus adjuster 18, the image sensor 20 and its configuration unit 22, the lighting unit 26 and the distance sensor 28, and is responsible for control tasks, evaluation tasks and other coordination tasks in the camera 10. The control and evaluation unit determines the region of interest based on the measurement data of the distance sensor 28 and configures the image sensor 20 according to the region of interest via the configuration unit 22 of the image sensor. The control and evaluation unit reads the image data of the sub-region thus configured from the image sensor 20 and performs further image processing steps on it. Preferably, the control and evaluation unit 42 is able to find the code area in the image data and decode it, thereby turning the camera 10 into a camera-based code reader.

[0048] Preferably, the reading and the first preprocessing steps (e.g. equalization, segmentation, binarization, etc.) are carried out in a preprocessing unit 44, which comprises, for example, at least one FPGA (Field Programmable Gate Array). Alternatively, preferably at least the preprocessed image data are output via an interface 46, and the further image processing steps are carried out in a higher-level control and evaluation unit, wherein practically any distribution of work is conceivable. Based on the measurement data of the distance sensor 28, further functions can be controlled, in particular the derivation of a desired focus position of the focus adjuster 18 or a triggering time for image recording.

[0049] The camera 10 is protected by a housing 48 which is closed by a front window 50 in the front region where the received light 12 is incident.

[0050] Figure 2 A possible application of the camera 10 is shown, in which the camera 10 is mounted on a conveyor belt 52. The camera 10 is only shown here and below as a symbol, without showing that it has been installed according to Figure 1 The structure is described, only the distance sensor 28 is still shown as a functional block. The conveyor belt 52 conveys the object 54 through the detection area 14 of the camera 10, as shown by the arrow 56. The object 54 can carry a code area 58 on its outer surface. The task of the camera 10 is to detect the properties of the object 54 and, in the preferred application as a code reader, to identify the code area 58, read the code attached to it, decode it, and associate it with the corresponding belonging object 54.

[0051] Preferably, the field of view of the camera 10 covers the flow of objects 54 over the full width and over a certain length. Alternatively, additional cameras 10 are used, the visual ranges of which complement each other to achieve the full width. Here, preferably, a smaller overlap is set in any case. The perspective view shown from above is particularly suitable in many cases. Alternatively, in order to better detect the side of the object, in particular the code area 60 attached to the side, it is preferred to use additional cameras 10 (not shown) from different perspectives. Here, lateral perspectives tilted from above or from the side as well as mixed perspectives are possible.

[0052] An encoder (not shown) for determining the feed or speed can be provided on the conveyor belt 52. Alternatively, the conveyor belt is reliably moved with a known motion profile, the corresponding information is transmitted by a higher-level controller, or the control and evaluation unit determines the speed itself by tracking certain geometric structures or image features. With the speed information, geometric information or image data recorded at different points in time and at different conveying positions can be combined in the conveying direction and correlated with one another. In particular, the read code information is thus preferably also correlated with the object 54 carrying the associated code 58, 60.

[0053] Figure 3 A three-dimensional view of an alternative embodiment of an apparatus with a camera 10 on a conveyor belt 52 is shown. Instead of the internal distance sensor 28, or in addition thereto, an upstream external geometry detection sensor 62, such as a laser scanner, is provided here in the direction opposite to the conveying direction. As described above, the measurement data of the geometry detection sensor 62 can be converted into the position of the camera 10 based on the speed information. Therefore, the following description based on the internal distance sensor 28 can be transferred to the case with an external geometry detection sensor 62 without having to mention it specifically.

[0054] Figure 4 A schematic cross-section of the camera 10 is shown above an object stream, which is represented here only by a single object 54. The optical axis 64 of the distance sensor 28 is at an angle to the optical axis 66 of the camera 10. The field of view 68 of the distance sensor 28 is therefore located in front of the field of view or detection area 14 of the camera 10. The distance sensor 28 therefore perceives the object 54 somewhat early and its measurement data are already available at the time of recording.

[0055] In order to reduce the image data to be processed right from the start, the control and evaluation unit 42 divides its detection area 14 and the corresponding area of ​​the image sensor 20 into relevant and irrelevant parts based on the measurement data of the distance sensor 28. The relevant part corresponds to the region of interest (ROI). Figure 4, differently shaded sub-fields of view 70 , 72 are shown for this purpose, a darker, relevant sub-field of view 70 with the object 54 and two brighter, partially non-relevant sub-fields of view 72 without the object 54 .

[0056] Figure 5 The relevant division is shown in a schematic top view of image sensor 20. The pixels in the boldly framed rows of region of interest 74 correspond to the relevant subfield of view 70, and the remaining pixels of region of non-interest 76 correspond to the non-relevant subfield of view 72. The rows belonging to region of interest 74 are selected by control and evaluation unit 42 via configuration unit 22, and only the image data of these pixels are read out and further processed.

[0057] Reference again Figure 4 An exemplary evaluation is explained, with which the pixels of the region of interest 74 are found from the measurement data of the distance sensor 28. The distance sensor 28 detects an object 54 of height h for the first time at time t. The trigger time t1 is determined based on the relative position and posture of the distance sensor 28 relative to the image sensor 20 and the transport speed, at which the object 54 has moved into the detection region 14. The length of the object 54 is determined by the measurement data of the distance sensor 28 up to time t1. For example, the distance sensor 28 is operated with a repetition rate f. The length of the object 54 corresponds to the number of detections with this repetition rate. It should be remembered that the position and length in the transport direction can be directly converted into time by the transport speed. Therefore, when the corresponding position and posture of the camera 10 and the distance sensor 28 or the external geometry detection sensor 62 are known, the object length determined by means of the distance sensor 28 can be converted trigonometrically into the relevant image line on the image sensor 20, taking into account the height of the object.

[0058] During the transmission process, according to Figure 4 The relevant sub-area 70 is moved opposite to the conveying direction, or Figure 5 The region of interest 74 on the image sensor 20 in the image sensor 20 is moved downward. The first image trigger can be adjusted in time so that the front edge of the object 54 is located at the edge of the detection area 14 and is therefore located in the uppermost row of the image sensor 20. The image sensor 20 can be repeatedly and dynamically reconfigured to record the object 54 or other structures of interest, such as code areas 58, 60 on the object, multiple times.

[0059] Figure 6 It is shown that the pixels of the image sensor 20 can be alternatively divided into a region to be read and a region not to be read. Figure 5On the contrary, the boundary condition that the region of interest 74 only allows to include the entire row is cancelled here. As a result, the image data to be read and processed is further reduced. The configuration unit 22 is correspondingly more flexible and also allows to exclude pixels within the row. Preferably, this still means that there is no individual pixel selection, which will lead to excessive circuit consumption, but a rectangular sub-area as shown can be selected. In order to be able to reasonably select pixels within the row, thereby spanning the flow of objects 54, the distance sensor 28 should preferably provide a lateral resolution, thereby continuously providing a resolved contour of the object 54 along the conveying direction and transverse to the conveying direction.

[0060] Figure 7 The schematic cross-sectional view shows the situation in which two objects 54a-54b of different heights are located in the detection area 14. By adjusting the focus position 78 by means of the focus adjuster 18, the depth of field range surrounded by the DOF (Depth of Field) upper limit and the DOF lower limit 80a-80b can be moved. The depth of field range of the corresponding focus position 78 depends on various factors, in particular the receiving optical device 16, but also on the decoding method, for example, because the decisive factor for obtaining sufficient image clarity when reading the code is whether the code is readable. For example, the control and evaluation unit 42 can access a lookup table with depth of field ranges previously determined by simulation, modeling or experience.

[0061] The control and evaluation unit 42 therefore knows how the focus position 78 must be changed in order to clearly record one of the objects 54a-54b based on the measurement data of the distance sensor 28 and the information about the depth of field range of a focus position 78. As long as there is a focus position 78 whose depth of field range is suitable for all objects 54a-54b, the number of rows to be read can be increased for two or more objects 54 or a further region of interest 74 to be read can be created on the image sensor 20 with the aid of the configuration unit 22. For a plurality of objects 54a-54b, a single recording may be sufficient, wherein repeated recording is still possible, and individual recording for each object 54a-54b is also possible.

[0062] However, in Figure 7 In the case of , the height difference of the objects 54a-54b is so great that there is no focus position 78 in which both objects 54a-54b are located within the depth of field. The control and evaluation unit 42 must make a decision and focus first on the taller object 54b.

[0063] Figure 8A schematic top view of an image sensor 20 provided by means of a configuration unit 22 for this case is shown. Only the region of interest 74 corresponding to the higher, clearly recorded object 54b is read. Alternatively to a rectangular sub-region, the entire image row extending to the right and to the left can be configured and read. There is another region of interest 82 corresponding to the lower object 54a, and the control and evaluation unit 42 knows this other region of interest by evaluating the measurement data of the distance sensor 28. However, since it is not expected that there will be sufficiently clear image data in the other region of interest 82 anyway, these image data are processed like non-interested regions 76 and are not read. In the case of a small height difference in which the lower object 54a is still within the depth of field, if the configuration unit 22 provides this function, or the two regions of interest 74, 82 are surrounded by a common region of interest, then the two regions of interest 74, 82 can be configured and read.

[0064] Fig. 9 and Fig.10 Shown with Figure 7 and Figure 8 Complementary situation. Now, the focus position 78 and the associated depth of field range are adjusted to the lower object 54a. Accordingly, the image data of the lower object is read, and the image data of the higher object 54b and the areas between and beside the objects 54a-54b are discarded directly in the image sensor 20.

[0065] Thus, a first recording with focus position 78 can be generated for the higher object 54b and a second recording can be generated immediately after refocusing and thus adjusting focus position 78 to the corrected object 54a as long as the lower object 54a is still in detection area 14. The control and evaluation unit 42 even detects the described conflict situation in good time based on the measurement data of the distance sensor 28 and can plan in advance.

[0066] In an alternative embodiment, the focus position 78 is changed periodically (e.g., by a step function or oscillation). A plurality of recordings is produced such that the depth of field range generally covers the entire possible distance range, preferably excluding the transport plane itself, as long as completely flat objects 54 are not expected. The regions of interest 74 that are sharply recorded at the current focus position 78 are respectively configured according to the measurement data of the distance sensor 28. The focus position 78 thus respectively determines the region of interest 74. It is ensured that every structure is sharply recorded and that unsharp image data are not even read.

[0067] Thus, according to the invention, the advantages of a large image sensor 20 are achieved without triggering a data flood that is no longer manageable. At the same time, the problem of blurring the image data of a plurality of consecutive objects 54a-54b with greatly differing heights in a single large recording is solved. Thus, the image sensor 20 can be used to cover the stream of objects 54 individually, or at least a portion of the stream of objects that is as large as possible, at least with respect to its viewing angle (e.g. from above or from the side).

[0068] Conventionally, the preprocessing unit must read all image data in order to discard image data outside the area of ​​interest only when necessary. Preprocessing is usually performed on the fly during reading in a pipeline structure, so that reading and preprocessing are practically viewed the same in terms of time requirements. For a high-resolution image sensor 20, this requires, for example, a processing time of 25 ms and therefore limits the recording frequency or frame rate to 40 Hz. More complex image processing steps (e.g., decoding) will exacerbate this situation and the possible recording frequency will be further reduced. If two objects with very different heights are close to each other, it may be too late to make a second recording after refocusing. According to the present invention, the amount of image data is reduced from the beginning so that only the relevant image area is read. Reducing the data load is an advantage in itself because it can save resources or use resources more targeted. Therefore, the camera 10 selectively becomes cheaper or more efficient. In addition, the strict limitation of the recording frequency corresponding to the processing time of the complete image is eliminated. Therefore, the recording frequency can be increased flexibly as a whole or even according to the situation. Therefore, even in the case where the heights of two closely following objects 54a-54b are very different, a second recording can be made in time after refocusing, such as with regard to Figure 7-10 As described.

Claims

1. A camera device (10) for detecting a flow of an object (54) moving relative to the camera device (10), wherein: The camera device (10) has: an image sensor (20) having a plurality of light receiving elements in a matrix arrangement, which is used to record image data of the object (54); a geometry detection sensor for measuring the object (54); and a control and evaluation unit (42) designed to determine at least one region of interest (74) based on the measurement data of the geometry detection sensor in order to restrict the evaluation of the image data to the region of interest (74), It is characterized in that The image sensor (20) has a configuration unit (22) in order to be able to read only an adjustable part of the respectively recorded image data, and the control and evaluation unit (42) is designed to read only the part of the image data determined according to the region of interest (74) from the image sensor (20), and the control and evaluation unit (42) has a preprocessing unit (44) in order to read and preprocess the image data from the image sensor (20), wherein the preprocessing unit (44) is designed such that reading and preprocessing the complete image data recorded by the image sensor (20) requires a complete preprocessing time, and wherein the image sensor (20) is operated with a recording frequency that leaves a time between two recordings that is shorter than the complete preprocessing time.

2. The camera device (10) according to claim 1, in, The control and evaluation unit (42) is designed to adjust the at least one region of interest (74) and / or the portion of the image data read out between recordings.

3. The camera device (10) according to claim 1 or 2, in, The recording frequency is a flexible recording frequency.

4. The camera device (10) according to claim 1 or 2, in, The configuration unit (22) is designed to select an image line.

5. The camera device (10) according to claim 1 or 2, in, The configuration unit (22) is designed to select a rectangular sub-region.

6. The camera device (10) according to claim 1 or 2, in, The control and evaluation unit (42) is designed to read from the image sensor (20) only a portion of the image data recorded according to a region of interest (74) within a depth of field range (80a, 80b) of the image sensor (20).

7. The camera device (10) according to claim 6, in, The control and evaluation unit (42) is designed to determine a suitable depth of focus range (80a, 80b) for a region of interest (82) outside the depth of focus range (80a, 80b) and to refocus on the suitable depth of focus range (80a, 80b) for subsequent recording.

8. The camera device (10) according to claim 1 or 2, in, The control and evaluation unit (42) is designed to determine the at least one region of interest (74) as a function of a depth of field range of the image sensor (20).

9. The camera device (10) according to claim 1 or 2, in, The control and evaluation unit (42) is designed to detect a code area (58, 60) in the image data and to read out a code content of the code area.

10. The camera device (10) according to claim 1 or 2, in, The geometry detection sensor is designed as a distance sensor.

11. The camera device (10) according to claim 1 or 2, in, The geometry detection sensor (28) is integrated with the image sensor (20) into the camera device (10) or is arranged externally and upstream of the image sensor (20) counter to the flow in order to measure the object (54) before recording the image data.

12. The camera device (10) according to claim 1 or 2, The camera device has a speed sensor for determining the speed of the flow, and / or the control and evaluation unit (42) is designed to determine the speed of the flow based on measurement data of the geometry detection sensor and / or the image data.

13. The camera device (10) according to claim 1 or 2, The camera device is fixedly mounted on a transport device (52) which transports the object (54) in a transport direction (56).

14. The camera device (10) according to claim 1 or 2, The camera device has at least one image sensor (20) for recording the flow from above and / or at least one image sensor (20) for recording the flow from the side.

15. The camera device (10) according to claim 10, in, The distance sensor is a photoelectric distance sensor based on the principle of light flight time method.

16. A method for detecting the flow of a moving object (54), wherein: recording image data of the object (54) using an image sensor (20) having a plurality of light receiving elements in a matrix arrangement, measuring the object (54) using a geometry detection sensor, and determining at least one region of interest (74) based on the measurement data of the geometry detection sensor to restrict evaluation of the image data to the region of interest (74), It is characterized in that The image sensor (20) is configured to read only a portion of the image data determined according to the region of interest (74) from the image sensor (20), a preprocessing unit (44) reads the image data from the image sensor (20) and preprocesses it, wherein the preprocessing unit (44) is designed so that reading and preprocessing the complete image data recorded by the image sensor (20) requires a complete preprocessing time, and wherein the image sensor (20) is operated at a recording frequency that leaves a time between two recordings that is shorter than the complete preprocessing time.

Images