Incubator and method

Abstract

The invention relates to an incubator for living cell cultures, to a method for using an incubator, and to a system comprising an incubator. An image capturing system is used to determine whether the incubator is occupied using a mathematical comparison operation.

Description

[0001] Incubator and procedures

[0002] The invention relates to a system with an incubator for living cell cultures and a method for working with the system.

[0003] Such incubators are used in biological and medical laboratories to maintain cells in cell culture under controlled environmental conditions, thus enabling the growth of living cells in vitro. To this end, the temperature and gas composition, or rather the humidity, of the atmosphere inside an incubator chamber isolated from the environment are maintained at the desired values ​​by the incubator's equipment. Eukaryotic cells require CO2 incubators. The atmosphere is formed by air with a specific CO2 and O2 content and a specific humidity; a suitable temperature is often 37°C.

[0004] Cell growth critically depends on the constancy of the atmospheric conditions within the incubator. Disturbances in the incubator atmosphere can negatively impact cell growth. In an "ideally" equipped laboratory, each user would be provided with a separately accessible incubation chamber for each sample to be incubated. However, this is not realistic for reasons of cost-effectiveness. In laboratory practice, it is common for a single incubator (or a few incubators) with a single incubation chamber and one or more storage areas on one or more storage plates within the incubation chamber to be used by multiple users.

[0005] The frequency of opening the incubator chamber door, and thus the disruption of the regulated atmosphere, scales with the number of users and the number of samples incubated there. The intensity of the disruption also depends on the duration of the door opening. The longer a user needs to access the interior of the incubator chamber, the longer the door remains open. There are various incubator usage scenarios that may require increased access time due to complications:

[0006] Scenario A) Placing new objects in the incubator

[0007] If a user places one or more objects, particularly cell culture containers, in the incubator, they require a free storage space in a storage area. If no free storage spaces are accessible due to disorganized storage, the user needs time to create this storage space; the more carefully the user moves or rearranges objects already present in the incubator (inventory objects), possibly even documenting this in writing, the more time-consuming the process becomes. If it turns out that there is no longer sufficient storage space available in the incubator chamber, the user repeats the process in another compartment of the incubator chamber or in any spare incubator in the laboratory. This extends the period in which the incubator door is open, i.e. the duration of exposure of the chamber interior to the environment (exposure duration).

[0008] Scenario B) Testing cell cultures

[0009] If a user checks the cell cultures they previously placed in the incubator, e.g., to assess the quality of the cell medium or the growth status, they will first search for the cell culture container in question within the incubator. This extends the exposure time. The more carefully the user moves or rearranges existing objects, the more time-consuming the search process becomes.

[0010] Scenario C) Removing the objects from the incubator

[0011] In this case, too, the user must first search for the corresponding object. The time-delaying factors mentioned in B) apply.

[0012] Furthermore, the more often an incubator is opened, the higher the risk of contamination of the interior. There are also cases, for example in forensics or reproductive medicine, where the value of a single sample, particularly the cell(s) in a cell culture vessel, is considered much higher than, say, the value of the entire incubator, so that loss of the sample due to contamination must be avoided at all costs. In any case, the frequency of contamination increases the risk of work stoppage, increases costs, and requires additional maintenance. After an incubator is contaminated, the chamber must be cleaned and sterilized before the incubator can be used again. During this time, unless a replacement incubator is available, work with cell cultures is interrupted.

[0013] In laboratories, there is therefore a fundamental need to keep the period during which the incubator door is open as short as possible, and also to minimize the frequency of opening the incubator chamber. For this purpose, the subsequently published European patent application with application number 21153810 describes an incubator with an image capture system for recording the storage areas within an incubator and for outputting information on the occupancy status of the incubator. This functionality allows user access to be limited to what is absolutely necessary and the exposure time—that is, the period during which the incubator door is open—and thus the duration of exposure of the interior of the incubator chamber to the incubator's environment, can be kept to a minimum.

[0014] The present invention is a further development of this approach.

[0015] The object underlying the present invention is to provide a solution to make an incubator efficiently usable, in particular to keep the risk of contamination in an incubator low.

[0016] The invention solves this problem by the system according to claim 1 and the method according to claim 15. Preferred embodiments are in particular the subject matter of the subclaims.

[0017] The system according to the invention for monitoring the occupancy of storage space in at least one incubator for incubating living cell cultures comprises: at least one incubator for incubating living cell cultures, comprising an incubator chamber for receiving objects, in particular cell culture containers, which has opposing inner walls and a chamber opening for the supply and removal of the objects by a user, and which has at least one storage area for storing the objects, which extends between the opposing inner walls, an incubator door for closing the chamber opening, an image acquisition system, comprising

[0018] • a lighting device,

[0019] • at least one camera device and

[0020] • a data processing device, in particular with a data storage device, wherein the image acquisition system is designed to, in particular when the incubator door is closed or open,

[0021] • to illuminate at least one storage area extending between the interior walls by means of the lighting device,

[0022] • to capture at least one image of the at least one storage area extending between the inner walls in the form of image data by means of the camera device, and

[0023] • preferably storing the at least one image in the form of image data in the data storage device by means of the data processing device; wherein the system comprises a data storage device and a programmable data processing device which is programmed to:

[0024] • to determine at least one occupancy value from the image data that characterizes the occupancy of at least one storage area;

[0025] • to perform at least one mathematical comparison operation that compares the at least one occupancy value with at least one occupancy reference value;

[0026] • to record the result of the at least one mathematical comparison operation in at least one occupancy evaluation parameter; and

[0027] • to store the at least one occupancy evaluation parameter in the data storage device.

[0028] The image acquisition system makes it possible to take images in the incubator chamber under controlled conditions and in a reproducible manner, which can provide versatile information about the occupancy of the storage area. Since the incubator according to the invention provides information about the occupancy of the storage area in the form of image data, users are able to retrieve information about the occupancy status of the incubator before opening it. Providing this information reduces unnecessary opening of the incubator and makes its use more efficient.

[0029] The processing of information about the occupancy of the storage area can consist of providing the user with an image of the storage area, for example by displaying it on an incubator display. In this way, the user can immediately get an idea of ​​whether the occupancy of the storage area allows the storage of further objects, and whether the corresponding storage area contains an object that they have placed there and that is now to be checked or removed - as far as identifiable for the user - and where this object might be found. Furthermore, the information can be further processed by obtaining data about free storage space in this storage area using automated image analysis and communicating this to the user. The recognition of object classes and individual object characteristics is also made possible by the availability of image data.

[0030] By providing the automated occupancy assessment in the form of occupancy assessment data, the incubator becomes even more efficient. An occupancy value is, for example, the area of ​​a sub-area of ​​at least one storage area, whereby this sub-area can have a predetermined shape, preferably rectangular. Such a sub-area can be determined from the image data using image processing algorithms, in particular segmentation methods. The sub-area can correspond to a parking space.

[0031] An occupancy reference value is a predetermined area size stored in the data storage device that can be used for the mathematical comparison operation. The choice of the occupancy reference value depends on the application context.

[0032] A mathematical comparison operation is a computer algorithm that returns the result of a mathematical comparison "less than", "equal to", or "greater than". This result is stored as an occupancy evaluation parameter. The value is preferably Boolean, the Boolean data type used in programming to represent the logical values ​​true and false. However, the result of the comparison can also have more than two possible values, in particular a classification of a size value carried out using a multi-level comparison. This can be used to determine the area size class. This can be typical for certain laboratory sample containers or can be standardized. A sub-area whose area is smaller than that of a first size class can simultaneously fall into a second size class with a smaller area (than the area of ​​the first size class).Such a sub-area can be identified as suitable to serve as a location for a laboratory sample container falling into the second size class.

[0033] Such an occupancy assessment can in particular relate to the observation of one, in particular precisely one, storage area in the incubator, and can in particular relate to the assessment of the free storage space, also referred to as a storage location, for an individual laboratory container requiring storage space. In this case, the occupancy assessment can in particular reduce the risk that a user, who in particular individually assesses the image reproduction of a storage area on an incubator monitor with regard to occupancy, opens the incubator door due to an individual misjudgment in order to deposit a laboratory container in a supposedly free storage location, but this free storage location turns out to be too small for the laboratory container, thus proving unnecessary to open the door, which leads to an additional exposure time of the samples already incubated in the incubator, which increases the contamination risk for the incubator interior.

[0034] Such an occupancy assessment can, in particular, be based on the observation of one or more than one storage area, in particular several or all storage areas of an incubator, and can, in particular, be based on the assessment of free storage space for more than one laboratory container requiring storage space. The occupancy assessment can be used to assess the occupancy or free capacity of at least one storage area. The occupancy assessment can be used to determine the number of free storage spaces and / or the number of occupied storage spaces.

[0035] In order to assess whether one or more storage spaces are free (and / or occupied) in the (entire) free storage space of a storage area, it can be determined in particular whether areas of one or more area components of the free storage space are larger than a comparison area, namely in particular the desired storage area for one or more storage spaces, which can include at least one specified size and at least one specified shape. Images of the storage area can be used to determine the free / occupied storage space. Images of the storage area from a bird's eye view are particularly easy to evaluate. Free or occupied storage space can be easily determined using computer-aided image processing, in particular using algorithms for image segmentation. In this way, the utilization of the incubators can be tracked, in particular.Based on the detection of free / occupied individual parking spaces, capacity statistics can be determined. Laboratory managers can use this basis to optimize resource planning and utilization, in particular, to plan the processing of a large number of laboratory samples using more than one incubator or even more than one laboratory, each with one or more incubators or systems according to the invention. A system according to the invention is sometimes referred to below as "system" for short. The detection of a free / occupied parking space can be accompanied by the data processing device being programmed to assign a parking space ID in the form of parking space ID data to the detected parking space.The parking space ID data can contain information about the position (parking space position data) of the individual parking space in a storage area of ​​an incubator, whereby this storage area can also be identified by a storage area ID, and whereby the incubator can be identified by an incubator ID. The information about the storage area ID and / or the incubator ID can also be part(s) of the parking space ID data. Parking space ID data can make the use of one or more incubators in the system according to the invention even more efficient. In this way, the user receives not only information about the presence of a free storage space in the incubator and / or the number of specific free / occupied parking spaces, but also information about whether an individualized parking space is free or occupied. Parking space ID data can be part of occupancy status data, which will be described below.

[0036] The locations of an incubator differ from one another in certain characteristics. For example, laboratory containers placed near the incubator door are easier to access, but may be moved and shifted more frequently by other users than laboratory containers located further back in the chamber. Since a particular goal in laboratories is the reproducibility of experiments and work processes, the use of individualized locations helps to record and store precise information about the individual location of the biological sample in a laboratory container, making it available for further use, particularly for logging or statistical analysis. Advantageous planning can then, in particular, provide for the designation of individualized locations for users who exhibit certain class characteristics, e.g.Storage spaces near the rear wall are better protected from frequent access by other users than those near the door. Samples that require more frequent inspection are preferably positioned closer to the door than to the rear wall of the chamber. Using a positioning guidance system for the laboratory containers, the user can then be efficiently guided to position a laboratory container in a previously identified and predetermined free storage space. A positioning guidance system that operates, in particular, using lighting is described below. Individual space occupancy detection can be carried out as follows: A storage space is uniquely identified by space ID data. A free storage space is defined as an area of ​​a free storage location that meets at least one specified criterion: in a storage area, a continuous area with a specified shape, e.g.Rectangle, square, circle, and a size that exceeds a specified minimum area size SPmin and optionally does not exceed a specified maximum area size SPmax: this means in particular that a parking space has a specified size, but does not necessarily have to have a specified position in the storage area - although this is also possible and preferred. Classes of parking spaces can be recorded, which can in particular correspond to the sizes of commercially available cell culture vessels and their classes. An individual parking space can be uniquely identified by parking space ID data. Parking space ID data does not necessarily have to enable localization in the storage area. It can also simply be linked to the incubator (incubator ID) and / or the storage area (storage area ID) of the incubator.The prerequisite is preferably the identification of at least one free parking space, which can be determined according to the occupancy assessment by comparison with a reference value.

[0037] The preferred embodiments and further developments of the system according to the invention, or its occupancy evaluation, described here are preferably implemented by programming the data processing device to execute the corresponding functions. This circumstance of implementing a function will be explained representatively: to implement individual space occupancy detection, the data processing device is preferably programmed to assign space ID data to a parking space previously recorded by image analysis, which uniquely identifies this parking space.

[0038] Image processing segmentation techniques, which can be used to implement computer-aided image analysis to detect free or occupied storage locations, are generally known from many image processing applications. Segmentation is based on the image processing or image analysis of single image data or video image data. Such segmentation methods can be implemented using relatively simple tools such as suitable cameras and image processing algorithms. The theoretical principles and their use for the practical implementation of segmentation are well known (e.g., "Clustering Techniques for Image Segmentation", Siddiqui, Yahya, Springer International Publishing, 2021). Ready-to-use image processing algorithms for segmentation are also freely available (OpenCV.org) and well documented. OpenCV (English abbreviationOpenCV (OpenCV stands for Open Computer Vision) is a free program library (BSD license) with algorithms for image processing and computer vision. The OpenCV program library also includes functions, libraries, and interfaces for using function libraries for segmentation, in particular OpenCV, cv2, matplotlib, numpy, and scikit-image.

[0039] An occupancy assessment can in particular relate to the observation of one or more than one storage area, in particular several or all storage areas of one or more incubators of a system according to the invention, and can in particular relate to the assessment of the free storage space for more than one laboratory container requiring a storage area.

[0040] An occupancy assessment can in particular be related to the observation of one or more than one storage area, in particular several or all storage areas of one or more incubators of a system according to the invention. The occupancy assessment can compare a storage area corresponding to more than one storage location for a laboratory container with an occupancy reference value specified for this situation. The occupancy assessment can compare a (total) free storage location of one or more storage areas of one or more incubators of a system according to the invention with an occupancy reference value specified for this situation. In this case, the occupancy assessment does not in particular determine the number of available or occupied individual storage locations, even if this is fundamentally possible.Preferably, instead, information is obtained and provided as to whether free storage space falls below / exceeds a specified minimum / maximum value (“ / ” here means “and / or”). If, for example, the (total) free storage space is greater than a specified maximum value, information can be obtained as to whether a minimum utilization has been reached / undershot. If, for example, the free storage space is smaller than a specified minimum value, information can be obtained as to whether a minimum utilization of an incubator has been reached / exceeded. If more than one incubator is included in the analysis of the total free storage space of a system, information can be obtained as to whether utilization limits have been reached / undershot / exceeded in an incubator network. Such a network can be formed by the incubators in one or more laboratory rooms or in a laboratory.In this way, a laboratory manager can gain valuable information for resource planning / utilization in one or more laboratories.

[0041] In a preferred application, the occupancy assessment can be used to detect when a minimum capacity is reached or undershot in order to plan the cleaning of one or more incubators. Since the incubator must be empty during cleaning, the occupancy assessment helps to identify the best time to clean an incubator – the fewer remaining samples that need to be removed and moved, the less effort is required. In a network of incubators, the occupancy assessment helps to select the most suitable incubator for cleaning – the (total) free storage spaces of the individual incubators are compared, and the incubator with the largest free storage space is determined.

[0042] The system can be configured to generate incubation report data for an individual sample contained in an individual laboratory sample container (cell culture container). For this purpose, the sample / laboratory sample container is assigned—using programming and computer-assisted operations—preferably a sample ID or sample ID data that allows the sample / container to be uniquely identified. Since the laboratory sample container is placed in a slot of the incubator, the sample ID can be assigned a slot ID (slot ID data), which are then stored together in the incubation report (in the incubation report data). If the laboratory sample container is tracked using an object tracking system, the sample ID data can also be assigned movement history data, which can include, in particular, time-dependent information about the movement history.Assuming that a laboratory sample container essentially always remains in its predetermined location, object tracking is not absolutely necessary to obtain movement data for an individual laboratory sample container. Thus, by observing the location and monitoring image changes in this location over time, a measure of the (unwanted) movement of the laboratory sample container can be obtained, which can then be stored as movement data. This data can also be assigned to the sample ID and stored in the incubation report data. In particular, time-dependent sensor data from the incubator's sensors, relating to gas concentrations inside the chamber (CO2, O2, N2, H2O) and / or the chamber's internal temperature and / or data on the number and duration of door openings, can be assigned to the sample ID data to obtain incubation report data.Using this incubation report data, sample treatment can be documented in detail and the basis for high reproducibility of experiments and sample processing can be created.

[0043] The incubator can have a camera device, in particular the camera device of the image capture system, by means of which at least one image of at least one storage location of at least one storage area can be captured. In particular, image documentation to be created for an individual storage location identified by location ID data can be captured and recorded and stored depending on the location ID data and / or sample ID data. The image documentation can be video documentation. The activity of the camera device and the image capture can be controlled depending on a door sensor in order to only record when movement of the laboratory sample container is at least possible. The image documentation can be saved as part of the incubation report data.

[0044] The system according to the invention can comprise a positioning guidance system. The positioning guidance system preferably comprises a lighting device by means of which an area or spot in the storage area, in particular a storage space, can be specifically illuminated. The lighting device can also be formed by the lighting device that is part of the image capture system. The lighting device is preferably arranged within the incubator chamber such that several areas, in particular all, in particular substantially all, areas of the at least one storage area can be illuminated. Light is preferably radiated onto the storage area and from there reflected and / or scattered toward the user, i.e., toward the door opening (incubator door).

[0045] A data processing device of the incubator, or the data processing apparatus, is preferably programmed to control the activity, i.e. switching on / off, and / or the illumination color, and / or the illumination intensity, and / or the temporal sequence of a pulsed activity of the illumination device.

[0046] A data processing device of the incubator, or the data processing apparatus, is preferably programmed to set the target of the illumination as a function of sample ID data and / or location ID data and to apply directed illumination to this target or location. The illumination device is preferably configured to set the target of the illumination and to apply directed illumination to this target or location. For this purpose, the illumination device can have an electrically controllable movement device which carries at least one light source, in particular an electrically operated light source (e.g., LED) or at least one light-emitting optical fiber, and which is configured for the targeted illumination of various locations.

[0047] The lighting device can be arranged on or at a storage area, in particular at / on a parking space, so that the emitted light falls from there into the eye of the observer.

[0048] The system according to the invention can comprise a user guidance system. The user guidance system can comprise a positioning guidance system. The user guidance system is configured to support the positioning of laboratory sample containers on and / or between locations of the incubator by illuminating at least one location and / or a laboratory sample container placed on a location according to a predetermined schedule (schedule data). For this purpose, a data processing device of the incubator, or the data processing apparatus, is preferably programmed to

[0049] * to enable the user to select or enter a sample ID using a user interface device,

[0050] * and / or to assign slot ID data, also referred to as reserved slot ID data, to this sample ID according to the predetermined schedule, which in particular determines a time period for incubation of the sample,

[0051] * and / or depending on this sample ID and the reserved parking space ID data, to control the lighting of the parking space reserved by predetermination by means of the lighting device according to a predetermined schedule.

[0052] In particular, parking space position data can also be used, which, similar to object position data, defines the position of a (free or occupied) parking space in a storage area. The parking space position data is preferably also part of the parking space ID data.

[0053] A storage area preferably has a predetermined size A2 (e.g. measured in square centimeters) and / or a predetermined dimension M, which in particular also contains the information about the shape, wherein the rectangular shape can be assumed as the default shape. For example, microtiter plates and cell culture flasks have a substantially rectangular storage area, while Petri dishes and other flask shapes can have a circular storage area. Preferably, different sizes A2 and dimensions are provided, which correspond to certain commercially available objects and which can be stored in the data storage device for comparison or automated image evaluation as comparison values ​​or occupancy reference values. For practical handling, some free space is also required, e.g. an edge of a width d (e.g.2 cm), this is added to determine the footprint required for a specific cell culture vessel. According to the ANSI / SBS 1-2004 standard, an SBS standard microtiter plate has the following dimensions (length x width, "outside dimension of the base footprint"): {127.76 mm ± 0.5 mm} x {85.48 mm ± 0.5 mm}. Including a rim d, this would result in approximately M=170x126} and A2=110.08 cm. 2 (without border) or A2=214cm 2 (with border).

[0054] Example pairings A2, M of commercial products are (without rim): Roth Selection cell culture flask with 175 cm 2 Growth area: M={20.5cm x 12.0cm}, A2=246cm 2

[0055] Roth Selection cell culture flask with 75 cm 2 Growth area: M={15.0 cm x 8.5 cm}, A2=127.5 cm 2

[0056] Roth Selection cell culture flask with 25 cm 2 Growth area: M={9.0cm x 5.0cm}, A2=45cm 2

[0057] Container tray Tissue Culture Tray, Mitchell Plastics™ M={40 cm x 19.0 cm}, A2=760 cm 2

[0058] The data storage device and / or the programmable data processing device of the system according to the invention can be components of the incubator, i.e. in particular can be installed within the incubator housing in which the incubator chamber is also arranged, but they can also be components of an external data processing device, in particular a computer, server, or other laboratory device which is connected to the incubator for the purpose of exchanging data.

[0059] An incubator is a laboratory device or a laboratory incubator. An incubator refers, in particular, to a laboratory device with an incubator chamber whose atmosphere can be regulated or is regulated by the incubator to a predetermined target temperature. In particular, it is a laboratory device with which controlled climatic conditions can be created and maintained for various biological development and growth processes. The incubator can be or contain a shaking incubator, i.e., an incubator with a movement device for moving objects arranged in the incubator chamber, and can be a microbial incubator (also without CO2). The incubator can, in particular, be designed as a cell cultivation device. The incubator serves, in particular, to create and maintain a microclimate with regulated gas, humidity, and / or temperature conditions in the incubator chamber, whereby this treatment can be time-dependent.The laboratory incubator, in particular a treatment device of the laboratory incubator, can in particular have a timer, in particular a timer, a heating / cooling device and preferably a setting for regulating an exchange gas supplied to the incubator chamber, a setting device for the composition of the gas in the incubator chamber of the incubator, in particular for adjusting the CO2 and / or the O2 and / or the N2 content of the gas and / or a setting device for adjusting the air humidity in the incubator chamber of the incubator. The incubator, in particular a treatment device of the incubator, has in particular the incubator chamber, further preferably a control device with at least one control circuit, to which the at least one heating / cooling device is assigned as an actuator and at least one temperature measuring device is assigned as a measuring element. The temperature in the incubator can be regulated by means of the control device.Depending on the design, the humidity can also be regulated via this. A water-filled tray in the incubator chamber can be heated or cooled to adjust the humidity through evaporation. Alternatively and / or additionally, a water evaporator can be provided as a component of the incubator, by means of which the humidity in the atmosphere of the incubator chamber is adjusted. CO2 incubators are used in particular for the cultivation of animal or human cells. Incubators can have turning devices for turning the at least one cell culture container and / or a shaking device for shaking or moving the at least one cell culture container. The incubator according to the invention is in particular not a bioreactor or fermenter.

[0060] The incubator can have a sensor device. A sensor device has, in particular, at least one temperature sensor, preferably a plurality of temperature sensors. A temperature sensor can be, for example, a Pt 100 or Pt 1000 temperature sensor. A sensor device preferably has a sensor for determining a relative gas concentration, in particular for determining the CO2 and / or O2 and / or N2 content. A sensor device preferably has a sensor for determining the relative humidity. An incubator preferably has one or a single incubator chamber. This can be divided into compartments. Compartments can be separated by—in particular perforated—bearing plates, whereby, in particular, gas exchange between the compartments is enabled.A support plate, in particular its lower side, can be configured to hold the camera device and can in particular have a holder for the camera device. A support plate, in particular its lower side, can be configured to hold the lighting device and can in particular have a holder for the lighting device. The lighting device or its holder can, however, also be arranged or mounted at another location in the incubator chamber, e.g., on an inner side wall of the incubator chamber, or on the floor or ceiling wall. A holder for the lighting device can have a rail system, a robot arm controlled by the control device, and / or magnet(s).

[0061] The incubator chamber has chamber walls or inner chamber walls and precisely one or at least one chamber opening through which the objects or cell culture containers can be placed and removed from the interior of the incubator chamber. This chamber opening can be closed by a closure element that is movably connected to the incubator chamber, in particular an incubator door that is movably mounted on the incubator chamber by means of a hinge, in particular one or more chamber doors. An incubator can have one or more inner doors, which can in particular be transparent, and can have an outer door, which is in particular opaque, which thermally insulates in particular the incubator chamber and optionally at least one inner incubator door, which closes or opens the chamber opening, from the environment. Images are preferably taken by the image acquisition system when the incubator door is closed oroutside door so that ambient light does not influence the lighting of the storage area, which is preferably provided exclusively by the lighting device. This leads to images that are particularly reproducible, easily comparable and easy to evaluate using image processing algorithms. However, it is also possible to create the images with the incubator door open. When the chamber opening is closed, the interior of the incubator chamber is preferably insulated from the environment in such a way that a desired temperature or atmosphere controlled by the incubator can be set, in particular regulated. When the chamber opening is open, gas exchange between the environment of the incubator and the interior of the incubator chamber is possible via this opening. The chamber opening is typically located in a front wall surrounding the chamber opening.

[0062] The incubator chamber preferably has a plurality of walls or inner wall surfaces, which can in particular be connected to one another in one piece and in particular without edges. The walls or inner wall surfaces are preferably essentially planar, but can also all or partly have a curved shape. The incubator chamber is preferably cuboid-shaped, but can also have other shapes, e.g. spherical, ellipsoidal, polyhedral. The walls or inner wall surfaces are preferably made of a low-corrosion material, in particular stainless steel, copper, brass, or a plastic, in particular a composite plastic. This makes cleaning / disinfecting the interior of the chamber easier. Independent of the chamber opening, which is used for loading / removing objects or cell culture containers, the incubator chamber can have at least one port for passing through a correspondingly dimensioned device ora cable connection from the inside of the incubator chamber to its outside or to the surroundings of the incubator.

[0063] The surfaces of the incubator's interior walls are preferably non-glossy or non-reflective, in particular by using a matte surface. The surface of the incubator's interior wall can be matte-finished by a surface treatment. The surface treatment can, in particular, be sanding with an abrasive, which can, in particular, have a specific grain size. The surface treatment can, in particular, be irradiation with a blasting agent, in particular sand or glass beads, in particular using compressed air, which can, in particular, have a specific grain size or a characteristic particle diameter. This can prevent or reduce disruptive reflections in an image recording.

[0064] A typical size of the interior of an incubator chamber is between 50 and 400 liters.

[0065] The incubator can have exactly one incubator chamber, but can also have multiple incubator chambers, whose atmosphere (temperature, relative gas concentration, humidity) can be individually or collectively adjustable. An incubator can have multiple incubator chambers, each of which can have its own chamber opening and its own chamber door for closing the chamber opening.

[0066] The incubator may have a housing that partially or completely surrounds the incubator chamber. The housing may be substantially cuboid-shaped and, in particular, may be designed such that the incubator is stackable.

[0067] A storage area of ​​the incubator is realized in particular by a storage plate, in particular a shelf insert and / or a moving platform, which can in particular consist of stainless steel, copper or similar or comprise this material. A storage plate serves as a base plate, in particular as an intermediate base plate. The storage plate can be removable from the incubator chamber (“storage plate insert”) or can be permanently connected to it. The incubator chamber can have holding sections or a holding frame for holding one or more storage plate inserts or insertable instruments. A storage plate can be configured on its underside to hold a camera, in particular have a holder for this camera. Alternatively or additionally, at least one of the inner walls of the incubator chamber can be configured to hold one or more storage plate inserts or insertable instruments, in particular the at least one camera.For this purpose, a holding structure integrated into the wall can be provided, in particular one or more projections, grooves or webs. A bearing plate increases the available bearing surface in the incubator chamber. Preferably, substantially all surfaces or at least one surface of the at least one bearing plate are non-glossy or non-reflective, in particular by using a matte surface. The surface of the incubator inner wall can be matte-finished by a surface treatment. The surface treatment can in particular be grinding with an abrasive, which can in particular have a specific grain size. The surface treatment can in particular be irradiation with a blasting agent, in particular sand or glass beads, in particular using compressed air, which can in particular have a specific grain size or a characteristic particle diameter. This can prevent or reduce disruptive reflections in an image recording.be reduced.

[0068] A support frame for the at least one support plate is also preferably made of a non-corrosive material, preferably stainless steel. The support frame is preferably designed as a standing object, having at least one base section that rests on the floor wall of the incubator chamber. However, it can also be supported on the side walls of the incubator chamber and / or suspended from the ceiling wall of the incubator chamber.

[0069] A bearing plate preferably extends - and in particular substantially completely - over a horizontal cross-section of the incubator chamber.

[0070] An incubator preferably has at least two storage plates arranged one above the other. The volume area between two storage plates, or between a floor wall of the incubator chamber and a lowest storage plate or between a ceiling wall of the incubator chamber and a topmost storage plate can be referred to as a storage compartment. A storage compartment as a whole can be regarded as a storage area. The surface of a storage plate suitable for storage can be regarded as a storage area. The height of a storage compartment is preferably dimensioned such that an object of a certain maximum height (measured perpendicular to the planar surface of a storage plate) or a stack of objects of a certain maximum height of the stack can be placed on the storage plate. The maximum height can in particular essentially correspond to the distance between two storage plates.

[0071] The distance between two bearing plates or the maximum height is in particular between 5 cm and 70 cm, preferably between 5 cm and 65 cm, preferably between 5 cm and 60 cm, preferably between 5 cm and 50 cm, preferably between 10 cm and 30 cm, preferably between 10 cm and 20 cm, preferably between 12 cm and 18 cm. The maximum height can in particular be up to 150 cm.

[0072] The distance between two bearing plates can preferably be selected by the user using a variable holding device for bearing plates.

[0073] An instrument that can be inserted into the interior of the incubator chamber, in particular a camera, can be designed as a module and enables automated observations to be carried out inside, preferably even when the incubator door is closed.

[0074] The camera device, or its at least one camera, is preferably arranged on a support plate, preferably arranged or arrangeable beneath a support plate and in particular fastened or attachable thereto. Preferably, at least one camera is mounted or mountable on the underside of a support plate, in particular in a geometric center of the underside, in particular at the intersection point of the diagonals of a rectangular underside.

[0075] Preferably, one or more cameras are mounted or mountable on the underside of a shelf insert in the incubator chamber, or on an underside of the upper inner wall (ceiling wall) of the incubator chamber, preferably vertically above the geometric center of the storage area monitored by the camera. One or more cameras can also be arranged or attached or can be arranged / attached to inner side walls of the incubator chamber or to a holding frame. Preferably, the at least one camera is set up and arranged such that it has an angle of view, generally measured in the image diagonal or alternatively also in the image vertical or horizontal, of between 90° and 210°, preferably between 120° and 180°, and preferably between 160° and 180°.

[0076] A camera of the image acquisition system can have a wide-angle lens, in particular a fisheye lens, whose angle of view in the image diagonal can be between 120° and 230°.

[0077] Preferably, exactly one camera is provided on the underside of a bearing plate, which in particular has one of the above-mentioned angles of view.

[0078] The field of view (FOV) of a camera can be defined, in particular, to have a specific angle of view, e.g., an angle of view corresponding to one of the ranges defined above, resulting in the display of image content dependent on this angle of view, or can be defined by an angle of view measured vertically across the image and an angle of view measured horizontally across the image. The aspect ratio of the image can, in particular, be one of the following formats: 4:3, 3:2, 16:9, or 1:1.

[0079] Preferably, at least one camera - preferably exactly one camera - is arranged on the underside of a storage plate, the field of view of which preferably covers more than X% of the storage area of ​​a storage plate lying in the field of view of the at least one camera. X is, in each case, preferably 20, 30, 40, 50, 60, 70, 80, 90, 100. In other words: the image recorded by this - exactly one or at least one - camera preferably shows more than X% of the storage area of ​​a storage plate lying in the field of view of the at least one camera. For example, several cameras can be provided which together cover the entire storage area, i.e. 100% of the storage area, or the portion X. Preferably, exactly one camera is provided which covers the entire storage area or the portion X. The larger or more complete the field of view, the more reliably and efficiently the storage area can be imaged and the image evaluated.

[0080] Preferably, at least one camera - preferably exactly one camera - is arranged on the underside of a storage plate, the field of view of which lies in a compartment of the incubator and which, preferably in addition to the storage area (part or the entire storage area) on the storage plate, preferably captures more than Y% of the wall area of ​​a compartment wall bounding the compartment, which is formed by an inner wall section of the inner wall of the incubator. Y is, in each case, preferably 20, 30, 40, 50, 60, 70, 80, 90, 100. In other words: the image recorded by this - exactly one or at least one - camera preferably shows more than Y% of the wall area of ​​one (or all) compartment walls bounding the compartment. For example, several cameras can be provided which together capture the entire area of ​​all compartment walls, i.e. 100% of the inner wall area of ​​a compartment, or the proportion Y.Preferably, exactly one camera is provided that captures the entire surface of all compartment walls, or the portion Y. Due to the correspondingly large field of view, objects or object stacks positioned at an edge of the planar storage area of ​​the storage plate can also be captured.

[0081] The data processing device is preferably programmed to automatically crop the image recorded by the camera so that an effective angle of view results which is smaller than the angle of view specified by the camera or so that an effective field of view results which is smaller than the field of view specified by the camera.

[0082] The incubator camera is particularly suitable for operating reliably in the respective incubator atmosphere over a period of several months or years, or for operating reliably during the service life measured under standard conditions (room temperature). Not every camera is suitable for operating in an incubator atmosphere. One possible commercially available camera is the 5MP wide-angle camera for Raspberry Pi, www.joy-it.net, available from Conrad Electronic SE, Germany, and / or another camera in combination with a wide-angle lens, e.g. the commercially available “Industrial lens HAL 250 2.3”, Entaniya Co., Ltd., Japan. Alternatively, a casing device can be provided for the at least one camera in order to shield or insulate it from the incubator atmosphere, wherein this casing device in particular has transparent areas or a transparent window.is transparent to allow image capture through the transparent area.

[0083] The camera device preferably has at least one optical filter with which the light incident on the camera is filtered. This allows the quality of the image recording to be optimized, in particular with regard to downstream digital image processing and evaluation. The camera device preferably has at least one polarizing filter with which the light incident on the camera is filtered. This allows reflections in the image recording to be reduced, which can arise from reflections of the light from the lighting device on objects in the storage area, elements of the storage area or the incubator chamber and / or inner walls of the incubator chamber, in particular with regard to downstream digital image processing and evaluation. The polarizing filter is preferably a circular polarizing filter, but can also be linear. Undesired reflections from smooth, non-metallic surfaces (e.g.The reflected light waves (e.g., the plastic surface of cell culture containers) can be suppressed using a polarizing filter. Light with perpendicular polarization is reflected noticeably more strongly from non-metallic surfaces, particularly when the exit angle to the surface is approximately 30° to 40°, i.e., close to the Brewster angle. If the polarizing filter is suitably aligned, the reflected light waves are suppressed, so that the unpolarized background is not overshadowed by the reflections. When using polarization, the objects to be recorded, particularly cell culture containers, are preferably arranged—especially in a direct line—between the illumination device and the at least one camera.

[0084] Preferably, the camera device has a first polarizing filter and the illumination device a second polarizing filter, wherein in particular the first and second polarizing filters are used rotated relative to one another. In this way, a portion of the light from the illumination device is initially blocked out by the polarizing filter in front of it. The polarizing filter in front of the camera is adjusted with respect to the polarizing filter of the illumination device or rotated relative to the incident light so that it now also blocks out the other portion of the light emitted by the illumination device. Ideally, only diffuse light remains. As a result, reflections, including those from metallic surfaces, are reduced or completely eliminated. This is particularly advantageous with regard to downstream digital image processing and analysis, in particular with regard to outline recognition of objects using image processing.

[0085] The illumination device preferably has at least one light source, in particular an LED. The illumination device preferably has at least two or more light sources, each with a different emission spectrum, i.e., different colors, e.g., red, green, blue. In this way, the image quality can be optimized, particularly with regard to downstream digital image processing and analysis, especially with regard to object outline detection using image processing.

[0086] The illumination device can have at least one light source whose emitted light has wavelengths greater than that of visible light or consists thereof, in particular whose emission spectrum lies in the infrared range with in particular a wavelength between 780 nm and 1 mm, in particular in the near infrared (780 nm to 3000 nm) or mid-infrared (3000 nm to 50000 nm) or contains such an infrared range. In this case, the camera device has at least one camera or one camera sensor that is suitable for detecting corresponding light, in particular infrared light. The illumination device can have at least one light source whose emitted light has wavelengths less than that of visible light or consists thereof. In this case, the camera device has at least one camera or one camera sensor that is suitable for detecting corresponding light.Preferably, at least two light sources are arranged at a distance from each other. This allows the field of view of the camera(s) to be illuminated more homogeneously and reduces the intensity of individual, light-direction-dependent reflection areas. This is particularly advantageous with regard to downstream digital image processing and analysis, especially with regard to object outline detection using image processing.

[0087] The lighting device preferably comprises at least one light source that is arranged or attached or can be arranged / attached to an underside of a support plate. The lighting device preferably comprises at least two or more light sources that are arranged or attached or can be arranged / attached at different positions along an underside of a support plate.

[0088] The illumination device preferably has at least one optical filter through which the light emitted by the illumination device is partially or completely filtered. The optical filter can be a polarizing filter, which can be matched in particular to a polarizing filter of the camera device, in particular to achieve the optimal desired filtering effect.

[0089] The lighting device preferably has at least one light diffuser, which allows the lighting device to emit diffused light. A light diffuser can be, for example, a cloudy Plexiglas plate or comprise such a plate. The light diffuser can reduce or prevent harsh shadows and reflections, which is particularly advantageous with regard to downstream digital image processing and analysis.

[0090] The image capture device, in particular the camera device and / or the illumination device, preferably has at least one diaphragm, preferably an aperture diaphragm, which in particular has a variable diameter, e.g., an iris diaphragm, to control the light flux of the illumination or the light flux entering the camera. The image capture device, in particular the camera device and / or the illumination device, preferably has at least one or more optical lenses.

[0091] Preferably, the image capture device, in particular the camera device and / or the illumination device, has at least one lens, preferably a wide-angle lens, preferably a wide-angle fisheye lens.

[0092] The image capture device preferably has a timer. The data processing device is preferably programmed to activate at least one or more light sources of the lighting device in a predetermined time sequence and, in particular, to deactivate them again after a predetermined activity time and / or to activate all or more light sources simultaneously. The data processing device is preferably programmed to capture multiple images of the storage area, each captured consecutively and, in particular, synchronously with the activity times of the lighting.

[0093] Preferably, the image acquisition system is configured to capture and store the time of entry of an object into the incubator chamber and / or the time of removal of an object from the incubator chamber.

[0094] Preferably, the image acquisition system is configured to

[0095] • to illuminate at least one or two objects located in this storage area using the lighting device,

[0096] • to capture an image of at least one or two objects in this storage area using the camera device, and

[0097] • storing the image of at least one or two objects in the data storage device in the form of image data using the data processing device. This enables various possible uses of the image data, in particular: distinguishing between objects in the storage area, in particular: assigning different identification data to the first and second objects; counting the objects; recognizing the object class; analyzing, storing, and recognizing individual characteristics; tracking objects during movement; recognizing and storing the time of entry or removal of the object.

[0098] Preferably, the data processing device is programmed to

[0099] • to distinguish between the first object and the second object shown in the image by evaluating the image data, in particular: to assign different identification data to the first and second object; to count the objects; to recognize the object class; to analyze, store, and recognize individual characteristics; to track objects during movement, in particular to capture the outlines of the first and second object in the image by means of image processing algorithms, and

[0100] • in particular to store information about the first and second object, in particular the bounding boxes and / or outlines of the first and second object, in the form of object data in the data storage device.

[0101] Preferably, in a preferred embodiment, the illumination device is configured, and in particular the data processing device is programmed, for the illumination device to be operated in at least two different illumination modes, and the image acquisition system is preferably configured, and in particular the data processing device is programmed,

[0102] • to illuminate the storage area of ​​the incubator chamber by means of the lighting device i) initially in a first lighting mode and ii) then in a different second lighting mode,

[0103] • to capture at least one image of the storage area during illumination using both the first and second illumination modes by means of the camera device, and

[0104] • provide the at least one image in the form of image data containing combined image information acquired during both the first and second illumination modes, wherein the data processing device is programmed to execute an image analysis program that extracts the combined image information from the image data. This embodiment makes it possible, in particular, to improve or optimize the quality of the image acquisition of the storage area, which is particularly advantageous with regard to subsequent image processing, in particular analysis, particularly for capturing the bounding box(es) and / or the outline(s) of one or more objects or cell culture containers.

[0105] A first illumination mode and a second illumination mode can differ, in particular, in that different light sources are used, and / or light sources arranged at different positions, and / or different exposure times of the light sources, and / or different emission spectra or light colors, and / or different light intensities. The different illumination modes can, in particular, improve or optimize the quality of image capture of the storage area, which is particularly advantageous with regard to downstream image processing, in particular image analysis, especially for capturing the outline(s) of one or more objects or cell culture containers.

[0106] Preferably, the at least one image of the storage area contains at least a first image of the storage area and a second image of the storage area that is different therefrom, wherein the first image is captured in the first illumination mode and the second image is captured in the second illumination mode, and the first image is provided in the form of first image data and the second image is provided in the form of second image data, wherein in particular the data processing device and / or the image evaluation program are programmed such that

[0107] • the first image data and the second image data are combined to obtain combined image data, which is created in particular by adding and / or averaging first and second image data, and

[0108] • the combined information is obtained from the combined image data. This embodiment allows, in particular, the quality of the image capture of the storage area to be improved or optimized, which is particularly advantageous with regard to subsequent image processing, in particular image analysis, especially for detecting the positions of one or more objects or cell culture containers in the image of the storage area using bounding box algorithms.

[0109] A typical program code for image-processing object tracking, which is preferably used, is based on the evaluation of the temporal sequence of images. A typical program code for object tracking uses a "bounding box" as an output format to identify an object in an image, to determine its collision boundaries, and, in particular, to localize it. In digital image processing, the "bounding box" refers to the coordinates of the rectangular frame that largely or completely encloses an object shown in the digital image. The use of bounding boxes in object tracking makes it more efficient, as image evaluation using such a numerical tool requires fewer computational steps and thus less computing power, especially compared to algorithms for object outline detection.Furthermore, the corresponding algorithms can be executed efficiently and cost-effectively using specialized graphics processing units (GPUs). Suitable application programming interfaces (APIs) for object tracking using bounding boxes are available in the OpenCV library under the names BOOSTING, CSRT, GOTURN, KCF, MEDIANFLOW, MOSSE, MIL, and TLD. Accordingly, OpenCV provides libraries ("MultiTracker") for the simultaneous tracking of multiple objects ("multiple object tracking"). As an alternative, deep learning algorithms for multi-object tracking (MOT) based on the "tracking-by-detection" principle are known.

[0110] However, it is also possible and preferred that for object tracking, the contour of the object to be tracked is determined in the image, and in particular the separation of object (foreground) and background by background subtraction.

[0111] Preferably, a plurality (N>=10) of illumination modes are used to either capture one image or capture multiple images, which then provide a combined image with combined image information in the form of combined image data. Preferably, 2<=N<=300, preferably 10<=N<=300, preferably 100<=N<=300. In this case, N<=500 or N<=1000 can preferably be used. Preferably, the at least one image of the storage area contains a multiply exposed image of the storage area, wherein the image acquisition system is particularly configured to

[0112] * to expose and capture the image of the storage area during illumination using both the first and second illumination modes by means of the camera device, and

[0113] * to provide the multi-exposed image in the form of image data.

[0114] Preferably, the at least one image contains information about objects arranged in the storage area, in particular information, optionally,

[0115] * about the positions of the objects in the storage area

[0116] * about the outer contours of the objects,

[0117] * over the area of ​​the objects, measured in a plane parallel to a planar surface of the storage area,

[0118] *The area of ​​the storage area not occupied by objects, measured in a plane parallel to a planar surface of the storage area.

[0119] Preferably, the lighting device comprises at least a first and a second light source, which are operated differently in the first and second lighting modes, wherein in particular the first and a second light source are arranged at a distance above a storage surface of the storage area, wherein in particular the first and second light source are arranged offset in a plane that lies parallel to a planar storage surface of the storage area, wherein in particular the storage area has a planar storage surface, wherein the first light source is arranged vertically above a first half of the storage surface and the second light source is arranged vertically above a second half of the storage surface, wherein in particular the lighting device comprises an LED strip with a plurality of LED light sources, which is arranged in a plane that lies parallel to a planar storage surface of the storage area, in particular in a meandering course,a spiral course, in particular in a course that is at least partially linear, wherein in particular the image capture system has a, in particular programmable, electronic control device that is set up or programmed such that o during an illumination phase of the first illumination mode, the first light source is operated differently than during an illumination phase of the second illumination mode, and / or o during an illumination phase of the first illumination mode, the second light source is operated differently than during an illumination phase of the second illumination mode, in particular that o during an illumination phase of the first illumination mode, the first light source is active and during an illumination phase of the second illumination mode, it is less active (i.e., emits with lower intensity) or is inactive,and / or o during a lighting phase of the first lighting mode, the second light source is less active or inactive and is active during a lighting phase of the second lighting mode, in particular that o during a lighting phase of the first lighting mode, the first light source is operated with a different emission spectrum than during a lighting phase of the second lighting mode, and / or o during a lighting phase of the first lighting mode, the second light source is operated with a different emission spectrum than during a lighting phase of the second lighting mode.,

[0120] Preferably, in particular, the at least one camera is arranged at a distance perpendicularly above a storage surface of the storage area, wherein the at least one camera preferably has a wide-angle lens, in particular a wide-angle or fisheye lens, wherein preferably exactly one camera is provided, which is arranged at a distance perpendicularly above a center of the storage surface of the storage area. Preferably, the image acquisition system is a modular component of the incubator, namely one that can be selectively inserted by the user, wherein in particular the incubator has a control device and a temperature control device for regulating the temperature in the interior of the incubator chamber, wherein the image acquisition system has another control device that is configured to control the image acquisition system, in particular in that this other control device contains the data processing device of the image acquisition system.Such a modular embodiment of an incubator with an image acquisition system preferably also has a data interface to the incubator so that, for example, image data can be displayed on the incubator display.

[0121] Preferably, the incubator in particular comprises a control device and a temperature control device for regulating the temperature in the interior of the incubator chamber, wherein this control device is particularly configured to control the image acquisition system, in particular by including the data processing device of the image acquisition system. This is the integral embodiment of an incubator with an image acquisition system.

[0122] Preferably, the incubator has a display and is preferably configured or programmed to display on the display preferably the image, and / or preferably image information extracted from the at least one image, and / or preferably an image of the storage area containing the combined image information.

[0123] Preferably, the image acquisition system is: an object acquisition system in which the data processing device is programmed to capture at least one object arranged in the storage area by means of the image evaluation program during the image acquisition of the at least one image, in particular an object recognition system for recognizing the object based on individual properties and / or for recognizing an object class based on object-specific class properties, in particular an object tracking system for tracking position changes of the at least one object in the storage area starting from a start position in order to capture its end position.

[0124] "Downward" refers to the direction of gravity, "upward" the opposite direction. "Vertical" means "along the gravitational vector," and "horizontal" means perpendicular to the vertical or in a planar plane perpendicular to the vertical. In intended use, incubators are arranged so that the top surfaces of the planar support plates are horizontal.

[0125] The incubator preferably has a treatment device for treating the at least one object, in particular a cell culture container. The term "treatment" means, in particular, that an object, in particular a cell culture or a cell culture container, is moved, and / or transported, and / or examined, and / or modified, in particular physically, chemically, biochemically, or in another way.

[0126] A treatment device can be a movement device by means of which the cell medium in at least one cell culture container is kept in motion, preferably via a movement program that is controlled by the control program. A movement device can be a shaking or pivoting device. A movement device preferably has a support device, in particular a plate, on which one or more cell culture containers are placed and / or fixed. A movement device preferably has a drive device, in particular in the case of a shaking device, for example an oscillator drive, in particular in combination with an eccentric, by means of which the desired movement program is implemented. A treatment device can be a pivoting device by means of which at least one cell culture container is pivoted.The components of the pivoting device can be similar to those of the shaking device, but are configured for pivoting movement. A treatment device can also be a transport device by means of which at least one cell culture container can be transported within the incubator chamber.

[0127] The transport device can be a lifting device comprising a carrier device on which at least one object, in particular a cell culture container, camera, or light source, can be placed. The transport device or lifting device preferably has a movement mechanism and / or an electrically controllable drive mechanism for driving the movement mechanism. The transport device can further be a movable and electrically controllable gripper arm for gripping and holding at least one cell culture container. The transport device can have a conveyor belt or a rail system for moving the at least one object placed thereon. By means of the transport, the at least one object can be moved in the incubator chamber, in particular to a processing position or receiving position, e.g. in a processing station, in the incubator chamber, and away from this processing position or receiving position.The control device can be configured to control the transport device depending on information from previously acquired image data.

[0128] A treatment device can also be a transport device by means of which at least one camera of the camera device and / or at least one light source can be transported in the incubator chamber. The transport device can in particular be arranged below or directly below a storage plate, and / or below or directly below a ceiling wall of the incubator chamber. Different lighting modes can be implemented using a moving or movable light source. In particular, the lighting mode can be adapted to an occupancy state, e.g. in order to variably implement a suitable lighting direction when a storage area is very densely occupied with objects. Using several cameras, as well as a moving or movable camera, different images or image sections of the storage area can be created, which can then be combined into an overall image of the storage area, in particular using digital image processing.In the case of a movable camera, it is also possible to adapt the camera position to the occupancy status, e.g. to achieve a suitable lighting direction when a storage area is very densely occupied with objects.

[0129] In particular, the data processing device can be programmed to transport the at least one camera of the camera device and / or at least one light source in the incubator chamber by means of the transport device in a predetermined or dynamically adapted manner. For example, the data processing device can be programmed to transport the at least one camera of the camera device and / or at least one light source in the incubator chamber to different recording positions by means of the transport device, in particular to evaluate the image created there in each case using an image processing algorithm and in particular to check whether a desired piece of image information, e.g. an individual feature of a cell culture container, in particular a barcode, was captured with sufficient quality, e.g. to clearly read the barcode.The data processing direction can be programmed to move the camera and / or the light source to other recording positions until a desired image information is captured.

[0130] The camera and / or lighting device can also be attachable to a transport device. The camera and / or lighting device can be or is attached to a positioning mechanism by means of which the camera and / or lighting device can be moved and positioned in the incubator chamber. The positioning mechanism can include a movable robot arm and is preferably electrically controllable, in particular by a control program of the control device. In this way, different recording situations can be captured one after the other with one or a few camera and / or lighting devices. The positioning mechanism can be designed as a component that can be inserted into the incubator chamber. The power supply to this component can be via a cable connection to the incubator, preferably via a wall opening, e.g., a port, or via such a cable connection to an external voltage source.The control device can be configured to control the positioning mechanism depending on cell monitoring data. The temperature control device of the incubator chamber, with which the atmosphere inside the incubator chamber is regulated to the desired value, in particular 37°C, can also be understood as a treatment device. The term "temperature control" refers to raising and lowering the atmospheric temperature by heating and cooling. Preferably, the temperature inside is adjusted by changing the temperature of the incubator walls. Temperature sensors of the corresponding temperature control device are distributed at at least one position inside and / or outside the incubator chamber, in particular on a wall of the incubator chamber.

[0131] The incubator preferably has a user interface device via which the user can input data into the data processing device or the control device, and / or via which information can be output to the user. Preferably, the incubator or this user interface device is configured to allow the user to input at least one operating parameter for operating the incubator or the image acquisition system at this user interface device or to receive information from it. In this way, a single user interface device can be used by the user to influence or control the incubator and also the at least one image acquisition system, or to receive information from them.In particular, the image capture system can be configured to display position data or free storage space to the user in response to a query made by the user via the user interface device of the incubator, or information derived from position data (e.g. identity of the user who brought about the change in position), in particular also statistical information, such as frequency and time of the change in position of an object (a sample) and / or - in particular as a percentage - available free storage space, and / or at least one optical image of the at least one object, in particular with or without the storage area.This is advantageous for the user, as this information provides essential information that allows for more precise experimental planning – before conducting an experiment, the user knows that space is available. Furthermore, changing the position of samples, particularly in the first few hours after seeding of adherent cells, negatively influences their adhesion; a uniform cell layer is then not formed. The inventive provision of information on position changes and their frequency allows the user to determine the causes of uneven cell growth and thus take them into account in future experiments.

[0132] A device-controlled treatment of the incubator is preferably a program-controlled treatment, i.e. a treatment controlled by a program. A program-controlled treatment of an object is to be understood as meaning that the treatment process essentially takes place by processing a plurality or multiplicity of program steps. The program-controlled treatment is preferably carried out using at least one program parameter, in particular at least one program parameter selected by the user. A parameter selected by a user is also referred to as a user parameter. The program-controlled treatment is preferably carried out by means of the digital data processing device, which is in particular a component of the control device. The data processing device or the data processing apparatus can have at least one processor, i.e. a CPU, and / or at least one microprocessor.The program-controlled treatment is preferably controlled and / or carried out according to the specifications of a program, in particular a control program. In particular, with a program-controlled treatment, essentially no user activity is required, at least after the user-specified program parameters have been recorded. A device-controlled treatment of the incubator can be carried out, in particular, depending on previously acquired image data. The image acquisition by the image acquisition system is, in particular, a program-controlled treatment, namely imaging of the storage area or object.

[0133] The data storage device or the data storage apparatus preferably comprises at least one data storage device, which may in particular be a volatile or non-volatile data storage device. The data acquired or received by the incubator can be stored on this at least one data storage device, in particular in at least one database, which may be stored in at least one data storage device. This data includes in particular at least one or all of the following data types: image data, still image data, video image data, object data, combined image data, first and second image data, identification data, ID position data, user identification data, user-related ID position data,

[0134] Object identification data, movement history data, class-related ID position data, individual-related ID position data, occupancy status data, in particular parking space ID data.

[0135] The data storage device / data storage apparatus is preferably a component of the incubator, i.e., in particular, arranged in a housing of the incubator. However, it can also be a component of an external data processing device with which the incubator or its data processing device communicates. The data storage device is a component of the system according to the invention.

[0136] A program parameter is a variable that can be set in a predetermined manner within a program or subprogram, valid for at least one execution (call) of the program or subprogram. The program parameter is set, e.g., by the user, and controls the program or subprogram and causes data output depending on this program parameter. In particular, the program parameter and / or the data output by the program influence and / or control the control of the device, in particular the control of the treatment by means of the at least one treatment device.

[0137] A program, program code, or computer program code is understood to mean, in particular, an executable computer program. This is stored in a data memory or on a data storage medium. A program is a sequence of instructions, particularly consisting of declarations and instructions, for processing and / or solving a specific functionality, task, or problem on a digital data processing device. A program is generally in the form of software that is used with a data processing device. The program can, in particular, be in the form of firmware; in the case of the present invention, this is in particular the firmware of the control device of the incubator or the system. The program is usually stored on a data storage medium as an executable program file, frequently in so-called machine code, which is loaded into the main memory of the computer of the data processing device for execution.The program is processed as a sequence of machine, i.e., processor, instructions by the computer's processor(s) and thus executed. The term "computer program" also refers, in particular, to the program's source code, from which the executable code can be created during the control of the laboratory device.

[0138] A user interface device can be a component of an incubator or a module. A user interface device preferably comprises: a control device for the user interface device; a communication device for establishing a data connection with a laboratory device, in particular an incubator, via an interface device thereof; an input device for detecting user inputs from a user; an output device, in particular a display and / or a screen, for outputting information to the user, in particular a touch-sensitive screen. The control device of the user interface device is preferably configured to exchange data with the control device of the incubator via the data connection.

[0139] An object is, in particular, a cell culture container. A cell culture container is, in particular, transparent. It is, in particular, made of plastic, in particular PE or PS, and in particular has a planar base plate that forms the growth surface for the cells. This base plate can have a surface treatment to promote cell adhesion. The cell culture container can be closable or provided with a PE cap or gas exchange cap, in particular a lid with an optionally included filter. The cell culture container is, in particular, stackable. An Eppendorf cell culture flask is, in particular, suitable. The object can be a stack of cell culture containers, in particular a stack of Petri dishes or cell culture flasks.

[0140] The data processing device is preferably programmed to detect (time-dependent) changes in the appearance (or the appearance) of the objects from one or more images, in particular between longer time intervals of minutes, hours or days. In this way, colour changes in the cell culture medium or colours in a cell culture container or structures, e.g. drops, on a cell culture container wall can be detected. Such colours, colour changes or structures can indicate problems with the respective cell culture, e.g. a nutrient deficiency, pH changes, mould or other contamination. The data processing device is preferably programmed to detect the appearance of a cell culture container orof these changes in the appearance of the cell culture container, to provide information to the user or operating personnel via a user interface and / or to store the data on this detection (in particular: what was detected and when) in a data storage device and to keep it available for retrieval.

[0141] Image-based object tracking techniques are generally well known, for example, in drones or driver assistance systems for vehicle or person tracking. Object tracking is based on image processing or image analysis of video data. Such object tracking methods can be implemented using relatively simple tools, such as suitable cameras and image processing algorithms. The theoretical foundations and their application in the practical implementation of object tracking techniques are well known (e.g., "Fundamentals of Object Tracking," S. Challa et al., Cambridge University Press, 2011). Ready-to-use image processing algorithms for object tracking are also freely available (OpenCV.org) and well documented. OpenCV (Open Computer Vision) is a free program library (BSD license) with algorithms for image processing and computer vision.The OpenCV library also includes functions for tracking multiple objects in real time. The application of object tracking in incubators has not been published before and represents a breakthrough.

[0142] A typical operation of image-processing object tracking, which is preferably also used in the object tracking system according to the present invention, is based on the evaluation of the temporal sequence of images. A typical program code for object tracking uses a "bounding box" as an output format to identify an object in an image, to define its collision boundaries, and in particular to localize it. In digital image processing, the "bounding box" refers to the coordinates of the rectangular frame that largely or completely encloses an object shown in the digital image. The use of bounding boxes in object tracking makes it more efficient, since image evaluation using such a numerical tool requires fewer calculation steps and thus less computing power, especially compared to algorithms for object outline detection.Furthermore, the corresponding algorithms can be executed efficiently and cost-effectively using specialized graphics processing units (GPUs). Suitable application programming interfaces (APIs) for object tracking using bounding boxes are available in the OpenCV library under the names BOOSTING, CSRT, GOTURN, KCF, MEDIANFLOW, MOSSE, MIL, and TLD. Accordingly, OpenCV provides libraries ("MultiTracker") for the simultaneous tracking of multiple objects ("multiple object tracking"). As an alternative, deep learning algorithms for multi-object tracking (MOT) based on the "tracking-by-detection" principle are known.

[0143] However, it is also possible and preferred that for object tracking, a determination of the contour of the object to be tracked in the image is carried out, and in particular the separation of object (foreground) and background by background subtraction.

[0144] The performance potential of an object tracking system is based, on the one hand, on the reliable automatic identification of an object in the incubator in various typical incubator usage scenarios, which are described below. On the other hand, the approach is efficient because no special adaptations are required on the object side. In particular, the object does not need to contain any passive (code, labeling) or active (e.g., a transmitter) identification aids. Rather, conventional objects (cell culture containers, devices, etc.) can be used with the incubator, particularly regardless of manufacturer and external appearance. In particular, the incubator according to the invention is capable of distinguishing objects with a completely identical appearance through tracking.

[0145] Possible scenarios for changing occupancy in an incubator include:

[0146] I. Setting up new objects

[0147] II. Removal of objects

[0148] III. Door is opened and objects are only moved without removing or repositioning any.

[0149] Subconditions: i) Object(s) are moved ii) Object(s) are not moved iii) Multiple objects are added / removed (sequence).

[0150] Assumption: All cell culture containers look the same externally. The question underlying the development of the invention was, in particular, which image-based methods are suitable, in particular whether still images are sufficient to enable object identification in typical incubator usage scenarios.

[0151] For scenario I (a new object is to be placed in the incubator chamber), it is initially assumed that before the incubator door is opened, there is a current still image of the storage area in the incubator chamber, taken by a camera placed in the incubator, which does not yet show the new object.

[0152] If the new object is placed in the incubator chamber without moving the existing objects (objects already arranged and located in the storage area) (case l.ii)), the new object can be identified (without difficulty and unambiguously) via the next still image (after the incubator door is closed). Object tracking is not necessary for case l.ii). The same applies to II.ii): if an object is removed, its identification is unambiguously possible from the evaluation of the still images before and after the door is opened.

[0153] If the new object is added and existing objects are moved in the process (case li)), the new object cannot be clearly identified from the next freeze frame. Location information about the already registered existing objects is lost. The same applies to the removal of an object and the subsequent relocation of the existing objects (case ll.i)). For relocation (condition i)), the concept of object tracking applies.

[0154] If multiple objects (case iii)) are added under condition i), i.e., without moving the existing objects, the new objects can be easily identified using before-and-after freeze-frames, but the information about the order of insertion is lost. Obtaining this information requires object tracking. The same applies to the removal of multiple objects in cases i) + iii).

[0155] Since moving inventory objects is the rule rather than the exception in the operation of an incubator, an evaluation of the before-and-after still images of the storage area is not sufficient in this case.

[0156] One question in particular when developing an object tracking system in an incubator is: When is an object identified, i.e. at what point in time or during which event is identification data assigned to the object? In most application scenarios (apart from a case such as iii)+i), where recording the order in which objects are added may be important), it is sufficient to collect this identification data when the new objects have been added and existing objects may have been moved in the process. This is because the moving of the already registered existing objects takes place under the object tracking measure. In the next freeze frame, i.e. in particular when the incubator door is closed, the new objects can then be registered based on the freeze frame.If it is desired to record the sequence, the moment an object first enters the camera's field of view, i.e. first appears in an image recorded by the camera (a start image, which in this case is a video image), it will be registered, i.e. an ID number will be assigned to the object, and this object will be tracked to its final position. Any moving of existing objects or their removal will preferably also be tracked.

[0157] In another practical scenario, it is assumed that the storage area (or several storage areas) is occupied by one or more objects (stock objects) which are registered using an initial start image (in this case, for example, a still image). Here, only any movement of these stock objects needs to be tracked. The movement of newly inserted objects does not need to be tracked during the insertion process, as they can be registered again in the next still image. The corresponding presence of these new objects in the video data can therefore be ignored. In this scenario, the information about the sequence in which multiple objects are inserted during a door opening is lost, but this information is not absolutely necessary.

[0158] The invention therefore proposes, in a preferred embodiment, to implement object tracking to ensure the correct localization of objects, as required, in different or all situations.

[0159] The data processing device is programmed in particular to assign identification data to the at least one object introduced into the interior. This means, in particular, that a new object is detected in the image data (still images or video data) of the camera. In particular, a new object is detected when it is moved from outside into the camera's field of view. Upon detection of the object, identification data and position data can be assigned to it. The position data, in particular the start and end positions of an object, are determined in particular with reference to an internal coordinate system, which is also used to define the position of the at least one storage area and thus also the position of the at least one object relative to the at least one storage area.This position information is particularly important if the position of at least one object in at least one storage area or in the incubator chamber is to be graphically illustrated to the user on a display.

[0160] Identification data can be or contain an identification number, and / or can contain an identification code consisting of any characters or information. This identification data can be predetermined, randomly generated, or specified by a user, particularly as long as it is suitable for clearly distinguishing the newly placed object in the incubator chamber from the identification data of the other existing objects. The identification data can also be predetermined and merely selected. The term "assign" encompasses the latter case, as does the creation of new identification data.

[0161] The data processing device is programmed, in particular, to determine the starting position of the at least one object from the starting image of the storage area. The starting image is preferably a still image recorded in a still image mode of the camera. It can also be a single image obtained from video data, in particular a video frame. The data processing device is programmed, in particular, to define an enveloping line figure, preferably a rectangle, or an enveloping body, or in particular a bounding box of the object, or an outer contour of the object, in the starting image and, in particular, to define the object as the area enclosed by the enveloping line figure, in particular the bounding box or an outer contour.

[0162] The data processing device is programmed, in particular, to determine the position changes of the at least one object by evaluating the video data. The data processing device is programmed, in particular, to track the movement of the object defined in the start image by means of the bounding box. The data processing device is programmed, in particular, to detect the movement of the image region containing the object, defined in the start image by means of the bounding box, by determining the position changes of this image region from frame to frame. Tracking the bounding box can, in particular, be used to determine the image region whose position changes due to the object movement.Video data contains, in particular, information that allows the individual images ("frames," which, when the video is displayed at a specific number per unit of time, i.e., a "frame rate") to be reconstructed. In the case of uncompressed video data, the latter can also contain the complete sequence of image data, with each "set" of image data representing an individual image. In the case of compressed image data, temporal changes in pixels of the camera image may also be recorded.

[0163] The data processing device is programmed in particular to determine the starting position of the object in the storage area from the starting image. The starting position can be determined in particular by the fact that the object, which was previously stationary in a first frame of an image series, shows a change in the object's position in the subsequent frame. The first frame in which the object shows a change in position compared to the previous frames can be defined as the starting image. Since the movement of the object begins at a time T1 and ends at a time T2, an image (still image or video image, also an image obtained from superposed images) captured before the time T1 can be used as the starting image from which the starting position of the at least one object is determined.

[0164] The data processing device is programmed in particular to determine the final position of the at least one object in the final image of the storage area from the position changes. The final position can be determined in particular by no longer determining any changes in the object's position from frame to frame. The first frame in which the object no longer shows any change in position compared to the previous frames can be defined as the final image. Since the movement of the object begins at a time T1 and ends at a time T2, an image (still image or video image, also an image obtained from superposed images) captured from time T2 onwards can be used as the final image from which the final position of the at least one object is determined. The final position can be determined in particular by the time at which the incubator door is detected closing by the door sensor.The end position can be determined in particular by the fact that an arm or hand of the user extending into the image area is no longer detected. For example, an image can be evaluated to determine whether a section located at the edge of the image, e.g. a strip-shaped section, corresponds to a reference state in which a reference section of the incubator chamber or incubator is completely visible. If this is not the case, it can be concluded that a user is still handling things inside the incubator and that the object or multiple objects are still being moved, so that in particular the video image acquisition and analysis must be continued. The end position of the object can be understood as the position at which the object no longer shows any change in position after previous changes in position, and can therefore be determined by the end of the object's movement.Alternatively or additionally, the object's final position can be defined such that the object's position when the incubator door is closed is detected by the door sensor. As a result, the final position will be the same in most cases.

[0165] The data processing device is preferably programmed to start the capture of the start image and / or video data by means of the camera when a sensor detects activity occurring on the incubator. The sensor can be a motion sensor that detects movement in a detection area located outside the incubator. The sensor can be a touch sensor that detects a user touching the incubator, e.g., a door handle of the incubator. The sensor can be a door opening sensor that detects, in particular, the opening of an incubator door, in particular an external door of the incubator. The sensor can be an external camera of the incubator that detects movement and / or a person in the camera's field of view by means of image analysis. The sensor can be a proximity sensor that detects the approach of a person to the incubator, e.g., by detecting a change in an electric field.The data processing device is preferably programmed to start the acquisition of the final image and / or to stop the acquisition of video data when a sensor detects activity occurring at the incubator. The sensor can be a door opening sensor, which in particular detects the closing of an incubator door, in particular an external door of the incubator. The sensor can be an external camera of the incubator, which uses image analysis to detect the cessation of movement and / or the disappearance of a person within the camera's field of view. The sensor can be a proximity sensor, which detects the departure of a person from the incubator, e.g., by detecting a change in an electric field.

[0166] The data processing device is particularly programmed to start the acquisition of video data by means of the camera initiated by the opening of an incubator door detected by a door sensor. Alternatively or additionally, an initial event sensor, in particular a motion sensor, or proximity sensor, or optical sensor / receiver (e.g.A sensor (e.g., a light barrier), a microphone, or an acceleration sensor in the incubator door can be arranged in the incubator, by means of which the approach of an object to the incubator chamber or another initial event can be detected; the trigger for initiating the camera and video data can also be a code entry on a door lock of the incubator, which can be carried out in particular by the data processing device without using measurement results from one of the said sensors; the data processing device can be programmed to start the acquisition of video data based on the data from such a sensor. The data processing device can be programmed to begin the search for a new object in the images (frames) available via the video data or the still image when the video data and / or a still image are available.Alternatively, the acquisition of video data can begin as soon as a user is identified at the incubator, or upon another predefined event. Continuous video data acquisition is also possible. The data processing device is particularly programmed to end the acquisition of video image data upon capture of the final image or upon registration of the absence of a hand / arm in the camera's field of view, or to end it based on the results of one of the aforementioned sensors (door sensor, motion sensor, etc.). The data processing device is particularly programmed to assign identification data to the at least one object in the storage area and to determine the position of the at least one object as ID position data and to store it in the data memory.The data processing device is programmed, in particular, to store the final position of the at least one object in the storage area as ID position data in the data memory, depending on the identification data of the at least one object. With this step, the incubator "knows" the object and its position. Together with other data, it can then output this data to a user, in particular displaying it on a display of the incubator. Together with data about the owner (definition of owner: the user who placed the object in the incubator chamber) of the object or a user of the object (e.g., a user who moved an inventory object belonging to another user), the incubator can store and collect these data records depending on the object's identification data.The identification data used to detect the position changes does not have to be identical to the identification data stored as ID position data; what is relevant is that the stored identification data is suitable for clearly distinguishing at least one object from other objects or existing objects. The ID code can therefore theoretically change during image processing.

[0167] The assignment of an owner to an object can be achieved in various ways. Preferably, the data processing device is programmed to register or identify the user placing the object in the incubator, to assign a user identification code to this user, and to store user-related ID position data of the object. For registration, biometric recognition, in particular facial recognition, speech recognition, and / or voice recognition, of the user can be performed, in particular using an external camera, a retina scanner, or a fingerprint sensor of the incubator. The corresponding registered biometric recognition data, in particular facial recognition data of the user, can be stored in the data storage device of the incubator or in an external data storage device.The user can be identified by comparing captured biometric recognition data with previously registered biometric recognition data. As an alternative to biometric recognition, a user can also be enabled to enter user identification data via a user interface device before, during, or after object registration or after determining the final position of a tracked object. The user interface device can be a keyboard, a touchscreen, and can be part of the incubator or an external device; or a user name / ID can be entered via voice input.

[0168] One advantage of object tracking is that it can generally be performed without knowledge of individual or class characteristics of the object to be tracked. However, it can also be combined with methods for object detection (and re-recognition) and / or object class detection or re-recognition. This is particularly useful when multiple objects are tracked in parallel using an object tracking system.

[0169] Object recognition can be implemented, in particular, as individual object recognition and / or object class recognition. The theoretical foundations and their practical application in object recognition technologies are well known (e.g., "Deep Learning in Object Detection and Recognition," X. Jiang et al., Springer Singapore, 2019). Algorithms for object recognition in images are well known and available, e.g., as part of OpenCV (for example, OpenCV 3.3, deep neural network (DNN) module).

[0170] Individual object recognition is based on the recognition of individual object characteristics (object characteristics), which enable the individual object to be recognized and distinguished from other individual objects. For example, a cell culture container, e.g., a disposable product, may have subsequently applied individual characteristics, such as a barcode or QR code. However, it can also be identified by any other characteristics that enable differentiation: e.g., a label, a different content, a micro-scratch pattern on the container surface, etc. Object class recognition is based on the knowledge of object class characteristics, which are compared during object inspection to assign the object to a class.For example, object class recognition can be used to determine whether an object is a specific type of cell culture flask, a specific type of Petri dish, or a specific type of microtiter plate, possibly taking into account other class characteristics, e.g., manufacturer, year of manufacture, model, etc.

[0171] The incubator preferably has an object detection system. The object tracking system is preferably configured in addition to the object detection system.

[0172] In the case of individual object recognition, a data processing device of the object recognition system or the object tracking system is preferably programmed to recognize individual characteristics of at least one object from a still image, the start image, the video data and / or the end image, and b) to store these individual characteristics of the object in the form of individual object data, in particular depending on identification data.Preferably, the data processing device is programmed to extract individual object characteristics of the at least one object from the start image, the video data and / or the end image, to compare the individual object characteristics with an individual object database and, if the individual object characteristics in the individual object database are linked to an individual object identifier: to identify the individual object identifier of the at least one object; or, if the individual object characteristics in the individual object database are not linked to an individual object identifier: to assign an individual object identifier to the at least one object and store it in the individual object database, and / or to assign the recognized individual object identifier to the ID position data of the at least one object and store it as individually related ID position data.An object's unique identifier is preferably different from its identification data; however, the object's unique identifier may also preferably be the same as its identification data.

[0173] In the case of object class recognition, a data processing device of the object recognition system or the object tracking system is preferably programmed to a) recognize class features of the at least one object in a still image, the start image, the video data and / or the end image, and b) store these class features of the object in the form of object class data, in particular depending on identification data.Preferably, the data processing device is programmed to recognize object class features of the at least one object in a still image, the start image, the video data and / or the end image, to compare the object class features with an object class database (which in particular contains previously known correlations between the object class and object class features) and to recognize the object class of the at least one object, and in particular to assign the recognized object class to the ID position data of the at least one object as object class data and in particular to store it as class-related ID position data.

[0174] The incubator preferably has a user identification device by means of which a user using the incubator can be identified in the form of user identification data. Preferably, a data processing device of the incubator is programmed to identify a user using the incubator by means of the user identification device and to assign user identification data to the user, and to store identification data and / or ID position data in the data memory as user-related identification data and / or user-related ID position data depending on the user identification data.

[0175] The user identification device preferably has an external camera, and the user identification device is preferably configured, and / or the data processing device is programmed, to carry out facial recognition by means of the external camera, by means of which facial recognition the user is identified. A user database is preferably provided, which is stored on a data storage device which may be part of the incubator, the user identification device or the object tracking system, or which may have a data exchange connection with the user identification device or the data processing device, e.g. via an intranet or the internet. Algorithms for facial recognition in images are generally known and available, e.g. as a component of OpenCV (“FaceRecognizer”). The user database may contain a correlation of user identification data and user feature data, so that the user orwhose user identification code (user identification data) can be determined based on the determined or read-in user feature data. The user feature data can contain information about the user's facial features or other biometric data, e.g., fingerprint data or voice recognition data. The user database can contain a correlation of user identification data and user IDs, whereby the user ID can be a personal identification code of the user, e.g., a multi-character string that a user can use to identify themselves when entered on a keyboard of the user interface device.

[0176] The external camera can be arranged or attached to an incubator door, in particular an external door on the incubator. The external camera is preferably an integral part of the incubator or the incubator door. However, it can also be connected to the user identification device or the data processing device via a signal connection, in particular via a data exchange connection, which can be wired or wireless. For example, it is possible to connect the external camera to the incubator or its user identification device or the data processing device via a flexible cable, so that the user can freely position the camera on the incubator.

[0177] The user identification device preferably has a user interface device by means of which user identity data can be read. The user interface device can include a keyboard, and / or a touchscreen, and / or a microphone for voice input or for implementing user identification by means of voice recognition. The user interface device can be configured to exchange data with an external data processing device (hereinafter also referred to as an "external device"). The external device can be a PC, a smartphone, a tablet computer, or another portable computer with a user interface.

[0178] The external device can have means for identifying and / or authenticating a user. Currently available smartphones, in particular, contain various means for user authentication, in particular for facial recognition. The external device preferably has software, e.g., an app, that is programmed to identify and / or authenticate a user, and in particular to transmit the result of this process via the user interface device to the incubator's user identification device. Since an external device often also has its own camera, which can be used to implement facial recognition, or a fingerprint sensor, or other hardware for user identification and authentication, the corresponding hardware components are dispensable if the incubator is connected to the external device on the incubator.

[0179] The incubator's user identification device can be programmed as part of the incubator's control software. The incubator preferably has a control device, which can in particular have a data processing device, which can in particular be programmed to contain all or some of the functions of the user identification device, in particular to control the data exchange with the external device.

[0180] The user identification device preferably has a user interface device by means of which user identity data can be selected. For this purpose, the user identification device can, in particular, have a display or a touchscreen, via which a list of possible users can be displayed, e.g., by specifying a name or image of the user. Input means can then be provided, e.g., buttons, a keyboard, touchpad, or the touchscreen, via which the user can make the selection from the list.

[0181] The user identification device can be programmed to perform user authentication by password-protecting the aforementioned reading of the user identity data or the aforementioned selection from the list, so that the user is only considered identified once the authentication has been successful. Preferably, the user identification device has a reader for reading a code identifying the user, wherein the reader is, in particular, an RFID reader, a barcode reader, or a QR code reader.

[0182] The user identification device or its data processing device can be programmed to unlock and / or lock a locked incubator door depending on the user identification, in particular to unlock a locked incubator door if the user has been successfully identified. In this case, this means that the user is also authorized to access the incubator. However, there can also be an additional access rights list based on which the incubator decides whether an identified user has access rights or, if applicable, what type of access rights the identified user has. The access right can, for example, be restricted to certain times, in particular days of the week, times of day, or authorization periods. If the incubator has multiple incubator doors, the access right can stipulate that the user has access rights only for a predetermined selection of these incubator doors.

[0183] The incubator preferably has exactly one—or more—incubator door(s) for closing the chamber opening. When closed, the incubator door forms, in particular, a part of the incubator housing, which serves as a thermal insulator for the incubator chamber. The incubator door can have a user interface device, in particular a display, on its exterior. A data processing device of the incubator or the user interface device can be programmed to display an image of the at least one storage area of ​​the incubator captured by the incubator's camera.

[0184] The incubator preferably has a door sensor for detecting the opening or closing of the incubator door. The incubator preferably has a motion sensor or a proximity sensor for detecting a person approaching the incubator. The data processing device is preferably programmed to start monitoring the interior of the incubator, in particular generating video data / still image data, depending on the detection of a door opening of the incubator and / or the approach of a person. The data processing device is preferably programmed to start monitoring the interior of the incubator, in particular generating video data / still image data, particularly depending on the detection of a door opening by a user identified by means of a user identification device.Preferably, the data processing device is programmed to terminate the monitoring of the interior of the incubator, in particular the generation of video data, depending on the detection of a door closure of the incubator; Preferably, the data processing device is programmed to determine, using the information from the user identification device and the object tracking device, which user moved which object in the interior, and to store the user identification data of this user together with the object identification data of this object in the data memory.

[0185] Preferably, the data processing device is programmed to determine the movement path of the at least one object within the incubator chamber from the start image, the video data, and / or the end image and to store it in the data memory in the form of movement history data, in particular to store it time-dependently. Preferably, the data processing device is programmed to determine a movement history of the at least one object within the incubator chamber from the start image, the video data, and / or the end image and to store it in the data memory in the form of movement history data, in particular to store it time-dependently, preferably with information about the number and / or times of the changes in the status of the door opening (open / closed) of the incubator door determined by the door sensor.The movement path preferably contains stored position data of the object, wherein this position data marks the movement path of the object, in particular between a start image and an end image, in particular between a starting position of the object—particularly stationary or moving—and a final position—particularly unmoved. The movement history data preferably contains time-dependent stored position data or movement paths, preferably within at least one time period or during the entire stay of this object in the incubator. Movement history data can also include information on the user who triggered the change in position in the form of user identification data. This is particularly advantageous for objects containing valuable samples.

[0186] The incubator preferably has a display (=screen). The screen is preferably a fixed component of the incubator, in particular the incubator door. However, it can also be located remotely from the incubator and can, in particular, be part of an external device that can be connected to the incubator's data processing unit for data exchange.

[0187] Preferably, the data processing device is programmed to display a graphic representation of the interior of the incubator chamber, in particular of the at least one storage area, on the screen. The graphic representation can contain a photo of the storage area, on which one or more inventory objects of the incubator can be displayed. The storage area can in particular be a storage plate in the incubator or a predetermined section thereof. The photo can show an image taken with the camera, which can optionally be post-processed. This post-processing preferably involves straightening an image taken by the camera in a distorted manner. Algorithms for such post-processing are generally known and freely available (for example: OpenCV, “Omnidirectional Camera Calibration”). The distortion can in particular be optically caused and be attributable to the use of a wide-angle or fisheye lens.

[0188] The graphic representation can in particular be an abstracted representation of an image or image section recorded by the camera. For example, the graphic representation can be an abstracted storage area shown from a bird's eye view (or another perspective), in particular the graphic representation of a rectangle or the perspective representation of a cuboid. The inventory objects can also be represented in an abstract form, e.g. as rectangular or cuboid graphic image objects. The aim of such a representation is in particular to inform the user about the location of the object(s) in the incubator or in the storage area. This enables the user to quickly access the desired object(s) and minimizes the time the incubator door is open.In the case where differentiation between individual objects contained in a stack of objects is to be made possible, a graphic representation from a perspective different from the bird's eye view is useful, for example from a side perspective, in order to be able to graphically highlight individual objects in a stack.

[0189] Preferably, the data processing device is programmed to graphically display where the object identified by the object position data is positioned in the storage area or in the interior of the incubator chamber, or to graphically display where all objects located in the interior are arranged.

[0190] Preferably, the data processing device is programmed to graphically highlight one or more objects on the display depending on at least one condition parameter. The condition parameter can designate the user identification data. Highlighting is possible both in an abstract representation and in a photographic reproduction of an image or image section of the storage area captured by the camera on the incubator display.

[0191] Preferably, the data processing device is programmed to graphically highlight on the display one or more objects assigned to the user as property, depending on user identification data of a user (individual user, a user group, or multiple users), for example by the user-related ID position data containing the user identification data of this user. The owner is the person who looks after the object and - in most cases themselves or with the help of an assistant - placed it in the incubator chamber. Preferably, the data processing device is programmed to determine, based on predetermined user identification data, where the objects assigned to this user identification data using the user-related object position data are positioned and, in particular, to graphically highlight these objects.

[0192] The condition parameter can also contain information about a period of time or a point in time, e.g., the length of time an object has been in the incubator chamber. This allows a user to quickly obtain an overview of how long one or more objects have been stored in the incubator chamber, perhaps forgotten by their owner. Alternatively, the incubator can graphically highlight one or more objects that require the attention of the user or laboratory staff, depending on an event detected by an incubator sensor or a schedule that may be stored in the incubator or an external device.

[0193] Or, in the case of implementing object class recognition, the condition parameter can contain information about a specific object class. This allows one or more objects of the same object class (or of different object classes) to be graphically highlighted, for example, to highlight the location of all Petri dishes (and not cell culture flasks) in the incubator interior.

[0194] Or, in the case of implementing individual object recognition, the condition parameter can contain information about a specific individual object. In this way, an object search based on individual characteristics can be implemented, for example, by providing the incubator with means for entering individual characteristics, such as a barcode, QR code, a custom label, or a photo of the individual object. This allows the individual object to be graphically highlighted and easily found.

[0195] Preferably, the data processing device is programmed to display a graphic representation of the interior of the incubator chamber, in particular of the at least one storage area, on the screen and, in particular, to graphically display or highlight the free storage space available in the incubator. For example, the storage area can be shown in an abstract manner, and a free storage position (or several available free storage spaces) can be graphically highlighted by displaying the corresponding area, for example, in green or white, or a time-changing (flashing) contrasting color against the background. In this way, the user does not have to spend time searching for a possible free storage space or creating one by moving inventory objects.

[0196] Furthermore, similar to the function of a parking attendant, the data processing device can be programmed to plan the occupancy of the interior of the incubator chamber or of the at least one storage area, and in particular to optimize the use of the available storage space in this way. For this purpose, the data processing device can be programmed to consider predetermined distances between one or more existing objects and a newly added object and, in particular, to suggest these distances to the user by highlighting the free storage space as available and / or unavailable.According to these examples, the incubator can have a computer / software-implemented planning program for occupancy of the incubator's interior, which program takes into account, in particular, the position of at least one object in the interior (stock object) and / or, in particular, the freely available storage space, possibly also the times at which the at least one stock object was newly added, or times in the future at which the addition of further objects to the incubator is planned. Such times can be known, in particular, if the incubator is connected to a laboratory information system (LIS) or another (laboratory) data exchange network. The incubator preferably has a timer, a clock, or a timepiece.

[0197] It is possible and particularly preferred that a data processing device of the incubator is programmed to determine an occupancy state of the interior of the incubator chamber and / or is preferably programmed to carry out one or more of the following steps, in particular to determine an occupancy state of the interior of the incubator chamber depending on the ID position data of the at least one object arranged in the interior, to determine an occupancy state of the interior of the incubator chamber depending on the class-related ID position data of the at least one object arranged in the interior, to determine an occupancy state of the interior of the incubator chamber depending on the individual-related ID position data of the at least one object arranged in the interior.

[0198] An occupancy state of the interior can be defined by information that describes the volume in the interior occupied by the at least one object, and / or that describes the volume in the interior not occupied by the at least one object, i.e. the free volume, and / or the storage area occupied by the at least one object in at least one storage area or in the total available storage area in the interior of the incubator chamber, and / or the storage area not occupied by the at least one object, i.e. the free, in at least one storage area or in the total available storage area in the interior of the incubator chamber, wherein this information can each relate to the total interior volume or the total storage area, wherein this information e.g.the ratio of an unavailable (occupied) or a free (unoccupied) interior volume to the total volume of the interior, or where this information can include, for example, the ratio of an unavailable (occupied) or a free (unoccupied) storage area to the total storage area in the interior.

[0199] Occupancy status data, which contains information about the occupancy status, can also contain ID position data, class-related ID position data, and / or individual-related ID position data. This makes it possible to specify a spatial resolution of the occupancy, i.e., the location of the occupancy within the interior, or a density distribution of the objects within the interior.

[0200] It is possible and particularly preferred that a data processing device of the incubator is programmed to store information about the occupancy status of the incubator in the form of occupancy status data in a data memory, in particular to transmit it to an external data processing device, in particular a laboratory device, a PC, or a mobile computer, in particular a tablet computer or a smartphone.

[0201] It is possible and particularly preferred for a data processing device of the incubator to be programmed to display information about the occupancy status of the incubator on a screen of the incubator or an external data processing device, in particular depending on occupancy status data that can be taken from a data storage device. The external data processing device can be part of a laboratory device, PC, a mobile computer, in particular a tablet computer, or a smartphone.

[0202] In test series on which embodiments of the present invention are based, it was found that the temporal temperature profile in the incubator chamber resulting from temperature control after the incubator door is opened depends on the occupancy of the incubator chamber. If a larger volume of the chamber interior is occupied by existing objects, there is a smaller free chamber interior volume, which results from the difference between the chamber interior volume and the occupancy volume occupied by the objects. A temperature control designed to control the entire interior volume may achieve different, undesirable results in this situation. Rapid overshoot may occur, which is undesirable, even if this may accelerate the restoration of the target temperature, e.g. 37 °C, i.e. even if the recovery time is shortened.If several new objects are introduced at temperatures lower than the target temperature, the recovery time may also be delayed. However, knowledge of colder, newly introduced objects can also be used to adjust the temperature control. Temperature control of the temperature inside the incubator chamber depends on control parameters.

[0203] Preferably, an electronic control device of the incubator is configured or programmed to operate at least one temperature control device of the incubator, which is arranged to control the temperature of the incubator chamber, with the electrical power Ptemp(t) during temperature control as a function of time t. In particular, the incubator can be configured to operate the temperature control device by means of pulse-width modulation (PWM) of the current. The power is then determined in particular by the duty cycle of the PWM, since the amplitude of the current is preferably constant. In particular, the aforementioned variables can be variables of the temperature control, i.e., control parameters.

[0204] Preferably, an electronic control device of the incubator is configured or programmed to adjust the temperature control or the control of the incubator gas supply (e.g., CO2, N2, and / or O2), in particular at least one control parameter, depending on the occupancy status of the incubator. In this way, the influence of objects arranged in the interior of the incubator chamber on the response behavior of the controlled system can be taken into account. In particular, the recovery time can be reduced in the case of a larger occupancy of the interior.

[0205] The data processing device of the image acquisition system or other system is preferably separate from a first data processing device of the incubator. However, it can also be part of the control device of the incubator (also referred to as the “first control device”), which controls the functions of the incubator. The functions of the control device are implemented in particular by electronic circuits. The data processing device of the image acquisition system can have at least one CPU and optionally at least one GPU. A GPU can be provided for image processing or for carrying out deep learning processes. As an alternative to a CPU or a GPU, the data processing device can also have a dedicated chip, e.g.the Nvidia Jetson, for image processing or the implementation of deep learning processes, which can preferably be used in object tracking, especially for possible object classification or individual object recognition. Such dedicated chips can be added to the data processing device as computational accelerators. A GPU is already present on many system-on-a-chip (SoC) systems (for rendering graphics and videos). A Raspberry Pi can also have a dedicated GPU unit as part of the SoC. The object tracking system can have a control device, which can be provided separately from the first control device. The terms "control device" and "control device" are used synonymously in this description. A control device can have a microprocessor, which can contain the data processing device. The microprocessor can be of the "Raspberry Pi" type.The control device and / or the data processing device is preferably designed to carry out a control method, which is also referred to as control software or control program—in each case related to the incubator and / or the object tracking system. The functions of the incubator and / or the object tracking system and / or the control device and / or the data processing device can be described in method steps. They can be implemented as components of the control program, in particular as subprograms of the control program.

[0206] A control device—as an optional component of the system according to the invention or of the incubator—generally comprises, in particular, the data processing device or the data processing apparatus, in particular a central processing unit (CPU) for processing data and / or a microprocessor, or is the data processing device. The data processing device of the control device of the incubator can preferably also be configured to control the object tracking system.

[0207] The data processing device of the image acquisition system is preferably a device located outside the incubator chamber or the incubator, and in particular optionally separate from it, also referred to as an external device or external data processing device. The data processing device and the incubator are preferably connected by a data link and are preferably components of a data exchange network.

[0208] The at least one camera of the image acquisition system is preferably connected to the control device or data processing device of the image acquisition system via a cable connection. For this purpose, the incubator chamber has a through-opening (port) through which the cable of the cable connection is routed. A seal, in particular a silicone seal, is preferably provided to seal the port to (largely) prevent any influence on the atmosphere in the incubator. Alternatively, the camera is connected to the control device or data processing device for wireless data exchange, e.g., via Bluetooth or Wi-Fi.

[0209] The incubator may comprise a sub-housing in which, in particular, at least one control device (of the incubator and / or the object tracking system) is arranged. The sub-housing is preferably arranged at the rear of the incubator, i.e., in particular, opposite the incubator door.

[0210] The system, the incubator, and / or the image acquisition system, and / or the data processing device, and / or the control device are preferably configured to use the position data of the at least one object or a plurality of objects to create an electronic documentation file in which the positions and / or movement of the objects and / or their residence time and / or the identification data of the user initiating the movement in the incubator are logged and documented. This documentation file is then stored, in particular, in a data storage device and preferably continuously updated. In this way, "correct" handling of the objects according to standard protocols can be certified if necessary. On the other hand, deviations from standard protocols can be subsequently identified and / or information correlations can be determined.By collecting such data, the quality of cell-based laboratory work and medical, biological, and pharmaceutical procedures can be significantly improved and made more reliable. The reproducibility of cell-based laboratory work can be increased, and deviations from normal characteristics can be detected early, giving the user the opportunity to correct them or repeat the experiment at an early stage. The documentation file can be made available to the user or an external data processing device from the control device via data exchange. Such documentation is particularly useful in critical applications, such as those with a forensic relevance or where cells of significant value are being cultivated.

[0211] The invention particularly also relates to a retrofit system for incubating living cell cultures, comprising an incubator for incubating living cell cultures, which comprises: an incubator chamber for accommodating objects, in particular cell culture containers, which has opposing inner walls and a chamber opening for the insertion and removal of the objects by a user, and which has at least one storage area for storing the objects, which extends between the opposing inner walls, an incubator door for closing the chamber opening, an image processing image capture system configured for retrofitting the incubator, which has a data processing device with a data memory, an illumination device, and a camera device configured to capture at least one storage area of ​​the incubator chamber, wherein the image capture system is configured,and in particular the data processing device is programmed to

[0212] • to illuminate the storage area of ​​the incubator extending between the inner walls by means of the lighting device,

[0213] • to capture at least one image of the storage area extending between the inner walls by means of the camera device, and

[0214] • optionally: to store the at least one image in the data storage device in the form of image data by means of the data processing device, to determine at least one occupancy value from the image data, which characterizes an occupancy of the at least one storage area; to carry out at least one mathematical comparison operation, which compares the at least one occupancy value with at least one occupancy reference value; ■ to record the result of the at least one mathematical comparison operation in at least one occupancy evaluation parameter; and

[0215] ■ to store the at least one occupancy evaluation parameter in the data storage device.

[0216] The above-mentioned retrofit system is thus based on an incubator that can be retrofitted with a retrofittable image capture system, as is an integral part of the incubator in claim 1, wherein the retrofittable image capture system must be compatible with the so-called “compatible incubator.” “Retrofitting” preferably includes in each case that the camera device can be suitably arranged or fastened in the incubator chamber, that the lighting device can be suitably arranged or fastened in the incubator chamber, that in particular the data processing device with data storage can be suitably arranged or fastened in / on the incubator, that in particular the camera device and / or the lighting device and / or the data processing device with data storage are connected to a data processing device of an incubator or an external computer for the purpose of data transmission.are connectable, in particular that the camera device and / or the lighting device and / or the data processing device with data storage are connected or can be connected to a power supply, which can be part of the image acquisition system or the incubator. It can also be provided, in particular alternatively or additionally, that the data processing device of the image acquisition system is formed by a data processing device of the incubator, in that the camera device and / or the lighting device are connected or can be connected to a data processing device of the incubator for the purpose of data transmission.

[0217] Preferably, a control device of the incubator according to the invention or of the compatible incubator, which in particular can also control the atmospheric parameters in the incubator chamber (temperature, partial gas pressure CO2, H2O, etc.), or its data processing device, is configured or programmed to determine at least one operating parameter of the incubator, in particular a parameter that controls the display of information on a screen of the incubator or a parameter that is displayed on the screen of the incubator, depending on data from the image acquisition system, in particular position data or the end position of at least one object in the storage area. In particular, position data or the end position of at least one object can be displayed on the screen.

[0218] Preferably, the system for incubating living cell cultures comprises: an external device separate from the incubator and connected to the incubator for data exchange, in particular a user identification device - in particular a mobile one - and in particular a data exchange device by means of which the data processing device can exchange data with the external device, in particular can determine user identification data on the basis of the user identification device.

[0219] The invention also relates to a method for image acquisition in an incubator which serves to incubate living cell cultures and which comprises:

[0220] • an incubator chamber for receiving objects, in particular cell culture containers, which has opposite inner walls and a chamber opening for the insertion and removal of the objects by a user, and which has at least one storage area for storing the objects, which extends between the opposite inner walls,

[0221] • an incubator door to close the chamber opening,

[0222] • an image capture system, comprising o a lighting device, o a camera device and o a data processing device, the method comprising the steps of:

[0223] • Illuminating the storage area extending between the interior walls using the lighting device,

[0224] • Capturing at least one image of the storage area extending between the interior walls during illumination using the camera device in the form of image data,

[0225] • Determining at least one occupancy value from the image data that characterizes an occupancy of at least one storage area;

[0226] • Performing at least one mathematical comparison operation that compares the at least one occupancy value with at least one occupancy reference value;

[0227] • Recording the result of the at least one mathematical comparison operation in at least one occupancy evaluation parameter; and

[0228] • Storing the at least one occupancy evaluation parameter in the data storage device.

[0229] The invention also relates to an image processing image acquisition system, particularly designed for retrofitting an incubator, comprising

[0230] • a lighting device,

[0231] • at least one camera device and

[0232] • a data processing device with a data memory, wherein the image acquisition system is configured to

[0233] • to illuminate the storage area extending between the interior walls using the lighting device,

[0234] • to capture at least one image of the storage area extending between the inner walls in the form of image data using the camera device, and

[0235] • to determine at least one occupancy value from the image data that characterizes the occupancy of at least one storage area;

[0236] • to perform at least one mathematical comparison operation that compares the at least one occupancy value with at least one occupancy reference value;

[0237] • to record the result of the at least one mathematical comparison operation in at least one occupancy evaluation parameter; and

[0238] • to store the at least one occupancy evaluation parameter in the data storage device. Further preferred embodiments of the inventive objects, in particular of the inventive method, can be derived from the description of the inventive system with incubator and its preferred embodiments. Furthermore, further embodiment options of the invention emerge from the exemplary embodiments in the figures. Identical parts of the exemplary embodiments are essentially identified by identical reference numerals unless otherwise described or apparent from the context. They show:

[0239] Fig. 1 shows a system according to the invention with an incubator according to an embodiment in a perspective view.

[0240] Fig. 2 shows the incubator from Fig. 1 in a front view.

[0241] Fig. 3 shows the incubator from Fig. 1 in a front view with a graphic representation of the occupancy of the incubator chamber with objects that are highlighted in color-coded user-specific manner.

[0242] Fig. 4a shows a smartphone with camera and display 63 as an external device, which can be part of a system 400 comprising the incubator 1 from Fig. 3 and the smartphone 69.

[0243] Fig. 4b shows a legend of the color coding used in the screen of Fig. 3 to highlight user-related objects.

[0244] Fig. 5a shows a schematic side view of an image acquisition system as a component of the incubator from Figs. 1 to 4b, in an example of a chamber with a single monitored storage plate.

[0245] Fig. 5b shows a schematic side view of an image acquisition system as a component of the incubator from Figs. 1 to 4b, in an example of a chamber with multiple monitored storage plates. Fig. 5c shows a perspective view of a storage area monitored by the object tracking system of Figs. 5a and 5b, as well as the starting position P1, position changes dP, and end position P2 of a tracked object relative to a coordinate system.

[0246] Fig. 5d shows a digital image captured by the wide-angle fisheye camera of the image acquisition system used in Fig. 5a and 5b, which appears distorted due to the optics.

[0247] Fig. 5e shows the image of Fig. 5d, which was rectified by the image acquisition system using straightening algorithms.

[0248] Fig. 5f shows a still image captured by the wide-angle fisheye camera of the image acquisition system used in Figs. 5a and 5b for output on a screen of the incubator, showing the bounding boxes of the image acquisition system, identification numbers and a color coding identifying the user / owner.

[0249] Fig. 5g shows a possible screen content that can be displayed on a screen of the incubator to explain the screen page shown in Fig. 5f.

[0250] Fig. 6 shows a schematic plan view of a storage area of ​​the incubator from Fig. 1 to 5f, including objects, which is arranged in an image capture section of a camera of the image capture system.

[0251] Fig. 7 shows, based on the section from Fig. 6, the detection of an object newly placed in the incubator between two existing objects.

[0252] Fig. 8 schematically shows the sequence of an exemplary method according to the invention. Based on the occupancy of the storage area in Fig. 7, Figs. 9a and 9b show the image analysis for performing a comparison operation and determining an occupancy value parameter using segmentation, as an example of the invention.

[0253] Fig. 1 shows an incubator 1 for storing laboratory samples, more precisely a CO2 incubator for storing living cell cultures in a defined atmosphere at a controlled temperature, e.g. 37°C. For this purpose, the chamber interior 5 of the incubator is thermally insulated and can be sealed gas-tight against the environment; the gas composition in the interior is also controlled and can be changed via gas connections 43. The chamber housing 2 of the incubator stands on pedestals 44, encapsulates the interior 5 and opens into the front 3 of the incubator. The front has the chamber opening 4 through which the chamber interior 5 is accessible. A transparent inner chamber door 6 serves to close the chamber opening when the chamber door is in the closed position.In the incubator 1, the chamber housing 2 is placed within the interior of an outer housing 40, so that the chamber housing 2 and the outer housing 40 are spaced apart and thermally insulated from each other. Shelf inserts 45 and a humidifier tray 46 are visible in the chamber interior. The front side 3 of the chamber housing and the front side of the outer housing coincide in this case.

[0254] The outer incubator door 41 and the chamber door 6 are shown in an open position. The outer door 41 is pivotally mounted on hinges on the outer edge of the outer housing and has a circumferential seal, in particular a silicone seal 42.

[0255] When the outer door 41 has been opened, the inner chamber door 6 of the incubator is initially still closed. The locking device (10, 7a, 7b) serves this purpose. With the chamber door 6 closed, the user can first view the interior 5 through the transparent door panel before opening the door and inserting or removing a laboratory sample. However, opening the outer incubator door 41 already constitutes a disturbance that can potentially damage the incubator atmosphere.

[0256] The incubator has an external camera 65 installed in the door 41 and facing forward. Images from the camera can be evaluated by the incubator's appropriately programmed data processing device, in particular to identify a user by means of facial recognition. The external camera 65 connected to the data processing device serves as a user identification device 66. The latter can also be done via the camera of the smartphone 69.

[0257] To protect the stored laboratory samples, it is effective to minimize the time during which the incubator's interior is exposed to the environment (opening time intervals). The present invention is based on the observation that the opening time intervals can be reduced by an image acquisition system 200. The incubator 1 has an image acquisition system (not shown in Figs. 1, 2).

[0258] The outside of the outer incubator door has, as shown in Fig. 2, a first screen, a touchscreen 61, via which operating parameters of the incubator 1 are displayed, e.g. the temperature of the incubator atmosphere or a gas partial pressure in the interior 5.

[0259] The outside of the outer incubator door 41 has a second screen 62, which can be a touchscreen. Instead of a second screen, however, all screen outputs can also be displayed on a single screen. The data processing device (not shown) of the incubator 1 is programmed to display the occupancy of the incubator's interior on screen 62. The screen 62 serves as a "digital window" that allows the user a (virtual) view into the incubator's interior. The graphical representation of the interior or at least one storage area of ​​the incubator and its occupancy with inventory objects can be programmed such that certain inventory objects are graphically highlighted depending on certain criteria or condition parameters.

[0260] The system according to the invention essentially consists of the incubator 1, in which the data storage device and the data processing device are installed, which perform the functions:

[0261] ■ to determine at least one occupancy value from the image data that characterizes the occupancy of at least one storage area;

[0262] ■ to perform at least one mathematical comparison operation that compares the at least one occupancy value with at least one occupancy reference value;

[0263] ■ to record the result of the at least one mathematical comparison operation in at least one occupancy evaluation parameter; and

[0264] ■ to store the at least one occupancy evaluation parameter in the data storage device.

[0265] Regarding Fig. 3: The data processing device of the incubator 1 is also programmed here to display one or more objects on the display 62 depending on at least one condition parameter, which in this case depends on user identification data, according to their respective position within the incubator's interior, which was determined by the image acquisition system. The inventory objects associated with specific user identification data that identify a specific user are highlighted in a specific user-dependent color. The legend 61a for this type of color coding is displayed to the user here via the upper display 61 in its sub-area 61a. The legend 61a is shown larger in Fig. 4b: the user IDs "Jane", "Joe", etc. are assigned the corresponding highlight colors used in the display 62, 63.

[0266] Fig. 4 shows that the output display 61 and / or 62 can also - alternatively or additionally - be component(s) of an external device, in this case a smartphone 69, which is in a data exchange connection with the incubator and has the display 63, which here functions as component of an incubator system. Fig. 5a shows a schematic front view of the shelf inserts 45a and 45b of the incubator 1, which serve as storage plates for objects and are arranged one above the other. The vertical distance between such shelf inserts 45 in incubators is usually not great and is, for example, between 10 and 40 cm, in the case of incubator 1 approximately 15 cm. This means that either several cameras have to be used in order to capture the entire storage area 45, in this case the entire storage surface of the shelf insert 45b and the “air space” above it up to the shelf insert 45a. The camera 70 or camera device 70' is or includes a wide-angle orWide-angle fisheye camera with a diagonally measured angle of view of approximately 200°.

[0267] Fig. 5a shows the image acquisition system 20 installed in the incubator 1, which, in the case of the retrofit system, is also designated by the reference numeral 200. The image acquisition system 20 includes the camera 70, a wide-angle fisheye camera, which, in the field of view or angle of view 71a of preferably 160° to 220°, captures the storage area of ​​the shelf insert 45b located below it and a large part of approximately 80% of the surfaces of the incubator inner wall sections 72a, 72b, which, together with the storage plates 45a and 45b extending between the inner walls 72a and 72b, delimit the compartment 73 of the incubator chamber. The wide viewing angle makes it possible to use a single camera to capture the entire storage area of ​​the lower shelf insert 45b, in particular the air space into which the (stock) objects 80' and 80 protrude, namely a stack 80' of cell culture containers and a cell culture container 80.The nominal viewing angle of the wide-angle fisheye camera is 200°, but only an image area is evaluated that corresponds to a viewing angle taken from the range of preferably 160° to 170°.

[0268] The camera is arranged vertically above the geometric center of the storage surface of the shelf insert 45b. The image acquisition system 20 also includes the lighting device 90, the control device 23, which has a data processing device 21 and a data memory 22 as further components of the image acquisition system 20. The data processing device 21 and the control device 23 are connected to the camera 70 and the additional cameras not shown in Fig. 5a via a cable connection 25, which enters the incubator chamber through the port 47 in the incubator chamber rear wall. These cameras are each provided to monitor all storage areas (all top sides of shelf inserts 45, see Fig. 1). The control device 23 also has a data interface 24, via which a data connection to other incubator device components is enabled, e.g., to output data or signals to a display 61, 62, 63 of the incubator.A lighting device 90 with a plurality of LEDs 90', 90" is mounted above the storage plate 45b and connected to the control device 23 via lines 25. Instead of the two LEDs shown, a plurality of LEDs can be provided. Using the optional lighting device 90, the storage area 45b can be illuminated for image capture purposes, if appropriate.

[0269] Fig. 5b shows a schematic side view of an image acquisition system as a component of the incubator from Figs. 1 to 4b in an example of a chamber with multiple monitored support plates 45a, 45b, 45c. The illustration is an extension of the principle from Fig. 5a, in which the incubator chamber is divided into several compartments 5a, 5b, and 5c, which are arranged one above the other here and connected for gas exchange, which is formed via holes in the support plates 45a, 45b, 45c. The storage area or storage plate 45a in compartment 5a is monitored by camera 70', the storage area or storage plate 45b in compartment 5b is monitored by camera 70, and the storage area or storage plate 45c in compartment 5c is monitored by camera 70", wherein cameras 70' and 70" are designed and arranged analogously to camera 70 in Fig. 5a.All cameras are connected to the control device 23 via a connecting cable bundle 26 inside the incubator chamber, which merges into the cable connection 25 already shown in Figure 5a and exits the incubator chamber through port 47 in the rear wall of the incubator chamber. The control device 23's data processing device 21 is configured to monitor all objects in all three compartments 5a, 5b, and 5c. The image acquisition system 20 of the incubator shown in Figure 5b comprises three cameras 70, 70', 70", the illumination device 90, the data processing device 21, the data storage device 22, and the connecting lines.

[0270] Fig. 5c shows a perspective view of a compartment 5b or storage area 45b monitored by the image acquisition system of Figs. 5a and 5b, as well as the object positions P1, P2 relative to a Cartesian coordinate system (x, y, z). If the image acquisition system is implemented as an object tracking system by capturing video data and using digital image processing, position changes dP of an object moving along its movement path B can also be tracked. The origin of the coordinate system can be fixedly located in a corner of the compartment.

[0271] Fig. 5d shows a digital image captured by the wide-angle fisheye camera of the image acquisition system used in Fig. 5a and 5b, which appears distorted due to the optics.

[0272] Fig. 5e shows the image of Fig. 5d, which was rectified by the image acquisition system using straightening algorithms.

[0273] Fig. 5f shows a still image captured by the wide-angle fisheye camera of the image acquisition system used in Figs. 5a and 5b for output on a screen of the incubator, showing the bounding boxes of the image acquisition system, identification numbers and a color coding identifying the user / owner.

[0274] Fig. 5g shows a possible screen image that can be displayed on an incubator monitor to illustrate the screen page shown in Fig. 5f. In addition to identifying the objects by identification numbers, a color coding identifying the user / owner is also shown, as well as the times of placing the objects in the incubator chamber, which can optionally be recorded by the incubator.

[0275] Fig. 6 shows a storage area, namely the top side of the shelf insert 45b, from a bird's eye view or top view. Also shown schematically is the image capture section 71 captured by the camera 70. The image capture section 71 is the area captured by the camera 70 in one or every image, because the camera 70 does not change its viewing angle or position here. Thus, every image shows this section 71. In the figures, the lower edge of the section 71 represents the area located near the incubator chamber opening 4. The camera 70 and / or the lighting device 90 with the two light sources (LEDs) 90' and 90" can, however, also be mounted so as to be movable or displaceable by means of the transport device 95, here a motorized rail system.

[0276] The image acquisition system 20, 200 is set up here to

[0277] • to illuminate the storage area 49 extending between the inner walls 72a, 72b by means of the lighting device 90,

[0278] • to capture at least one image of the storage area 49 extending between the inner walls 72a, 72b by means of the camera device 70, and

[0279] • to store the at least one image in the form of image data in the data storage device 22 by means of the data processing device 21.

[0280] The image acquisition system is also designed to

[0281] • to illuminate at least two objects 80, 80' arranged on this storage area 49 by means of the lighting device 90,

[0282] • to capture an image of the at least two objects 80, 80' on this storage area 49 by means of the camera device 70 - that is to say an image in which these at least two objects are shown - and

[0283] • to store the image of the at least two objects 80, 80' and the storage area 49 in the form of image data in the data storage device 22 by means of the data processing device 21.

[0284] The data processing device is optionally programmed to

[0285] • by evaluating the image data, to distinguish the first object 80 and the second object 80' shown in the image (in particular: to assign different identification data to the first and second object; to count the objects; to recognize the object class; to analyze, store, recognize individual features; to track objects during movement), in particular to capture the outlines of the first 80 and the second object 80' in the image by means of image processing algorithms, and

[0286] • in particular, to store information about the first 80 and second object 80', in particular the outlines of the first 80 and second object 80', in the form of object data in the data storage device 22. The illumination device 90 is optionally operable here, and the data processing device is optionally programmed here to operate the illumination device 90 in at least two different illumination modes, and the image acquisition system 20, 200 is configured to

[0287] • to illuminate the storage area 49 of the incubator chamber by means of the lighting device 90 o initially in a first lighting mode, in particular with active LED 90' and inactive LED 90", and o then in a different second lighting mode, in particular with active LED 90" and inactive LED 90',

[0288] • to capture at least one image of the storage area 49 during illumination by means of both the first and the second illumination mode by means of the camera device 70, and

[0289] • to provide the at least one image in the form of image data containing combined image information acquired during both the first and the second illumination mode, wherein the data processing device is programmed to execute an image evaluation program which obtains the combined image information from the image data.

[0290] Preferably, the at least one image of the storage area 49 contains at least a first image of the storage area 49 and a second image of the storage area 49 which is different therefrom, wherein the first image is captured in the first illumination mode and the second image is captured in the second illumination mode, and the first image is provided in the form of first image data and the second image is provided in the form of second image data, wherein the data processing device and the image evaluation program are programmed such that

[0291] • the first image data and the second image data are combined to obtain combined image data, which is created in particular by adding and / or averaging first and second image data, and

[0292] • the combined information is obtained from the combined image data. The data processing device of the image acquisition system 20, 200 is programmed to capture and evaluate the image data using the camera 70 during illumination, depending on the detection of the closed state of the incubator's outer door 41. By comparing chronologically successive images, it can be determined whether a new object 81 has entered the camera field 71.

[0293] Fig. 7: An image containing a newly appearing outline 81a in the section 71, which is attributable to the object 81 introduced into the interior, is considered a modified image. Based on this modified image, identification data is assigned to the newly appearing outline 81a, assuming that it is a new object 81 to be introduced into the incubator.

[0294] Fig. 8 shows the sequence of the method according to the invention, which was also indirectly described in the above description of the previous figures.

[0295] The method 300 is for image acquisition in an incubator which is used for incubating living cell cultures and which comprises:

[0296] • an incubator chamber for receiving objects, in particular cell culture containers, which has opposite inner walls and a chamber opening for the insertion and removal of the objects by a user, and which has at least one storage area for storing the objects, which extends between the opposite inner walls,

[0297] • an incubator door to close the chamber opening,

[0298] • an image capture system, comprising o a lighting device, o a camera device and o a data processing device with a data memory, wherein the method 300 comprises the program-controlled steps:

[0299] • Illuminating the storage area extending between the interior walls by means of the lighting device, (301) • Capturing at least one image of the storage area extending between the interior walls during the illumination by means of the camera device, (302)

[0300] • optionally: storing the at least one image in the form of image data in the data storage device, (303)

[0301] • Determining at least one occupancy value from the image data that characterizes an occupancy of the at least one storage area; (310)

[0302] • Performing at least one mathematical comparison operation that compares the at least one occupancy value with at least one occupancy reference value; (311)

[0303] • detecting the result of the at least one mathematical comparison operation in at least one occupancy evaluation parameter; (312) and

[0304] • Storing the at least one occupancy evaluation parameter in the data storage device (313).

[0305] In particular, step 310 for determining the occupancy value A1 preferably includes at least one of the following steps, which can also be carried out repeatedly if necessary:

[0306] • Determining a contiguous free surface segment 101 (area not occupied by an object), optionally determining several contiguous free surface segments by means of computer-aided image processing, in particular using segmentation;

[0307] • Determine the area A1 (e.g. measured in cm 2 ) of segment 101 ;

[0308] • Determining a contiguous free surface segment 101 that is larger than an area A2, optionally: determining a plurality of contiguous free surface segments that are each larger than an area A2, by means of computer-aided image processing, in particular using segmentation;

[0309] • Determining a contiguous free surface segment 101 that is larger than an area A2 and whose shape corresponds to a predefined dimension, in particular a suitable footprint A2 with a suitable dimension for placing a laboratory sample container. The suitable dimension M can be taken from a group of different suitable dimensions M that can be stored in the data storage device. A suitable dimension is assigned to one of several laboratory sample containers, e.g., an SBS standard microtiter plate, cell culture flasks of different sizes, or Petri dishes of different sizes.

[0310] The step 311 of performing a mathematical comparison operation that compares the at least one occupancy value with at least one occupancy reference value includes in particular the following step:

[0311] • Determine using the mathematical comparison function V=V(A1 ; A2) whether the following applies to the areas A1 and A2: A1 >= A2.

[0312] The step 312 of detecting the result of the at least one mathematical comparison operation in at least one occupancy evaluation parameter includes in particular the following step:

[0313] • Determine the occupancy evaluation parameter PBB = V = (0 or 1 ), which here is of type Boolean, where PBB=1 if the area A1 is greater than or equal to the area A2, and where PBB=0 if the area A1 is less than the area A2.

[0314] Figure 9a shows the situation from Figure 7 in an image 71 of the storage area 45b, which is evaluated using computer-aided image processing methods. By segmentation, the area segment 101 that is not occupied by a cell culture container 80', 81, 80 is identified. The area of ​​this area segment 101 is determined to be A1. The total area of ​​the storage area can be divided into a grid for computational processing, and the positions of the grid points can be recorded in a Cartesian coordinate system (x; y). A grid area or grid point can be assigned the occupancy attribute BA (occupied or free). An occupied area with a specific size can be assigned to an object. A free area with a specific size A2 can be assigned to a free parking space 102. Free parking spaces 102, 104 can be successively determined by determining the free segment area 101, 10T and comparing it with at least one value A2 orA4 and a suitable dimension (Fig. 9b). This automatic allocation can be continued until the area of ​​storage area 45b is optimally utilized by storage spaces 102, 104, ..., which can be noted as "free, reserved," or "occupied." Such automated planning can be used to implement a user guidance system.

[0315] As an alternative to a dynamic allocation of storage spaces 102, 104 to the area of ​​the storage area, a grid with predefined storage spaces could also be used, which are then recognized as “free” or “occupied” by image capture.

[0316] Preferably, the method 300 also includes the steps of:

[0317] Monitoring, over time, the incubator chamber 2, 5 by means of at least one camera 70 of the camera device 70' of the incubator, which is arranged to record at least one storage area 49 in the interior of the incubator chamber, into which the at least one object 80; 80'; 81 is introduced; (304)

[0318] Assigning identification data to the at least one object 80; 81, which is captured in an image of the storage area 49 taken by the at least one camera 70 after it has been positioned in the storage area; (305) Storing the position of the at least one object 80; 81 in the storage area 49 as a function of the identification data of the at least one object as ID position data in the data memory. (306)

[0319] Preferably, the method 300 also includes the steps:

[0320] Reading in user identification data identifying the user of the incubator 1 who introduced the at least one object 80; 81 into the incubator chamber by means of a user identification device (307) and

[0321] Storing user identification data in a data storage of the incubator. (308)

[0322] Preferably, the method 300 also includes the step:

[0323] Storing the ID position data depending on the user identification data as user-related ID position data. (309)

Claims

Patent claims System for monitoring the occupancy of storage space in at least one incubator for incubating living cell cultures, comprising at least one incubator for incubating living cell cultures, comprising: an incubator chamber for receiving objects, in particular cell culture containers, which has opposite inner walls and a chamber opening for the supply and removal of the objects by a user, and which has at least one storage area for storing the objects, which extends between the opposite inner walls, an incubator door for closing the chamber opening, an image acquisition system, comprising • a lighting device, • at least one camera device and • a data processing device, wherein the image acquisition system is configured to • to illuminate at least one storage area extending between the interior walls by means of the lighting device, • to capture at least one image of the at least one storage area extending between the inner walls in the form of image data by means of the camera device, wherein the system comprises a data storage device and a programmable data processing device which is programmed to: determine at least one occupancy value from the image data, which Occupancy of at least one storage area; • to perform at least one mathematical comparison operation that compares the at least one occupancy value with at least one occupancy reference value; • to record the result of the at least one mathematical comparison operation in at least one occupancy evaluation parameter; and • store the at least one occupancy evaluation parameter in the data storage device. The system according to claim 1, wherein the data processing device is programmed to: * to determine from the image data, as an occupancy value, a preferably contiguous, and in particular unoccupied, sub-area of ​​a storage area of ​​the at least one storage area, and to determine the size of this free storage area. System according to claim 2, wherein the data processing device is programmed to: * perform the mathematical comparison operation in which the size of the free storage area is compared with a reference value of a free storage area. The system according to claim 3, wherein the free storage area is the footprint of a storage space for a laboratory sample container, and wherein the occupancy reference value is a predetermined comparison value suitable for this footprint. The system according to claim 4, wherein the data processing device is programmed to: * to assign parking space ID data to the parking space, which uniquely identifies this parking space from other parking spaces. System according to claim 4 or 5, wherein the data processing device is programmed to: • to determine the availability of a parking space depending on the occupancy assessment parameter, i.e. to answer the question whether the parking space is free or occupied, and in particular to save the answer to this question as occupancy assessment data.

7. System according to one of the preceding claims, wherein the data processing device is programmed to: * To set occupancy rating data depending on the occupancy rating parameter * provide the user with information about the occupancy rating data via a user interface device.

8. The system according to claim 7, wherein the information about the occupancy rating data is one of the following: * the number of free spaces for laboratory sample containers in a predetermined space determined using the mathematical comparison operation; * the number of occupied spaces for laboratory sample containers in a predetermined space determined using the mathematical comparison operation; * exceeding or falling below a threshold value which characterises an under-utilisation or an overload of at least one storage area of ​​the incubator.

9. System according to one of the preceding claims, wherein the data processing device is programmed to: * To generate incubation report data for an individual sample contained in an individual laboratory sample container by assigning a sample ID to the laboratory sample container that uniquely identifies the laboratory sample container and a slot ID to the laboratory sample container.

10. System according to one of the preceding claims, comprising a camera device which can in particular be the camera device of the image capture system, by means of which at least one image of at least one parking space of at least one storage area can be captured from the image data, in particular in order to capture image documentation of the parking space. System according to one of the preceding claims, which has a positioning guidance system which has a lighting device by means of which an area or spot in the storage area, in particular a parking space, can be specifically illuminated. System according to claim 11, wherein a data processing device of the incubator, or the data processing device, is programmed to set the target of the lighting as a function of sample ID data and / or parking space ID data and to exert directed lighting on this target or this parking space.A system according to one of the preceding claims, comprising a user guidance system having a lighting device and configured to assist in positioning laboratory sample containers on and / or between locations of the incubator by illuminating at least one location and / or a laboratory sample container placed on a location according to a predetermined schedule. A system according to one of the preceding claims, comprising two or more incubators connected to one another for the purpose of data exchange, each of the incubators comprising: an incubator chamber for accommodating objects, in particular cell culture containers, which has opposing inner walls and a chamber opening for the addition and removal of the objects by a user, and which has at least one storage area for storing the objects, which extends between the opposing inner walls. an incubator door for closing the chamber opening, an image acquisition system, comprising • a lighting device, • at least one camera device and • a data processing device, wherein the image acquisition system is configured to • to illuminate at least one storage area extending between the interior walls by means of the lighting device, • to capture at least one image of the at least one storage area extending between the inner walls in the form of image data by means of the camera device, wherein the system comprises a data storage device and a programmable data processing device which is programmed to: • to determine at least one occupancy value from the image data of each of the at least two incubators, which characterises an occupancy of at least one storage area of ​​the respective incubator; • to perform at least one mathematical comparison operation that compares the at least one occupancy value with at least one occupancy reference value; • to record the result of the at least one mathematical comparison operation in at least one occupancy evaluation parameter; and • storing the at least one occupancy evaluation parameter in the data storage device. A method for image acquisition in an incubator used to incubate living cell cultures, comprising: • an incubator chamber for holding objects, in particular cell culture containers tern, which has opposite inner walls and a chamber opening for the supply and removal of the objects by a user, and which has at least one storage area for storing the objects, which extends between the opposite inner walls, • an incubator door to close the chamber opening, • an image capture system, comprising o a lighting device, o a camera device and o a data processing device with a, the method comprising the steps of: • Illuminating the storage area extending between the interior walls using the lighting device, • Capturing at least one image of the storage area extending between the interior walls during illumination by means of the camera device in the form of image data, • • Determining at least one occupancy value from the image data that characterizes an occupancy of at least one storage area; • Performing at least one mathematical comparison operation that compares the at least one occupancy value with at least one occupancy reference value; • Recording the result of the at least one mathematical comparison operation in at least one occupancy evaluation parameter; and • Storing the at least one occupancy evaluation parameter in the data storage device.

Images