Incubator and method

Abstract

The present invention relates to an incubator for live cell cultures, a method of working with the incubator, and a system comprising the incubator.An imaging system is used to determine the occupancy of the incubator using a mathematical comparison operation.

Description

[Technical Field]

[0001] The present invention relates to a system comprising an incubator for live cell culture and to a method of working with this system. [Background technology]

[0002] Such incubators allow cells in cell culture to be maintained under controlled ambient conditions in biological and medical laboratories, enabling the growth of live cells in test tubes. To this end, the temperature and atmospheric gas composition or humidity within the incubator chamber, which is insulated from the environment, are maintained at desired values by the incubator's mechanical means. Eukaryotic cells require CO2 incubators. The atmosphere is formed by air with a specific CO2 and O2 content and a specific humidity, and the appropriate temperature is often 37°C.

[0003] Cell growth is critically dependent on constant atmospheric conditions, especially within the incubator. Disturbances to the atmosphere within the incubator can have a detrimental effect on cell growth. In an "ideally" equipped laboratory in this regard, each user would be provided with an individually accessible incubation chamber for each sample to be cultured. However, this is not realistic for reasons of cost efficiency. In practice, laboratories typically have only one incubator (or a small number of incubators) for use by multiple users, each with a single incubation chamber and one or more storage areas on one or more storage plates within the incubation chamber.

[0004] The frequency with which the incubator chamber door is opened, and therefore the frequency with which it interferes with the incubator's controlled atmosphere, varies with the number of users and the number of samples being incubated therein. Furthermore, the intensity of the interference also depends on the length of time the door is open: the longer users have access to the interior of the incubator chamber, the longer the door is open.

[0005] There are various usage scenarios for the incubator, which together can increase access times.

[0006] Scenario A) A new object is placed in the incubator. When a user places one or more objects, especially cell culture vessels, into the incubator, they need free storage space in the storage area. If free storage space is inaccessible due to disorganized storage, the user will need time to make this space available. The more carefully the user moves or rearranges the objects already in the incubator (existing objects), and possibly even records this on a file, the more time the task takes. If it turns out that there is not enough storage space in the incubator chamber, the user will repeat the process in another compartment of the incubator chamber or in a spare incubator that may be in the lab. This increases the time the incubator door is open, and therefore the time the chamber interior is exposed to the environment (exposure time).

[0007] Scenario B) Checking cell cultures. When a user checks a cell culture that was previously placed in an incubator, for example to determine the quality and growth status of the cell culture medium, the user must first search for the cell culture vessel in the incubator, which increases exposure time. The more carefully the user moves or rearranges existing objects, the longer the search process will take.

[0008] Scenario C) Removing an object from the incubator. In this case, the user must first search for the object. This is the time delay factor mentioned in B).

[0009] Furthermore, the more frequently an incubator is opened, the higher the risk of its interior becoming contaminated. For example, in forensic science or regenerative medicine, the value of individual samples, especially the cells contained in cell culture vessels, can be estimated to be much higher than the value of the entire incubator, and loss of samples due to contamination is absolutely necessary. In any case, frequent contamination increases the risk of work stoppages, increases costs, and requires additional maintenance. If an incubator becomes contaminated, the chamber must be cleaned and sterilized before it can be used again. During this time, work with cell cultures is interrupted unless a spare incubator is available.

[0010] Therefore, it is essential in laboratories to minimize the time that the incubator door is open and to minimize the frequency with which the incubator chamber is opened. To this end, the subsequently published European patent application with application number 21153810 describes an incubator equipped with an imaging system for detecting storage areas within the incubator and outputting information about the occupancy status of the incubator. This functionality makes it possible to limit user access to the bare minimum and reduce exposure time, i.e., the time that the incubator door is open and thus the time that the interior of the incubator is exposed to the incubator environment.

[0011] The present invention is a further development of this approach. Summary of the Invention [Problem to be solved by the invention]

[0012] The object of the present invention is to provide a solution for enabling efficient utilization of incubators, in particular for reducing the risk of contamination in incubators. [Means for solving the problem]

[0013] The above problem is solved by a system according to claim 1 and a method according to claim 15. Advantageous configurations are in particular the subject matter of the dependent claims.

[0014] The system according to the invention for monitoring the occupancy of storage locations in at least one incubator for cultivating live cell cultures comprises: The apparatus includes at least one incubator for culturing live cell cultures, the incubator having an incubator chamber with opposing interior walls, a chamber opening for a user to place and remove objects from, and at least one storage area extending between the opposing interior walls for storing objects; an incubator door for closing the chamber opening; an imaging system, A lighting device; at least one camera device; a data processing device, in particular comprising a storage device; The imaging system comprises, in particular, illuminating at least one storage area extending between the interior walls with a lighting device; capturing, by a camera device, at least one image in the form of image data of at least one storage area extending between the interior walls; Preferably, the at least one image is stored in the form of image data by the data processing device in the data storage device; The system includes a data storage device and a programmable data processing device, the programmable data processing device comprising: determining at least one occupancy ratio characterizing occupancy of the at least one storage area from the image data; performing at least one mathematical comparison operation comparing the at least one occupancy rate to at least one occupancy criterion value; incorporating a result of the at least one mathematical comparison operation into at least one occupancy evaluation parameter; It is programmed to store the at least one occupancy assessment parameter in a data storage device.

[0015] The imaging system allows for reproducible imaging of information about the occupancy of storage areas that can be used for various purposes within the incubator chamber under controlled conditions. The incubator according to the present invention provides information about the occupancy of the storage areas in the form of image data, allowing users to access information about the occupancy status of the incubator before opening it. Providing such information reduces unnecessary opening of the incubator, thereby making the incubator more efficient to use.

[0016] Providing information about the occupancy of a storage area can be achieved by presenting an image of the storage area to the user, for example by displaying it on the incubator's display. In this way, the user can immediately have an "image" as to whether the occupancy of the storage area allows for the addition of further objects, and whether and where an object that the user has added and subsequently wishes to check or remove (if it is identifiable to the user) is located in the corresponding storage area. This information can also be provided by using automatic image evaluation to obtain data about the free storage locations in this storage area and convey this to the user. The availability of image data also makes it possible to recognize object classes and individual object characteristics.

[0017] By providing automatable occupancy assessment in the form of occupancy assessment data, incubators can be utilized more efficiently.

[0018] The occupancy rate is, for example, the area of a sub-area of at least one storage area, which sub-area may have a predetermined shape, preferably a rectangular shape. Such a sub-area can be determined from the image data using an image processing algorithm, in particular a segmentation method. The sub-area may correspond to an installation location.

[0019] The occupancy criterion is a predetermined area stored in a data storage device suitable for use in mathematical comparison operations. The choice of occupancy criterion depends on the context of the application.

[0020] The mathematical comparison operation is a computer algorithm that outputs the result of the mathematical comparison as "less than," "equal," or "greater than." This result is stored as an occupancy evaluation parameter. The value is preferably a Boolean value, i.e., a Boolean data type used in programming to represent the logical truth values "true" and "false." However, the result of the comparison may have more than two possible values, particularly a size classification performed by a multi-level comparison. This value can be used to determine an area class. The area class may be typical or standardized for a particular laboratory sample container. A subarea whose area is smaller than the area of a first size class can also fall into a second size class having a smaller area (than the area of the first size class). Such a subarea can be identified as suitable for use as a location for laboratory sample containers that fall into the second size class.

[0021] Such an occupancy assessment can be based in particular on the observation of one, in particular precisely one, storage area in the incubator, and in particular on the assessment of the free storage space, also called the installation location, for each individual experimental vessel that requires a surface. In particular, the occupancy assessment can reduce the risk of users individually erroneously opening the incubator door, who individually assess the occupancy, in particular the graphical display of the storage areas on the incubator monitor. They do so in order to place an experimental vessel in a storage space they believe is free, only to discover that this free space is too small for the experimental vessel and therefore opening the door is pointless, resulting in an additional exposure time for samples already cultivated in the incubator and an increased risk of contamination of the incubator interior.

[0022] Such occupancy assessment may be based in particular on observation of one or more storage areas, particularly several or all storage areas, of the incubator, and may in particular be based on an assessment of available storage space for one or more laboratory vessels requiring a footprint. The occupancy assessment may be utilized to assess the occupancy or available capacity of at least one storage area. The occupancy assessment may be utilized to determine the number of available and / or occupied storage spaces.

[0023] To determine whether a location or locations in the (total) available storage space of a storage area are free (and / or occupied), it is possible to determine, in particular, whether the surface of one or more surface elements of the available storage space is larger than a comparison surface, i.e., a desired surface for one or more storage locations, which may in particular include at least one predetermined size and at least one predetermined shape. Images of the storage area can be used to determine available / occupied storage locations. Images of the storage area taken from a bird's-eye view are particularly easy to evaluate. Available or occupied storage locations can be easily determined by computer-aided image processing, in particular by algorithms for image segmentation. In this way, the operating status of incubators, in particular, can be tracked. Based on the detection of available / occupied individual storage locations, capacity statistics can be determined. This allows laboratory managers to optimize resource planning and utilization, in particular to plan the processing of large numbers of experimental samples using one or more incubators or one or more incubators each equipped with the system according to the invention, as well as one or more laboratories. Hereinafter, the system according to the invention may also be referred to as the "system" for short.

[0024] The detection of vacant / occupied locations may involve the data processing device being programmed to assign a location ID to the detected locations in the form of location ID data. The location ID data may include information about the location of the individual locations in a storage area of an incubator (location location data), where the storage area may also be identified by a storage area ID and the incubator by an incubator ID. Information about the storage area ID and / or the incubator ID may also be part of the storage area ID data. The location ID data may enable a more efficient utilization of one or more incubators in the system according to the invention. In this way, the user receives information about the presence of vacant storage locations in the incubator and / or the number of specific vacant / occupied locations, as well as information about whether the individual locations are vacant or occupied. The location ID data may be part of the occupancy status data, as described below.

[0025] The locations within an incubator differ from one another in certain characteristics. For example, laboratory vessels located near the incubator door are easily accessible, but may be moved or displaced by other users more frequently than vessels located further inside the chamber. Since a particular goal in laboratories is the reproducibility of experiments and work processes, the use of individualized locations helps to capture and store accurate information about the individual locations of biological samples within laboratory vessels, thereby making it available for other uses, particularly for recording or statistical analysis. In this case, advantageous designs can identify individualized locations with specific class characteristics for users. For example, locations near the rear wall are more protected from frequent access by other users than locations near the door. Samples that require frequent testing are preferably positioned near the door rather than the rear wall of the chamber. Using a laboratory vessel positioning and guidance system, users can be efficiently guided to position laboratory vessels in pre-detected and predetermined vacant locations. A positioning and guidance system that operates using lighting is described below.

[0026] Individual location occupancy detection is performed as follows: A location is uniquely identified by location ID data. An empty location is defined as a surface of empty storage locations that meets at least one predetermined criterion. That is, a continuous surface in the storage area that has a predetermined shape, e.g., rectangular, square, or circular, and a size that exceeds a predetermined minimum area SPmin and, optionally, does not exceed a predetermined maximum area SPmax. This particularly means that a location has a predetermined size, but does not necessarily have a predetermined location in the storage area. However, it is also possible and preferred for a location to exist. In particular, it is possible to know the classes of locations that correspond to the sizes and classes of commercially available cell culture vessels. Each individual location can be uniquely identified by location ID data. The location ID data does not necessarily allow for the identification of a location in the storage area. It may simply be a link to an incubator (incubator ID) and / or a storage area of the incubator (storage area ID). The prerequisite is preferably the determination of at least one empty location, which can be determined by an occupancy evaluation and a comparison with a reference value.

[0027] The preferred embodiments and developments of the system according to the invention or of its occupancy evaluation described herein are preferably implemented by a data processing device programmed to carry out the corresponding functions, which can be explained by way of example in that, in order to implement individual location occupancy detection, the data processing device is preferably programmed to assign parking space ID data to a parking space that uniquely identifies an installation location previously determined by image evaluation.

[0028] Image processing-based segmentation techniques, which can be used to realize computer-aided image evaluation for detecting vacant or occupied storage spaces / locations, are fundamentally known in many image processing applications. Segmentation is based on image processing or evaluation of individual or video image data. Such segmentation methods can be implemented using relatively simple means, such as appropriate cameras and image processing algorithms. The theoretical foundations of segmentation and their practical implementations are known (e.g., "Clustering Techniques for Image Segmentation," Siddiqui and Yahya, Springer International Publishing, 2021). Ready-to-use image processing algorithms for segmentation are also freely available (OpenCV.org) and well documented. OpenCV (short for Open Computer Vision) is a free program library (BSD license) with algorithms for image processing and computer vision. The OpenCV program library also includes functions, libraries, and interfaces for utilizing the segmentation function library, particularly OpenCV, cv2, matplotlib, numpy, and scikit-image.

[0029] The occupancy assessment may be based in particular on the observation of one or more storage areas, in particular of several or all storage areas of one or more incubators of a system according to the invention, and may in particular be based on the assessment of available storage space for one or more laboratory vessels requiring installation surfaces.

[0030] The occupancy assessment can be based in particular on the observation of one or more storage areas of the system according to the invention, in particular several or all storage areas of one or more incubators. The occupancy assessment can compare the footprint corresponding to one or more installation locations for laboratory vessels with an occupancy reference value set for this situation. The occupancy assessment can compare the (total) free storage space of one or more storage areas of one or more incubators of the system according to the invention with an occupancy reference value set for this situation. Here, the occupancy assessment does not specifically determine the number of individual installation locations that are available or occupied, although this is fundamentally possible. Instead, information is preferably obtained and provided as to whether the free storage space is below / above a predetermined minimum / maximum value (herein " / " means "and / or"). For example, if the (total) free storage space is greater than a predetermined maximum value, information can be obtained as to whether the minimum utilization rate of the incubator will be achieved / below. For example, if the free storage space is less than a predetermined minimum value, information can be obtained as to whether the minimum utilization rate of the incubator will be achieved / exceeded. When one or more incubators are included in the consideration of the total available storage space in the system, information can be obtained about whether the critical utilization rate of a complex incubator is reached / under / exceeded. Such complexes can be formed by one or more laboratory spaces or incubators within a laboratory. In this way, laboratory managers can obtain valuable information regarding resource planning / utilization in one or more laboratories.

[0031] In a preferred application, occupancy assessment can be used to detect whether a minimum utilization rate is met / falling short in order to plan the cleaning of one or more incubators. Since incubators must be empty during cleaning, occupancy assessment helps to identify the right time to clean an incubator, since the fewer remaining samples that need to be removed and moved, the less effort is required. In the case of multiple incubators, occupancy assessment helps to select the most suitable incubator for cleaning. In this case, the (total) free storage space of the individual incubators is compared with each other to determine the incubator with the most free storage space.

[0032] The system can be configured to generate culture report data for each sample contained in each laboratory sample container (cell culture container). To this end, the sample / laboratory sample container is assigned a sample ID or sample ID data, preferably capable of uniquely identifying the sample / container, using programming and computer-assisted operations. As the laboratory sample container is placed in a location in the incubator, the sample ID can be assigned a location ID (location ID data), which is then stored together in the culture report (culture report data). If the laboratory sample container is tracked using an object tracking system, the sample ID data can be assigned movement history data, which can include, in particular, time-dependent information regarding the movement history.

[0033] Assuming that laboratory sample containers essentially always remain in their designated locations, object tracking is not necessarily required to obtain movement data for individual laboratory sample containers. Therefore, by observing the location and observing the time-dependent image changes at this location, the dimensions of (unwanted) laboratory sample container movements can be obtained and saved as movement data. These data are also assigned to sample IDs and saved in the culture report data. Time-dependent sensor data obtained from incubator sensors, particularly regarding gas concentrations inside the chamber (CO2, O2, N2, HO) and / or the chamber's internal temperature and / or the number and duration of door openings, can also be assigned to sample ID data to obtain culture report data. This culture report data can be used to record sample processing in detail, providing a basis for high reproducibility of experiments and sample processing.

[0034] The incubator may have a camera device, particularly a camera device of an imaging system, which can capture at least one image of at least one location in at least one storage area. In particular, image files created for each location identified by location ID data can be captured and stored according to location ID data and / or sample ID data. The image files may be video files. The operation of the camera device and the imaging can be controlled according to a door sensor to capture images only when at least movement of a laboratory sample container is possible. The image files can be stored as part of the culture report data.

[0035] The system according to the invention may comprise a positioning and guidance system, which preferably comprises a lighting device capable of illuminating an area or spot in the storage area, in particular an installation location, which lighting device may also be formed by a lighting device that is part of the imaging system.

[0036] The lighting device is preferably arranged in the incubator chamber so as to illuminate a plurality of areas, in particular all areas, in particular substantially all areas, of at least one storage area, the light being preferably irradiated into the storage area and reflected and / or scattered therefrom towards the user, i.e. towards the door opening (incubator door).

[0037] The data processing device or data processing unit of the incubator is preferably programmed to control the operation, i.e. the time sequence of switching on / off and / or the lighting color and / or the lighting intensity and / or the pulsing of the lighting device.

[0038] The data processing device or data processing unit of the incubator is preferably programmed to set a lighting target depending on the sample ID data and / or the location ID data and to apply directional lighting to this target or this location. The lighting device is preferably configured to set a lighting target and to apply directional lighting to this target or this location. For this purpose, the lighting device can have an electrically operable movable device, which is equipped with at least one light source, in particular an electrically driven light source (e.g. an LED) or at least one emitting optical fiber, and is configured to illuminate the various locations in a targeted manner.

[0039] The lighting device may be positioned in the storage area, particularly at the side or top of the installation location, so that the light emitted therefrom is directed into the viewer's eyes.

[0040] The system according to the invention may comprise a user guidance system configured to assist in positioning laboratory sample containers on and / or between the incubator locations by illuminating at least one location and / or laboratory sample containers arranged on the location according to a predetermined sequence plan (sequence plan data).

[0041] For this purpose, the data processing device or data processing unit of the incubator may: allowing a user to select or input a sample ID using a user interface device; and / or assigning to this sample ID a location ID data, also called reserved location ID data, according to a predetermined sequence plan that defines in particular the time of the incubation of the sample, It is preferable that the lighting device is programmed to control the lighting of the predetermined reserved installation location according to a predetermined sequence plan in response to the sample ID and reserved installation location ID data.

[0042] In particular, location location data defining the location of (vacant or occupied) surfaces on the storage area may also be used as well as object location data. The location location data is preferably also part of the location ID data.

[0043] The footprint preferably has a predetermined size A2 (e.g., measured in square centimeters) and / or predetermined dimensions M, which may include information about the shape, but a rectangle may be assumed as the default shape. For example, microtiter plates and cell culture bottles have substantially rectangular footprints, while petri dishes and other bottle shapes may have circular footprints. Various sizes A2 and dimensions are provided, corresponding to specific commercially available objects, and can be stored in a data storage device as comparison or occupancy standards for comparison or automated image evaluation. For practical handling, a certain margin of footprint space, e.g., a border of width d (e.g., 2 cm), is also required, and this is considered appropriate for the footprint used for a particular cell culture vessel. SBS standard microtiter plates have the dimensions length x width ("external dimensions of the base footprint") specified in the ANSI / SBS 1-2004 standard: {127.76 mm ± 0.5 mm} x {85.48 mm ± 0.5 mm}. Including the edge d, the approximate dimensions are M=170x126, A2=110.08cm. 2 (borderless) or A2=214cm 2 (There is a connection)

[0044] Example of a combination of A2 and M commercially available products (borderless) Growth area: 175cm 2 Ross Selection cell culture flask with: M = {20.5 cm x 12.0 cm}, A = 246 cm 2 . Growth area: 75cm 2 Ross Selection cell culture bottle with: M = {15.0 cm x 8.5 cm}, A2 = 127.5 cm 2 . Growth area: 25cm 2 Ross Selection cell culture bottle with: M = {9.0cm x 5.0cm}, A2 = 45cm 2 . Mitchell Plastics® Tissue Culture Tray: M = {40 cm x 19.0 cm}, A2 = 760 cm 2

[0045] The data storage devices and / or programmable data processing devices of the system according to the invention may be components of the incubator and therefore in particular can be mounted in the incubator housing in which the incubator chamber is also located, but they may also be components of an external data processing device, in particular a computer, server or other laboratory equipment, connected to the incubator for the purpose of exchanging data.

[0046] An incubator is a laboratory device or laboratory incubator. An incubator is particularly a laboratory device with an incubator chamber, the atmosphere of which can be or is controlled by the incubator to a predetermined target temperature. It is particularly a laboratory device that can create and maintain controlled climatic conditions for various biological growth and proliferation processes. The incubator can be or include a shaking incubator (shaker), i.e., an incubator with a moving device for moving objects placed in the incubator chamber, or it can be a microbiological incubator (including without CO2). The incubator can particularly be configured as a cell culture device. The incubator is particularly used to create and maintain a microclimate with controlled gas and / or air humidity and / or temperature conditions in the incubator chamber, which may be time-dependent. The experimental incubator, particularly the processing device for the experimental incubator, may include a time generator, particularly a timer, a heating / cooling device, a regulator for controlling the exchange gas preferably supplied to the incubator chamber, a regulator for regulating the gas composition in the incubator chamber of the incubator, particularly the content of CO2 and / or O2 and / or N2 gases, and / or a regulator for regulating the air humidity in the incubator chamber of the incubator. The incubator, particularly the processing device for the incubator, preferably has an incubator chamber and further preferably a control device with at least one control circuit, to which at least one heating / cooling device is assigned as an operating element and at least one temperature measuring device as a measuring element. The control device can control the temperature in the incubator. In some embodiments, the air humidity can also be controlled. A water-filled tub in the incubator chamber can be heated or cooled to regulate the air humidity by evaporation. Alternatively and / or additionally, a water vapor generator can be provided as a component of the incubator to regulate the humidity in the atmosphere of the incubator chamber. CO2 incubators are used specifically for culturing animal and human cells.The incubator may comprise a rotating device for rotating the at least one cell culture vessel and / or a vibration device for vibrating or moving the at least one cell culture vessel. The incubator according to the invention is in particular not a bioreactor or a fermenter.

[0047] The incubator may comprise a sensor device. The sensor device in particular comprises at least one temperature sensor, preferably a plurality of temperature sensors. The temperature sensor may be, for example, a Pt100 or Pt1000 temperature sensor. The sensor device preferably comprises a sensor for determining relative gas concentrations, in particular a sensor for determining the content of CO2 and / or O2 and / or N2. The sensor device preferably comprises a sensor for determining relative air humidity.

[0048] The incubator preferably has one or only one incubator chamber. The incubator chamber can be divided into compartments. The compartments can be separated by (especially perforated) storage plates, in particular allowing gas exchange between the compartments. The storage plate, in particular its underside, can be configured to hold a camera device, in particular may have a holder for the camera. The storage plate, in particular its underside, can be configured to hold a lighting device, in particular may have a holder for the lighting device. However, the lighting device or its holder can also be arranged or attached elsewhere in the incubator chamber, for example to the inner side wall or floor or ceiling wall of the incubator chamber. The holder for the lighting device can comprise a rail system, a robotic arm controlled by a control device, and / or magnet(s).

[0049] The incubator chamber has a chamber wall or an inner chamber wall and exactly one or at least one chamber opening through which objects or cell culture vessels can be placed inside and removed from the incubator chamber. This chamber opening can be closed by an incubator door, particularly one or more chamber doors, movably attached to the incubator chamber using a closure element, particularly a hinge, movably connected to the incubator chamber. The incubator may have one or more inner doors, which may be particularly transparent, and one outer door. The outer door insulates the chamber opening, and optionally at least one inner incubator door that closes or opens the chamber opening, from the environment. Preferably, the imaging system captures the incubator door or outer door in a closed state, thereby preventing ambient light from affecting the illumination of the storage area, which is preferably performed exclusively by the lighting device. This results in particularly reproducible, easily comparable, and easily evaluated by image processing algorithms. However, imaging with the incubator door open is also possible.

[0050] In the closed position of the chamber opening, the interior of the incubator chamber is preferably insulated from the surroundings so that the desired atmosphere controlled by the incubator can be regulated, in particular controlled, therein. In the open position of the chamber opening, gas exchange between the surroundings of the incubator and the interior of the incubator chamber is possible through this opening. The chamber opening is typically in a front wall that surrounds the entire periphery of the chamber opening.

[0051] The incubator chamber has a plurality of walls or inner wall surfaces that can be connected to one another, particularly integrally, and particularly edge-free. The walls or inner wall surfaces are preferably substantially planar, but can also be fully or partially curved. The incubator chamber is preferably rectangular, but can also be formed in other shapes, such as a sphere, an ellipsoid, or a polyhedron. The walls or inner wall surfaces are preferably made of a low-corrosion material, particularly stainless steel, copper, brass, or plastic, particularly composite plastic. This facilitates cleaning / disinfection of the chamber interior. In addition to the chamber opening for inserting or removing objects or cell culture vessels, the incubator chamber may have at least one port for passing appropriately sized devices or connection cables from the inside of the incubator chamber to the outside or around the incubator.

[0052] The surface of the inner wall of the incubator is preferably made non-glossy or non-reflective, particularly by using a matte surface. The surface of the inner wall of the incubator can be made matte by surface treatment. The surface treatment may be polishing with an abrasive, which may have a specific grain size. The surface treatment may be blasting with a blasting material, particularly sand or glass beads, particularly with compressed air, which may have a specific grain size or characteristic particle size. This can prevent or reduce harmful reflections in imaging.

[0053] Typical sizes for the interior of an incubator chamber are 50-400 liters.

[0054] An incubator may have exactly one incubator chamber, but may also have multiple incubator chambers, the atmospheres of which (temperature, relative gas concentrations, air humidity) can be specifically regulated individually or collectively. An incubator may have multiple incubator chambers, each with its own chamber opening and its own chamber door for closing the chamber opening.

[0055] The incubator may have a housing that partially or completely encloses the incubator chamber. The housing may be formed as a substantially rectangular parallelepiped, in particular so that the incubator is stackable.

[0056] The storage area of the incubator is realized, in particular, by storage plates, in particular shelf inserts and / or movable platforms, which may be made of or include, in particular, stainless steel, copper, or similar materials. The storage plate serves as a floor, in particular an intermediate floor. The storage plate can be removable from the incubator chamber ("storage plate insert") or inserted into the incubator chamber and permanently fixed therein. The incubator chamber may have a holder or frame for holding one or more storage plate inserts or insertable devices. The storage plate may be configured to hold a camera on its underside, in particular a holder for this camera. Alternatively or additionally, at least one of the interior walls of the incubator chamber may be configured to hold one or more storage plate inserts or insertable devices, in particular at least one camera. For this purpose, a holding structure, in particular one or more protrusions, grooves, or webs, integrated into the wall may be provided. The storage plate expands the available storage area within the incubator chamber.

[0057] It is preferable that substantially the entire surface or at least a portion of the surface of at least one storage plate is made non-glossy or non-reflective, particularly by using a matte surface. The surface of the inner wall of the incubator can be made matte by surface treatment. The surface treatment may be polishing, particularly with an abrasive, which may have a specific grain size. The surface treatment may be blasting, particularly with compressed air, using a blasting material, particularly sand or glass beads, which may have a specific grain size or characteristic particle size. This can prevent or reduce harmful reflections in imaging.

[0058] The holding frame for the at least one storage plate is likewise preferably made of a non-corrosive material, preferably stainless steel. The holding frame is preferably designed as a standing structure by having at least one platform resting on the bottom wall of the incubator chamber. However, the holding frame can also be supported on the side walls of the incubator chamber and / or suspended from the top wall of the incubator chamber.

[0059] The storage plate preferably extends in particular substantially completely across the horizontal cross section of the incubator chamber.

[0060] The incubator preferably has at least two storage plates arranged one above the other. The volume area between the two storage plates, or between the bottom wall of the incubator chamber and the bottom storage plate, or between the top wall of the incubator chamber and the top storage plate, can be referred to as a storage compartment. The storage compartment as a whole can be considered as a storage area. The surface of the storage plate suitable for storage can be considered as a storage area. The height of the storage compartment is preferably dimensioned so that an object of a certain maximum height (measured perpendicular to the flat surface of the storage plate) or a stack of objects of a certain maximum height can be placed on the storage plate. This maximum height can in particular substantially correspond to the spacing between the two storage plates.

[0061] The distance between the two storage plates or the maximum height is in particular 5 cm to 70 cm, preferably 5 cm to 65 cm, preferably 5 cm to 60 cm, preferably 5 cm to 50 cm, preferably 10 cm to 30 cm, preferably 10 cm to 20 cm, preferably 12 cm to 18 cm. The maximum height is in particular possible up to 150 cm. The distance between the two storage plates can be selected by the user using a variable holding device for the storage plates.

[0062] Equipment, in particular cameras, can be formed as modules that can be inserted inside the incubator chamber, allowing automatic observation of the interior even when the incubator door is closed.

[0063] The camera device or its at least one camera is preferably arranged on the storage plate, preferably arranged or arrangeable below the storage plate, in particular fixed or fixable thereto. Preferably, the at least one camera is assembled or assemblable below the storage plate, in particular at the geometric center of the lower side, in particular at the intersection of the diagonals of the lower side of the rectangle.

[0064] Preferably, the camera or cameras are mounted or mountable to the underside of a shelf insert within the incubator chamber or to the underside of the upper interior wall (ceiling wall) of the incubator chamber, preferably vertically above the geometric center of the storage area that the camera monitors, respectively. However, one or more cameras may also be located or fixed or positionable / fixable to the interior side wall or holding frame of the incubator chamber.

[0065] Preferably, at least one camera is configured and arranged to have an image angle of between 90° and 210°, preferably between 120° and 180°, preferably between 160° and 180°, typically measured in the image diagonal, alternatively also in the image vertical or horizontal.

[0066] The camera of the imaging system may have a wide-angle lens, in particular a fisheye lens, the image angle of which in the image diagonal may be between 120° and 230°.

[0067] Preferably, exactly one camera, in particular having one of the above image angles, is provided on the underside of the storage plate.

[0068] The "field of view" or field of view (FOV) of a camera can be defined in particular to have a specific image angle, for example one of the ranges defined above, with image content being represented that depends on this angle of view, or it can be defined by an image angle measured at the image vertical and an image angle measured at the image horizontal. The aspect ratio of the image can be in particular one of the following formats: 4:3, 3:2 or 16:9, 1:1.

[0069] At least one camera, preferably exactly one camera, is arranged below the storage plate, and its field of view preferably captures at least X% of the storage plate's mounting surface within the field of view of the at least one camera. X is preferably 20, 30, 40, 50, 60, 70, 80, 90, or 100, respectively. In other words, the exact one or at least one image captured by the exact one or at least one camera preferably shows at least X% of the storage plate's mounting surface within the field of view of the at least one camera. For example, multiple cameras can be provided that together capture the entire storage area, i.e., 100% of the mounting surface, or a percentage X thereof. Preferably, exactly one camera is provided that captures the entire storage area or a percentage X thereof. The larger or more complete the field of view, the more reliably or efficiently the storage area can be captured and the images evaluated.

[0070] At least one camera, preferably exactly one camera, is arranged below the storage plate, and its field of view is within the incubator compartment, and preferably captures not only the mounting surface (part or all of the mounting surface) on the storage plate but also at least Y% of the surface of the compartment walls defining the compartment, which compartment walls are preferably formed by the interior walls of the incubator. Y is preferably 20, 30, 40, 50, 60, 70, 80, 90, or 100, respectively. In other words, exactly one or at least one image captured by this exactly one or at least one camera preferably shows at least Y% of the surface of one (or all) of the compartment walls defining the compartment. For example, multiple cameras can be provided that together capture the entire surface of all compartment walls, i.e., 100% of the interior wall surface of the compartment, or a percentage Y thereof. Preferably, exactly one camera is provided that captures the entire surface of all compartment walls or a percentage Y thereof. The correspondingly large field of view also allows imaging of objects or object stacks positioned at the edge of the flat storage area of the storage plate.

[0071] The data processing device is preferably programmed to automatically crop the image captured by the camera to produce an effective image angle that is smaller than the image angle that defines the camera, or to produce an effective field of view that is smaller than the field of view that defines the camera.

[0072] Incubator cameras are particularly suited to operate reliably in the respective incubator atmosphere for months or years, or for a lifetime measured under standard conditions (room temperature). Not all cameras are suitable for functioning in the incubator atmosphere. Commercially available cameras include the 5MP wide-angle camera for Raspberry Pi sold by Konrad Elektronik SE, Germany (www.joy-it.net), and / or other cameras combined with wide-angle lenses, such as the commercially available "Industrial Lens HAL2502.3" (Interniya Co., Ltd., Japan). Alternatively, a cover device can be provided to shield or insulate at least one camera from the incubator atmosphere. This cover device may have a transparent area or window, or may be transparent, to allow imaging through the transparent area.

[0073] The camera device preferably includes at least one optical filter that filters the light entering the camera. This optimizes the quality of the image capture, particularly for subsequent digital image processing and image evaluation. The camera device preferably includes at least one polarizing filter that filters the light entering the camera. This reduces potential reflections of the illumination device's light from objects in the storage area, elements of the storage area or incubator chamber, and / or the interior walls of the incubator chamber during image capture, particularly for subsequent digital image processing and image evaluation. The polarizing filter is preferably a circular polarizing filter, but may also be a linear polarizing filter. Unwanted reflections from smooth, non-metallic surfaces (e.g., plastic surfaces of cell culture vessels) can be suppressed by the polarizing filter. Non-metallic surfaces reflect vertically polarized light significantly more strongly, especially at exit angles to the surface of approximately 30°-40°, close to the Brewster's angle. When the polarizing filter is properly aligned, reflected light waves are suppressed, preventing the unpolarized background from being completely illuminated by reflections. When using polarized light, the object to be imaged, in particular the cell culture vessel, is preferably positioned between the illumination device and the at least one camera, in particular in a straight line.

[0074] The camera device has a first polarizing filter, and the lighting device has a second polarizing filter, preferably rotated relative to each other. In this way, a portion of the light from the lighting device is first blocked by the polarizing filter in front of it. The polarizing filter in front of the camera is then adjusted or rotated relative to the incident light so as to block the remaining portion of the light emitted from the lighting device. Ideally, only diffuse light remains. As a result, reflections, including those from metal surfaces, are reduced or completely eliminated. This is particularly advantageous for subsequent digital image processing and image evaluation, especially for detecting the contours of objects by image processing.

[0075] The lighting device preferably has at least one light source, in particular an LED. Preferably, the lighting device has at least two or more light sources, each with a different radiation spectrum, i.e., different colors, for example, red, green, and blue. In this way, the image quality can be optimized, especially with regard to subsequent digital image processing and image evaluation, in particular with regard to object contour detection by image processing.

[0076] The lighting device can have at least one light source, the emitted light of which has or consists of wavelengths greater than those of visible light, in particular the radiation spectrum of which is in or includes the infrared range, in particular having wavelengths between 780 nm and 1 mm, in particular the near-infrared range (780 nm to 3000 nm) or the mid-infrared range (3000 nm to 50000 nm). In this case, the camera device has at least one camera or camera sensor suitable for detecting the corresponding light, in particular infrared light. The lighting device can have at least one light source, the emitted light of which has or consists of wavelengths shorter than those of visible light. In this case, the camera device has at least one camera or camera sensor suitable for detecting the corresponding light.

[0077] Preferably, the at least two light sources are spaced apart from one another, which allows the field of view of the one or more cameras to be illuminated more homogeneously and reduces the intensity of individual reflection areas that depend on the direction of the light, which is advantageous for subsequent digital image processing and image evaluation, in particular for detecting the contours of objects by image processing.

[0078] The lighting device preferably comprises at least one light source that is or is fixed or positionable / fixable to the underside of the storage plate, and preferably comprises at least two or more light sources that are or is fixed or positionable / fixable to different positions along the underside of the storage plate.

[0079] The lighting device preferably comprises at least one optical filter by which the light emitted from the lighting device is partially or completely filtered, which may in particular be a polarizing filter synchronized with the polarizing filter of the camera device in order to achieve an optimal desired filtering effect.

[0080] Preferably, the lighting device has at least one light diffuser, so that the lighting device emits diffused light. The light diffuser can be or have, for example, a milky white Plexiglas plate. The light diffuser can reduce or prevent particularly sharp shadows and reflections, which is particularly advantageous for subsequent digital image processing and image evaluation.

[0081] The imaging system, in particular the camera device and / or the lighting device, preferably has at least one diaphragm, preferably in particular an aperture diaphragm with a variable diameter, for example an iris diaphragm, to control the light flow of the illumination or the light flow into the camera.

[0082] The imaging device, in particular the camera device and / or the lighting device, preferably comprises at least one optical lens.

[0083] The imaging system, in particular the camera arrangement and / or the lighting arrangement, preferably comprises at least one lens, preferably a wide-angle lens, preferably a wide-angle fisheye lens.

[0084] The imaging device preferably comprises a time generating device. The data processing device is preferably programmed to activate at least one or more light sources of the lighting device in a predetermined temporal sequence, in particular to deactivate them again after a respective predetermined operating time, and / or to activate all or a plurality of light sources simultaneously, and the camera device is preferably programmed to acquire a plurality of images of the storage area, each image being acquired consecutively, in particular synchronized with the operating time of the lighting.

[0085] The imaging system is preferably configured to capture and store the time when an object enters the incubator chamber and / or when an object is removed from the incubator chamber.

[0086] The imaging device illuminating at least one or two objects disposed in the storage area with a lighting device; capturing an image of at least one or two objects in the storage area with a camera device; Preferably, the data processing device is configured to store images of at least one or two objects in the data storage device in the form of image data, in this way enabling various uses of the image data, in particular to differentiate objects in the storage area, in particular to assign different identification data to first and second objects, to count objects, to recognize object classes, to analyze, store and recognize individual features, to track objects in motion, and to recognize and store the time of entry and exit of objects.

[0087] The data processing device comprises: distinguishing between a first object and a second object appearing in the image by evaluating the image data, in particular assigning different identification data to the first object and the second object, counting the objects, recognizing object classes, analyzing, storing and recognizing individual features, tracking the objects in motion, in particular detecting the contours of the first object and the second object in the image by means of an image processing algorithm; Preferably, the device is programmed to store information relating to the first object and the second object, in particular the bounding boxes and / or contours of the first object and the second object, in the form of object data in the data storage device.

[0088] In a preferred embodiment, the illumination device and in particular the data processing device are preferably configured and programmed such that the illumination device operates in at least two illumination modes, and the imaging system and in particular the data processing device are preferably configured and programmed such that: The lighting device illuminates the storage area of the incubator chamber. i) initially illuminating in a first illumination mode; ii) then illuminating with a second, different illumination mode; capturing at least one image of the storage area with a camera device while illuminated by both the first illumination mode and the second illumination mode; configured and programmed to provide at least one image in the form of image data including composite image information acquired during both the first illumination mode and the second illumination mode; Here, the data processing device is programmed to execute an image evaluation program for obtaining composite image information from image data.

[0089] This embodiment makes it possible to improve or optimize the quality of imaging, particularly of the storage area, which is advantageous in particular for subsequent image processing, in particular image evaluation, in particular for obtaining bounding boxes and / or contours of one or more objects or cell culture vessels.

[0090] The first and second illumination modes can be distinguished in particular by the use of different light sources, and / or light sources arranged at different positions, and / or different exposure times of the light sources, and / or different radiation spectra or light colors, and / or different luminous intensities. The different illumination modes can improve or optimize the quality of the imaging of, in particular, the storage area, which is advantageous in particular for subsequent image processing, in particular image evaluation, in particular for obtaining bounding boxes and / or contours of one or more objects or cell culture vessels.

[0091] Preferably, the at least one image of the storage area comprises at least one first image of the storage area and a second image of a different storage area, the first image being acquired in a first illumination mode and the second image being acquired in a second illumination mode, the first image being provided in the form of first image data and the second image being provided in the form of second image data; In particular, the data processing device and / or the image evaluation program combining the first image data and the second image data to obtain combined image data, in particular resulting from adding and / or averaging the first image data and the second image data; The system is programmed to derive composite information from the composite image data. This embodiment allows for improving or optimizing the quality of the imaging of the storage area, which is particularly advantageous for subsequent image processing, especially image evaluation, in particular for detecting the position of one or more objects or cell culture vessels in the image of the storage area by means of a bounding box algorithm.

[0092] A typical program code for object tracking using image processing is preferably based on the evaluation of a temporal sequence of images. Typical program code for object tracking uses a "bounding box" as an output format to identify an object in an image, determine its collision limits, and particularly locate it. In digital image processing, the coordinates of a rectangular frame that largely or completely surrounds an object shown in a digital image are called a "bounding box." The use of bounding boxes for object tracking makes object tracking more efficient, since such numerically assisted image evaluation requires fewer calculation steps and therefore fewer computing power, especially compared to algorithms for object contour detection. Furthermore, the use of specialized graphics processors (GPUs) allows for efficient and cost-effective implementation of corresponding algorithms. Suitable programming interfaces (APIs) for object tracking using bounding boxes are provided in the OpenCV program library under the names BOOSTING, CSRT, GOTURN, KCF, MEDIANFLOW, MOSSE, MIL, and TLD. In response to this, the OpenCV programming library provides a "multiple object tracking" feature for tracking multiple objects simultaneously. Alternatively, deep learning algorithms based on the "tracking-by-detection" principle for multiple object tracking (MOT) are known.

[0093] However, for object tracking it is also possible and advantageous to determine the contour of the object to be tracked in the image, in particular the separation of object (foreground) and background by background subtraction.

[0094] Preferably, a plurality of (N≧10) illumination modes are used to capture one image or to capture multiple images, which then provide a composite image containing composite image information in the form of composite image data. Preferably, 2≦N≦300, preferably 10≦N≦300, preferably 100≦N≦300. In this case, it is preferred that N≦500 or N≦1000.

[0095] Preferably, the at least one image of the storage area comprises a multiple exposure image of the storage area, and the imaging system is particularly exposing and capturing images of the storage area by a camera device while illuminated by both the first illumination mode and the second illumination mode; It is configured to provide multiple exposure images in the form of image data.

[0096] At least one image may contain information about objects placed in the storage area, in particular, optionally information about the location of the object in the storage area; Information about the outer contour of the object, information about the surface of the object, measured in a plane parallel to the flat surface of the storage area; Preferably, it includes information about the area of the storage area that is not occupied by objects, measured in a plane parallel to the flat surface of the storage area.

[0097] The lighting device preferably has at least one first light source and a second light source that operate differently in a first lighting mode and a second lighting mode, in particular the first light source and the second light source are arranged at a distance above the storage surface of the storage area, in particular the first light source and the second light source are arranged offset in a plane parallel to the flat storage surface of the storage area, in particular the storage area has a flat storage surface, the first light source is arranged vertically above a front half of the storage surface and the second light source is arranged vertically above a rear half of the storage surface, in particular the lighting device has an LED light strip with a plurality of LED light sources, this LED light strip is arranged in a plane parallel to the flat storage surface of the storage area, in particular in a serpentine path, a spiral path, in particular in an at least partially linear path, in particular the imaging system has a particularly programmable electronic control unit, in particular the electronic control unit During an illumination stage of the first illumination mode, the first light source operates differently than during an illumination stage of the second illumination mode, and / or such that during an illumination stage of the first illumination mode the second light source operates differently than during an illumination stage of the second illumination mode; especially, the first light source is active during an illumination stage of the first illumination mode and is less active (i.e. emits at a lower intensity) or inactive during an illumination stage of the second illumination mode; and / or the second light source is less active or inactive during illumination stages of the first illumination mode and is active during illumination stages of the second illumination mode; especially, During an illumination stage of the first illumination mode, the first light source operates with a different emission spectrum than during an illumination stage of the second illumination mode; and / or The second light source is set or programmed to operate with a different emission spectrum during the illumination phase of the first illumination mode than during the illumination phase of the second illumination mode.

[0098] Preferably, at least one camera is arranged at a distance vertically above the storage surface of the storage area, preferably at least one camera has a wide-angle optical system, in particular a wide-angle lens or a fisheye lens, and preferably exactly one camera is provided that is arranged at a distance vertically above the center of the storage surface of the storage area.

[0099] The imaging system is preferably a modular, i.e., user-selectable, component of the incubator, in particular the incubator comprising a control device and a temperature control device for controlling the temperature inside the incubator chamber, and the imaging system comprising a further control device arranged to control the imaging system, in particular whereby this further control device comprises a data processing device for the imaging system. Such a modular embodiment of the incubator with an imaging system preferably further comprises a data interface with the incubator, e.g., so that image data can be displayed on a display of the incubator.

[0100] The incubator preferably comprises, in particular, a control device and a temperature control device for controlling the temperature inside the incubator chamber, in particular the control device being configured to control the imaging system, in particular for which the control device includes a data processing device of the imaging system, this being an embodiment of an integrated incubator and imaging system.

[0101] The incubator preferably has a display, and is preferably configured or programmed to display on the display an image of the storage area, preferably including an image and / or image information preferably derived from at least one image and / or composite image information.

[0102] the imaging system is an object recognition system, wherein the data processing device is programmed to detect, by an image evaluation program, at least one object placed in the storage area during the capture of at least one image; 1. An object recognition system for recognizing objects, in particular based on individual features, and / or for recognizing object classes based on object-specific class characteristics, Preferably, the object tracking system is for tracking the change in position of at least one object within a storage area, particularly starting from an initial position and detecting its final position.

[0103] "Down" refers to the direction of gravity, and "up" refers to the opposite direction. "Vertical" means "along the vector of gravity," and "horizontal" means perpendicular to the vertical line or in a plane perpendicular to the vertical line. In normal use, the incubator is oriented with the top side of the flat storage plate horizontal.

[0104] Preferably, the incubator comprises a processing device for processing at least one object, in particular a cell culture vessel. The term "processing" means moving and / or transporting and / or examining and / or modifying in particular an object, in particular a cell culture or a cell culture vessel, in particular physically, chemically, biochemically or in other ways.

[0105] The treatment device may be a moving device that keeps the cell culture medium in at least one cell culture vessel moving, preferably according to a movement program controlled by a control program. The moving device may be a vibration device or a rocking device. The moving device preferably has a support device, in particular a plate, on which one or more cell culture vessels are placed and / or fixed. The moving device preferably has a drive device, in particular in the case of a vibration device, for example an oscillator drive device, which realizes the desired movement program, in particular in combination with an eccentric. The treatment device may be a rocking device that rocks at least one cell culture vessel. The components of the rocking device may be identical to those of the vibration device, but are configured for rocking movement.

[0106] The processing device may be a transfer device capable of transferring at least one cell culture vessel within an incubator chamber.

[0107] The transport device may be a lifting device having a support device on which at least one object, in particular a cell culture vessel, a camera or a light source, can be placed. The transport device or the lifting device preferably has a movement mechanism and / or an electrically operable drive mechanism for driving the movement mechanism. The transport device may further be an electrically operable movable gripping arm for gripping and holding at least one cell culture vessel. The transport device may have a conveyor belt or a rail system for moving the at least one placed or positioned object. The transport can move the at least one object within the incubator chamber, in particular, to and from a processing or imaging position of a processing station within the incubator chamber, for example. The control device can be configured to control the transport device depending on information from previously acquired image data.

[0108] The processing device may be a transport device capable of transporting at least one camera of the camera device and / or at least one light source within the incubator chamber. The transport device may be arranged, in particular, below or directly below the storage plate and / or below or directly below the ceiling wall of the incubator chamber. A movable or mobile light source allows different lighting modes to be realized. In particular, the lighting mode can be adapted to the occupancy situation, e.g., when the storage area is very densely occupied with objects, a suitable lighting direction can be variably realized. With multiple cameras, even with a movable or mobile camera, different images or image portions of the storage area can be generated and then combined into an overall image of the storage area, in particular by digital image processing. Furthermore, in the case of a mobile camera, the camera position can also be adapted to the occupancy situation, e.g., when the storage area is very densely occupied with objects, a suitable lighting direction can be variably realized.

[0109] In particular, the data processing device can be programmed to transport at least one camera of the camera device and / or at least one light source in the incubator chamber in a predetermined or dynamically adapted manner by a transport device. For example, the data processing device can be programmed to transport at least one camera of the camera device and / or at least one light source in the incubator chamber to different image capture positions by a transport device and to respectively evaluate the images generated there by an image processing algorithm, in particular to check whether desired image information, for example individual features of the cell culture vessel, in particular a barcode, is detected with sufficient quality so that, for example, the barcode can be uniquely read. The data processing device can be programmed to move the camera and / or the light source to other image capture positions until the desired image information is acquired.

[0110] Furthermore, the camera device and / or the lighting device can be fixed to the carrier device. The camera device and / or the lighting device can be fixed to or can be fixed to a positioning mechanism that can move and position the camera device and / or the lighting device within the incubator chamber. The positioning mechanism can include a movable robot arm and is preferably electrically controllable, in particular by a control program of the control device. In this way, different imaging situations can be captured successively with one or a few camera devices and / or lighting devices. The positioning mechanism can be configured as a component that can be inserted into the incubator chamber. The component can be powered via a cable connection to the incubator, preferably through a wall opening, e.g., a port, or via such a cable connection to an external power source. The control device can be configured to control the positioning mechanism in response to cell monitoring data.

[0111] A temperature control device for the incubator chamber, which controls the atmosphere in the incubator chamber to a desired value, in particular 37°C, is also understood as a processing device. The term temperature control refers to increasing or decreasing the temperature of the atmosphere by heating and cooling. The internal temperature is preferably regulated via a temperature change in the incubator walls. Temperature sensors of the corresponding temperature control device are distributed inside and / or outside the incubator chamber, in particular in the walls of the incubator chamber.

[0112] The incubator preferably has a user interface device that allows a user to input data into the data processing or control device and / or output information to the user. The incubator or its user interface device is preferably configured to allow a user to input at least one operating parameter for operating the incubator or the imaging system into the user interface device or to obtain information therefrom. In this way, a user can use a single user interface device to influence, control, or obtain information from the incubator and / or at least one imaging system. In particular, the imaging system can be configured to display to the user, in response to a user query made using the incubator's user interface device, location data or available storage locations, or to display information derived from the location data (e.g., the identity of the user who caused the location change), particularly also statistical information, such as the frequency and time of object (sample) location changes and / or available free storage locations (especially as a percentage), and / or an optical image of at least one object, particularly with or without storage areas. This is advantageous for the user, since based on this information the user can obtain important information. This information allows users to plan experiments more accurately, knowing that storage surfaces are available before conducting an experiment. On the other hand, while sample position shifts can negatively affect adherence, especially during the first few hours after seeding of adherent cells, preventing the formation of a uniform cell lawn, the present invention provides information about position shifts and their frequency, allowing users to identify the causes of uneven cell growth and take them into account in future experiments.

[0113] The incubator's device control process is preferably a program-controlled process, i.e., a process controlled by a program. Program-controlled processing of an object is understood to mean that the process is substantially carried out by executing a plurality of program steps. The program-controlled process is preferably carried out using at least one program parameter, in particular at least one program parameter selected by a user. The user-selected parameter is also referred to as a user parameter. The program-controlled process is preferably carried out using a digital data processing device, in particular a part of the control device. The data processing device or data processing unit may have at least one processor, i.e., a CPU, and / or at least one microprocessor. In particular, the program-controlled process is preferably controlled and / or executed according to the conditions specified by a program, in particular a control program. In particular, in program-controlled processing, substantially no user interaction is required, at least after the user has acquired the required program parameters. The incubator's device control process can be carried out in particular according to previously acquired image data. The imaging by the imaging system is in particular a program-controlled process, i.e., photocopying of the storage area or object.

[0114] The data storage device or data storage unit may be at least one volatile or non-volatile data memory, preferably having at least one data memory. Data acquired or received by the incubator can be stored in this at least one data memory, in particular in at least one database, which can be stored in the at least one data memory. This data includes at least one or all of the following types of data: image data, still image data, video image data, object data, composite image data, first and second image data, identification data, ID location data, user identification data, user-associated ID location data, object identification data, movement history data, class-related ID location data, individual ID location data, occupancy status data, and in particular installation location ID data. The data storage device / data storage device is preferably a component of the incubator, in particular located in the incubator housing. However, the data storage device / data storage device may also be a component of an external data processing device communicating with the incubator or its data processing device.

[0115] Program parameters are understood to be variables that can be set in a program or subprogram in a predetermined way and valid for at least one execution (call) of the program or subprogram. Program parameters are determined, for example by a user, to control the program or subprogram and cause data output depending on the program parameters. In particular, program parameters and / or data output by a program influence and / or control the control of an apparatus, the control of processing by at least one processing device.

[0116] A program or program code or computer program code is understood to mean, in particular, an executable computer program, which is stored in a data memory or data storage medium. A program is a sequence of instructions, in particular statements and commands, that enable a digital data processing device to perform and / or solve a specific function, task statement, or problem statement. A program is usually present as software used in conjunction with a data processing device. A program can be present, in particular, as firmware, in the present case, in particular as firmware for an incubator control device or system. A program is usually present on a data carrier as an executable program file, often in so-called machine code, which is loaded into the working memory of the computer of the data processing device for execution. A program is processed and executed by the computer processor as a sequence of machine instructions, i.e., processor instructions. The term "computer program" is also understood to mean a program source text, which may result from executable code, in particular in the control of laboratory equipment.

[0117] The user interface devices may be components or modules of the incubator. Each user interface device preferably comprises a control device for the user interface device, a communication device for establishing a data connection with the laboratory equipment, in particular the incubator, via the interface device, an input device for detecting user input by a user, and an output device for outputting information to the user, in particular a display device and / or display, in particular a touch-sensitive display. The control device of the user interface device is preferably configured to exchange data with the control device of the incubator via the data connection.

[0118] The object is in particular a cell culture vessel. The cell culture vessel is in particular transparent. The cell culture vessel is in particular made of plastic, in particular PE or PS, and has a flat bottom plate forming the growth surface for the cells. The bottom plate may have a surface treatment to promote cell adhesion. The cell culture vessel can be closed or equipped with a PE cap or gas exchange cap, in particular a lid optionally comprising a filter. The cell culture vessel is in particular stackable. Eppendorf cell culture bottles are particularly suitable. The object may be a stack of cell culture vessels, in particular a stack of Petri dishes or cell culture bottles.

[0119] The data processing device is preferably programmed to detect (time-dependent) changes in the appearance (or appearance) of objects from one or more images, particularly over relatively long time intervals of minutes, hours, or days. In this way, color changes in the cell culture medium, or the color inside the cell culture vessel, or irregularities on the cell culture vessel wall, such as droplets, can be detected. Such color, color changes, or irregularities may indicate problems with the respective cell culture, such as nutrient deficiencies, pH changes, or possible mold or other contamination. The data processing device is preferably programmed to, upon detection of the appearance of the cell culture vessel or a change in the appearance of the cell culture vessel, output information to a user or administrator via a user interface and / or store data related to this detection (particularly what was detected and when) in a data memory for continuous recall.

[0120] Image processing-based object tracking techniques are commonly known for their use in drones and driver assistance systems, for example, to track vehicles or people. Object tracking is based on image processing or image evaluation of video image data. Such object tracking methods can be achieved using relatively simple means, such as appropriate cameras and image processing algorithms. The theoretical foundations of object tracking technology and its practical applications are known (e.g., "Fundamentals of Object Tracking," S. Challa et al., Cambridge University Press, 2011). Ready-to-use image processing algorithms for object tracking are also freely available (OpenCV.org) and well documented. OpenCV (short for Open Computer Vision) is a free program library (BSD license) with algorithms for image processing and computer vision. The OpenCV program library also includes functionality for tracking multiple objects in real time. This application of object tracking to incubators is unprecedented and innovative.

[0121] A typical operating method for object tracking using image processing, which is also preferably used in object tracking systems, is based on the evaluation of a temporal sequence of images. Typical program codes for object tracking use "bounding boxes" as an output format to identify objects in images, determine their collision limits, and, in particular, locate them. In digital image processing, the coordinates of a rectangular frame that almost or completely surrounds an object shown in a digital image are called "bounding boxes." The use of bounding boxes in object tracking makes object tracking more efficient because such numerically assisted image evaluation requires fewer calculation steps and therefore fewer computing power, especially compared to algorithms for object contour recognition. Furthermore, the use of specialized graphics processors (GPUs) allows the algorithms to be implemented efficiently and cost-effectively. Suitable programming interfaces (APIs) for object tracking using bounding boxes are provided in the OpenCV program library under the names BOOSTING, CSRT, GOTURN, KCF, MEDIANFLOW, MOSSE, MIL, and TLD. In response to this, the OpenCV programming library provides a "multiple object tracking" feature for tracking multiple objects simultaneously. Alternatively, deep learning algorithms based on the "tracking-by-detection" principle for multiple object tracking (MOT) are known.

[0122] However, for object tracking it is also possible and advantageous to determine the contour of the object to be tracked in the image, in particular the separation of object (foreground) and background by background subtraction.

[0123] The performance of the object tracking system is based on, on the one hand, the reliable automatic identification of objects in the incubator in various typical use scenarios of the incubator, which are described below. On the other hand, this approach is efficient because it does not require any special adaptations on the part of the objects. In particular, the objects do not need to include passive identification means (codes, characters) or active identification means (e.g., transmitters). Rather, ordinary objects (cell culture vessels, devices, etc.) can be used with the incubator, regardless of their manufacturer or appearance. In particular, the incubator according to the present invention can distinguish between objects with exactly the same appearance by tracking them.

[0124] Possible scenarios for changes in occupancy within the incubator include: I. Insert a new object. II. Take out the object. III. Open the door and move the object, do not take out or put in a new one.

[0125] Secondary Conditions i) Moving an object. ii) Do not move the object. iii) Putting / taking out multiple objects (in order).

[0126] Assumption: All cell culture vessels look the same externally. The question underlying the development of the present invention was: what image-based methods are possible, and in particular are still images sufficient to enable object identification in typical usage scenarios of incubators?

[0127] In Scenario I (putting a novel object into the incubator chamber), we assume that the current still image of the storage area taken by a camera placed inside the incubator before the incubator chamber door is first opened does not yet contain a novel object.

[0128] If a new object is introduced into the incubator chamber without moving any existing objects (objects already placed and located in the storage area) (case I.ii)), the new object can be identified (unproblematically and uniquely) in the next still image (after the incubator door is closed). In case I.ii), no object tracking is required. The same is true for II.ii). In the case of removing an object, its identification is uniquely possible by evaluating still images before and after the door is opened.

[0129] If a new object is introduced and an existing object is moved in the process (Case Ii), this new object cannot be uniquely identified in the next still image. The position information of the already registered existing object is lost. The same applies when an object is removed and an existing object is moved in the process (Case IIi). In the case of movement (Condition i), the concept of object tracking is applied.

[0130] In case of condition i), i.e., when multiple objects are inserted without moving existing objects (Case III), the new objects can be easily identified by the still images before and after, but information about the order of insertion is lost. To obtain this information, object tracking is necessary. The same applies to the case of removing multiple objects in Cases I) + iii).

[0131] Since moving existing objects is the norm rather than the exception in incubator operation, evaluating static before and after images of the storage area alone is not sufficient in this case.

[0132] When developing an object tracking system for an incubator, a particular question is when an object is identified, i.e., at what time or event is identification data assigned to the object. In most use scenarios (except for cases like iii)+i, where it may be important to know the order in which objects are added), it is sufficient to collect identification data when a new object is added and existing objects are moved at the same time, because the movement of already registered existing objects is handled by the object tracking mechanism. The new object can be registered using the next still image, particularly when the incubator door is closed. If order tracking is desired, it is preferable to register the object the moment it first enters the camera's field of view, i.e., the moment it first appears in the image captured by the camera (the initial image, which is a video image in this case), and therefore assign an ID number to the object. This object is then tracked to its final position, even if existing objects are moved or removed.

[0133] In another practical scenario, assume that a storage area (or multiple storage areas) is occupied by one or more objects (existing objects), which are registered based on the first initial image (e.g., a still image in this case). Here, it is only necessary to track the movements of these existing objects. The movements of new objects to be inserted do not need to be tracked here during the insertion process, because their registration can be performed again in the next still image. Therefore, the presence of these new objects in the video data can be ignored. In this scenario, information about the order in which the objects are inserted during the door opening is lost, but this information is not necessarily required.

[0134] Therefore, the present invention proposes that the preferred embodiment implements object tracking to ensure accurate location of objects in various or all situations as needed.

[0135] The data processing device is particularly programmed to assign identification data to at least one object introduced therein. This means, in particular, that a new object is detected in the image data (still image or video data) of the camera. In particular, a new object is detected when a new object is introduced into the field of view of the camera from the outside. Once the object is detected, it is assigned identification data on the one hand and position data on the other hand. The position data of the object, in particular the initial position and the final position, are determined in particular with reference to an internal coordinate system, on the basis of which the position of the at least one storage area and thus also the position of the at least one object relative to the at least one storage area are defined. This position information is particularly important when the position of the at least one object in the at least one storage area or incubator chamber is to be graphically displayed to the user on a display.

[0136] The identification data may be or include an identification number and / or may include an identification code consisting of any symbols or information. These identification data may be predetermined, randomly generated, or user-specified, especially if they are suitable to uniquely distinguish the object newly placed in the incubator chamber from the identification data of other existing objects. The identification data may also be predetermined and simply selected. The concept of "assigning" includes this case as well as the creation of new identification data.

[0137] The data processing device is particularly programmed to determine an initial position of at least one object from an initial image of the storage area. The initial image is preferably a still image taken in still image mode of the camera. It may also be a single image, in particular a video frame, obtained from video data. The data processing device is particularly programmed to determine an envelope diagram, preferably a rectangle, or an envelope or bounding box, or an outer contour of the object in the initial image, and in particular to define the object by the envelope diagram, in particular the bounding box or outer contour.

[0138] The data processing device is particularly programmed to determine the change in position of at least one object by evaluating the video data. The data processing device is particularly programmed to track the movement of the object defined by a bounding box in the initial image. The data processing device is particularly programmed to recognize the movement of the area containing the object defined by the bounding box in the initial image, and therefore determine the change in position of this image area frame by frame. In particular, in this case, the tracking of the bounding box can be used to determine the image area whose position changes due to the movement of the object. The video data particularly contains information from which the individual images ("frames", which, when displaying the video, are expressed as a certain number per unit time, i.e., "frame rate") characterizing the video can be reconstructed. If the video data is not compressed, the video data can also include a complete sequence of image data, with each "set" of image data representing a single image. If the image data is compressed, the temporal change in the pixels of the camera image may also / only be obtained.

[0139] The data processing device is particularly programmed to determine an initial position of an object in the storage area from an initial image. The initial position can be particularly identified by an object that was previously motionless in the first frame of the image series showing a change in position of the object in subsequent frames. The first frame in which the object shows a change in position relative to the previous frame can be defined as the initial image. Since the object movement starts at time T1 and ends at time T2, images (still images or video images, including a single image obtained by superimposing images) taken before time T1 can be used as the initial image for determining the initial position of at least one object.

[0140] The data processing device is particularly programmed to determine the final position of at least one object in a final image of the storage area from the position change. The final position can be determined, in particular, when a position change of the object can no longer be identified from frame to frame. The first frame in which the object no longer shows a position change relative to the previous frame can be defined as the final image. Since the object's movement begins at time T1 and ends at time T2, images (still images or video images, including a single image obtained by superimposing images) taken after time T2 can be used as the final image for determining the final position of at least one object. The final position can be particularly determined by the time when the door sensor detects the closure of the incubator door. The final position can be particularly determined when the user's arm or hand entering the image field is no longer detected. For example, the image can be evaluated to see if a band, for example, located within the edge of the image corresponds to a reference state in which the incubator chamber or a reference portion of the incubator is fully visible. If not, it can be concluded that the user is still operating within the incubator and one or more objects are still moving, and the acquisition and evaluation of video images, in particular, continues. The final position of an object can be understood as the position at which the object no longer exhibits a position change after a previous position change, and can therefore be identified by the end of the object's movement. Alternatively or additionally, the final position of an object can be defined as the position the object occupies when the closure of the incubator door is detected by the door sensor. As a result, in most cases the final position will be the same.

[0141] The data processing device is preferably programmed to initiate initial image and / or video data acquisition by the camera when the sensor detects an activity occurring in the incubator. The sensor may be a motion sensor that detects movement within a detection range located outside the incubator. The sensor may be a touch sensor that detects a user's contact with the incubator, in particular the incubator door handle. The sensor may be a door opening sensor that detects the opening of an incubator door, in particular the outer door of the incubator. Thus, the sensor may be an external camera of the incubator that detects movement and / or people within the camera's field of view as image evaluation. The sensor may be a proximity sensor that detects a person approaching the incubator, for example by detecting a change in an electric field.

[0142] The data processing device is preferably programmed to start capturing final images and / or stop capturing video data when the sensor detects activity occurring in the incubator. The sensor may in particular be a door-opening sensor that detects the closure of an incubator door, in particular an outer door of the incubator. The sensor may be an external camera of the incubator that detects the end of activity and / or the disappearance of a person within the camera's field of view by image evaluation. The sensor may be a proximity sensor that detects when a person is moving away from the incubator, for example by detecting a change in the electric field.

[0143] The data processing device is particularly programmed to initiate the acquisition of video data by the camera upon detection of the opening of the incubator door by the door sensor. Alternatively or additionally, a triggering event sensor, particularly a motion sensor, proximity sensor, optical sensor / receiver (e.g., a light barrier), microphone, or acceleration sensor, can be disposed on the incubator door or within the incubator, to detect an object approaching the incubator chamber or another triggering event. The trigger for the camera and video data can also be the entry of a code into the incubator door lock, which can be performed by the data processing device, particularly without using the measurement results of any of the above sensors. The data processing device can be programmed to initiate the acquisition of video data based on data from such sensors. The data processing device can be programmed to initiate a search for new objects in images (frames) or still images available from the video data upon the presence of video data and / or still images. Alternatively, the acquisition of video data can be initiated immediately upon the identification of a user in the incubator or upon other predetermined events. Continuous acquisition of video data is also possible. The data processing device is particularly programmed to terminate the acquisition of video image data when a final image has been acquired or when no hand / arm is registered within the field of view of the camera, or based on the result of any one of the sensors mentioned above (door sensor, motion sensor, etc.).

[0144] The data processing device is particularly programmed to assign identification data to at least one object in the storage area and to determine the position of each of the at least one object as ID position data and store the ID position data in the data memory. The data processing device is particularly programmed to store the final position of the at least one object in the storage area as ID position data in the data memory according to the identification data of the at least one object. At this stage, the incubator "knows" the "object" and its position. The incubator can then output this data, along with other data, to the user, particularly displaying it on the incubator display. The incubator can store and collect these data sets, along with data about the object's owner (defined as the user who placed the object in the incubator chamber) or the object's user (e.g., a user who removed another user's existing object), according to the object's identification data. The identification data detected when a position change occurs need not be identical to the identification data stored as ID position data; what is important is that the stored identification data is suitable for uniquely distinguishing the at least one object from other or existing objects. Therefore, theoretically, the ID code can be changed during image processing.

[0145] Assigning an owner to an object can be accomplished in various ways. The data processing device is preferably programmed to register or identify a user who places an object in the incubator, assign a user identification code to the user, and store ID location data associated with the object's user. For registration, biometric recognition of the user, in particular facial recognition, language recognition, and / or voice recognition, can be performed, in particular via an external camera, retina scanner, or fingerprint sensor of the incubator. Corresponding registered biometric data, in particular the user's facial recognition data, can be stored in the incubator's data storage device or an external data storage device. Identification of the user can be achieved by matching the acquired biometric data with registered biometric data. As an alternative to biometric recognition, the user can enter user identification data via a user interface device before, during, or after performing object registration or after determining the final location of the tracked object. The user interface device can be a keyboard, a touchscreen, part of the incubator or an external device, or the user name / user identifier can be entered via verbal input.

[0146] The advantage of object tracking is that it can be performed essentially without knowing the individual or class characteristics of the object being tracked. However, object tracking can also be combined with methods of object recognition (and object re-recognition) and / or object class recognition or class re-recognition. This is particularly useful when multiple objects are tracked in parallel by an object tracking system.

[0147] Object recognition can be configured as individual object recognition and / or object class recognition, among others. The theoretical foundations of object recognition techniques and their practical implementations are known (e.g., "Deep Learning in Object Recognition and Cognition," X. Jiang et al., Springer Singapore, 2019). Algorithms for object recognition in images are commonly known and available, for example as part of OpenCV (e.g., OpenCV 3.3, Deep Neural Network (dnn) module).

[0148] Individual object recognition is based on the recognition of individual features of an object (individual object features), which allow the individual object to be recognized and distinguished from other individual objects. For example, a cell culture vessel, e.g., a disposable product, may have a post-applied individual feature, e.g., a barcode or a QR code. However, a cell culture vessel can also be identified via any distinguishing feature, e.g., a string of characters, different contents, a micro-scratch pattern on the vessel surface, etc.

[0149] Object class recognition is based on knowledge of object class features that are matched during object inspection to assign a class to an object. For example, object class recognition can recognize whether an object is a particular type of cell culture bottle, a particular type of petri dish, or a particular type of microtiter plate, possibly taking into account other class features such as manufacturer, year of manufacture, specifications, etc.

[0150] The incubator preferably comprises an object recognition system. Preferably, an object tracking system is provided in addition to the object recognition.

[0151] In the case of individual object recognition, the data processing device of the object recognition system or object tracking system is preferably programmed to: a) recognize individual features of at least one object in the still image, the initial image, the video data, and / or the final image; and b) store these individual features of the object in the form of individual object data, in particular according to identification data. The data processing device preferably extracts individual object features of at least one object in the initial image, the video data, and / or the final image, compares these individual object features with an individual object database, and if the individual object features in the individual object database are associated with an individual object label, identifies the individual object label of the at least one object; if the individual object features in the individual object database are not associated with an individual object label, assigns an individual object label to the at least one object and stores it in the individual object database, and / or assigns the recognized individual object label to ID location data of the at least one object and stores it as individual-associated ID location data. The individual object label is preferably different from its identification data, although the individual object label can preferably be the same as its identification data.

[0152] In the case of object class recognition, the data processing device of the object recognition system or object tracking system is preferably programmed to: a) recognize at least one object class feature in the still images, the initial images, the video data and / or the final images, and b) store these object class features in the form of object class data, in particular depending on the identification data. The data processing device is preferably programmed to recognize at least one object object class feature in the still images, the initial images, the video data and / or the final images, match these object class features with an object class database (in particular containing previously known correlations between object classes and object class features), recognize the object class of the at least one object, in particular assign the recognized object class to ID-position data of the at least one object as object class data, and store in particular as class-related ID-position data.

[0153] Preferably, the incubator includes a user identification device by which users of the incubator can be identified in the form of user identification data. Preferably, the data processing device of the incubator is programmed to identify users of the incubator by means of the user identification device, assign user identification data to the users, and store such identification data and / or ID location data in the data storage device as user-associated identification data and / or user-associated ID location data in response to the user identification data.

[0154] The user identification device preferably comprises an external camera, and the user identification device and / or the data processing device are preferably configured or programmed to perform face recognition by means of the external camera and thereby identify the user. A user database is provided which is stored in a data memory which may be part of the incubator, the user identification device or the object tracking system or which is preferably connected to the user identification device or the data processing device for data exchange, for example via an intranet or the internet. Algorithms for recognizing faces in images are generally known and are available, for example as part of OpenCV ("FaceRecognizer class").

[0155] The user database may include correlations between user identification data and user characteristic data, and may determine a user or their user identification code (user identification data) based on the determined or read user characteristic data. The user characteristic data may include information about the user's facial features or other biometric data, such as fingerprint data or voice recognition data. The user database may include correlations between user identification data and user indicia, which may be a user's personal identification code, such as a multi-digit string, which can be entered on a keyboard of a user interface device to identify the user.

[0156] The external camera can be arranged or fixed, in particular on, above or next to the incubator door, in particular the outer door of the incubator. The external camera is preferably a fixed component of the incubator or the incubator door. However, the external camera can also be connected to the user identification device or data processing device via a signal connection, in particular via a data exchange connection, which can be wired or wireless. For example, the external camera can be connected to the incubator or its user identification device or data processing device via a flexible cable, allowing the user to freely position the camera on the incubator.

[0157] The user identification device preferably comprises a user interface device capable of reading user identification data. The user interface device may include a keyboard, and / or a touch screen, and / or a microphone for language input or for realizing user identification by language recognition. The user interface device may be configured to exchange data with an external data processing device (hereinafter also referred to as "external device"). The external device may be a PC, a smartphone, a tablet computer, or other portable computer with a user interface.

[0158] The external device may have means for identifying and / or authenticating the user. In particular, currently available smartphones include various means for user authentication, in particular facial recognition. The external device preferably has software, e.g., an app, programmed to identify and / or authenticate the user and, in particular, to transmit the results of this process to the incubator's user identification device via the user interface device. The external device also often has its own camera capable of performing facial recognition, or a fingerprint sensor, or other hardware for user identification and authentication, so that the incubator can be connected to an external device from the incubator without the need for corresponding hardware components.

[0159] The user identification device of the incubator can be programmed as a component of the incubator's control software. The incubator preferably includes a control device, which may in particular comprise a data processing device that can be programmed to include all or some of the functions of the user identification device, in particular to control the exchange of data with external equipment.

[0160] The user identification device preferably has a user interface device by which user identification data can be selected. For this purpose, the user identification device can in particular have a display or a touch screen, and can display a list of possible users by specifying, for example, the name or image of the user. Input means such as keys, keyboard, touchpad, touch screen, etc. can then be provided to allow the user to select from the list.

[0161] The user identification device can be programmed to perform user authentication by password-protecting the reading of the user identification data or the selection from the list, and to consider the user identified only after successful authentication.

[0162] The user identification device preferably comprises a reader for reading a code identifying the user, which reader is in particular an RFID reader, a barcode reader or a QR code reader.

[0163] The user identification device or its data processing device can be programmed to unlock and / or lock locked incubator doors depending on the user identification, in particular to unlock locked incubator doors when the user is successfully identified. In this case, this means that the user is also authorized to access the incubator. However, there may also be an additional list of authorized accesses, based on which the incubator determines whether an identified user has access rights, or possibly what type of access rights the identified user has. This access right can be limited, for example, to certain times, in particular days of the week, times of day, or authorized time slots. If the incubator has several incubator doors, the access rights can be such that a user has access rights only to a predetermined selection of these incubator doors.

[0164] The incubator preferably has exactly one or more incubator doors for closing the chamber openings. In the closed state, the incubator doors form part of the incubator housing, which in particular serves as a thermal insulator for the incubator chamber of the incubator. The incubator doors can be provided on their exterior with a user interface device, in particular a display. A data processing unit of the incubator or of the user interface device can be programmed to display an image of at least one storage area of the incubator taken by a camera of the incubator.

[0165] The incubator preferably has a door sensor for detecting the opening or closing of the incubator door. The incubator preferably has a motion or proximity sensor for detecting a person approaching the incubator. The data processing device is preferably programmed to initiate monitoring of the interior of the incubator, in particular the generation of video / still image data, in response to detecting the opening of the incubator door and / or the approach of a person. The data processing device is preferably programmed to initiate monitoring of the interior of the incubator, in particular the generation of video / still image data, in particular upon detection of the door being opened by a user identified by the user identification device. The data processing device is preferably programmed to terminate monitoring of the interior of the incubator, and in particular the generation of video data, upon detection of the closure of the incubator door. The data processing device is preferably programmed to use information from the user identification device and the object tracking device to ascertain which user has moved which object within, and to store the user identification data of that user in the data memory along with the object identification data of that object.

[0166] The data processing device is preferably programmed to determine a movement path of at least one object in the incubator chamber from the initial images, the video data, and / or the final images and store the determined movement path in the data memory in the form of movement history data, particularly time-dependently. The data processing device is preferably programmed to determine a movement history of at least one object in the incubator chamber from the initial images, the video data, and / or the final images and store the determined movement path in the data memory in the form of movement history data, particularly time-dependently, preferably together with information on the number and / or time of changes in the state (open / closed) of the door opening of the incubator door, determined by the door sensor. The movement path preferably includes stored position data of the object, which position data mark the movement path of the object, particularly between the initial and final images, particularly between an initial position of the object that is motionless or has moved, and a final position of the object that is motionless. The movement history data preferably includes time-dependent stored position data or movement path, preferably within at least one time segment or over the entire stay time of the object in the incubator. The movement history data may also include information on the user who caused the position change in the form of user identification data. This is particularly advantageous in the case of objects containing valuable samples.

[0167] The incubator preferably has a display (=screen). The screen is preferably a fixed component of the incubator, in particular of the incubator door. However, the screen can also be located remotely from the incubator and may in particular be a component of an external device that can be connected for data exchange with the incubator's data processing device.

[0168] The data processing device is preferably programmed to display on a screen a graphical reproduction of the interior of the incubator chamber, in particular of at least one storage area. The graphical reproduction may comprise a photograph of the storage area, in which one or more existing objects of the incubator can be displayed. The storage area may in particular be a storage plate or a predetermined part thereof within the incubator. The photograph may represent an image taken by a camera, which may optionally be post-processed. This post-processing preferably comprises correcting distorted images taken by the camera. Algorithms for such post-processing are generally known and freely available (e.g. OpenCV "Omnidirectional Camera Calibration"). Distortions arise in particular from optical causes and may result from the use of wide-angle or fisheye optics.

[0169] The graphic playback image may be an abstracted representation of an image or image portion captured by a camera, in particular. For example, the graphic playback image may be an abstracted storage area shown from a bird's-eye view (or another perspective), in particular a rectangular graphic playback image or a perspective representation of a cuboid. Existing objects may also be represented in abstracted form, for example as rectangular or cuboid graphic image objects. The purpose of such a representation is to inform a user of the current location of one or more objects, in particular in an incubator or storage area. This allows the user to quickly access one or more desired objects and minimizes the time the incubator door is open. If it allows for the differentiation of individual objects contained in a stack of objects, it may be useful to have a graphic playback image from a perspective other than a bird's-eye view, for example a lateral perspective, so that the individual stacked objects can be graphically highlighted.

[0170] The data processing device is preferably programmed to provide a graphical display of where an object identified by the object location data is located within the storage area or incubator chamber, or alternatively, a graphical display of where all objects located therein are located.

[0171] The data processing device is preferably programmed to graphically highlight one or more objects on the display in response to at least one condition parameter, which condition parameter may represent user identification data.

[0172] Preferably, the data processing device is programmed to graphically highlight one or more objects on the display in response to at least one condition parameter, which may represent user identification data. The highlighting can be either an abstract representation or a photographic representation of an image or image portion of the storage area taken by a camera on the incubator display.

[0173] The data processing device is preferably programmed to graphically highlight on the display one or more objects that are assigned as property to a user depending on the user identification data of a user (single user, group of users or multiple users), i.e., for example because the ID location data associated with the user contains the user identification data of this user. The owner is the person who has control over the objects and who, in most cases, placed them in the incubator chamber by himself or with the help of an assistant. The data processing device is preferably programmed to determine, based on given user identification data, where the objects assigned to the user identification data are located by the object location data associated with the user, and in particular to graphically highlight these objects.

[0174] Alternatively, the condition parameters may include information regarding time or time of day, such as how long the object has already been placed in the incubator chamber, allowing a user to quickly get insight into how long one or more objects have already been stored in the incubator chamber, or perhaps forgotten by their owner. Alternatively, the incubator may graphically highlight one or more objects that require the user's or lab staff's attention in response to an event detected by a sensor in the incubator or in response to a schedule that may be stored in the incubator or on an external device.

[0175] Alternatively, when implementing object class recognition, the condition parameters may include information about a particular object class, e.g., one or more objects of the same object class (or different object classes) may be graphically highlighted to highlight the current location of all petri dishes (or cell culture bottles) inside an incubator.

[0176] Alternatively, when implementing individual object recognition, the condition parameters may include information about a specific individual object. In this way, for example, an incubator may have a means for inputting individual features, such as a barcode, a QR code, individual character strings, or a photo of an individual object, thereby realizing object search based on individual features. Thus, individual objects are graphically enhanced and can be easily found.

[0177] The data processing device is preferably programmed to display on a screen a graphical reproduction of the interior of the incubator chamber, in particular of the at least one storage area, and in particular to graphically indicate or highlight available free storage locations within the incubator. For example, the storage area can be shown in abstraction and a free storage location (or multiple available free storage locations) can be graphically highlighted by displaying the corresponding area in, for example, green or white or in a contrasting color against a time-alternating (flashing) background. In this way, the user does not have to waste time searching for potentially free storage locations or moving existing objects to create free storage locations.

[0178] Furthermore, the data processing device can be programmed to plan the occupancy of the interior of the incubator chamber or at least one storage area, similar to the function of a parking attendant, and in this way, optimize the use of available storage surfaces. For this purpose, the data processing device can be programmed to take into account a predetermined distance between one or more existing objects and a new object, and to present this to the user, in particular by highlighting available and / or unavailable storage locations, respectively. In accordance with these examples, the incubator may have a computer / software-implemented planning program for occupying the interior of the incubator. This planning program takes into account, in particular, the location of at least one object (existing object) inside and / or available free storage locations, possibly also the time at which at least one existing object was newly introduced or the time at which another object is scheduled to be introduced into the incubator in the future. Such times can be recognized, in particular, if the incubator is connected to a laboratory information system (LIS) or other (laboratory) data exchange network. The incubator preferably has a time generator, clock, or timer.

[0179] The data processing device of the incubator can be and particularly preferably is programmed to determine the occupancy status inside the incubator chamber and / or to perform one or more of the following steps: determining the occupancy status inside the incubator chamber in dependence on the ID position data of at least one object placed therein, determining the occupancy status inside the incubator chamber in dependence on the class-related ID position data of at least one object placed therein, determining the occupancy status inside the incubator chamber in dependence on the individual-related ID position data of at least one object placed therein.

[0180] The interior occupancy status can be defined by information representing the volume occupied by at least one object within the interior, and / or information representing the volume not occupied by at least one object within the interior, i.e., the free volume, and / or information representing the storage area occupied by at least one object within at least one storage area or the total available storage area within the incubator chamber, and / or information representing the free storage area not occupied by at least one object within at least one storage area or the total available storage area within the incubator chamber. These items can be related to the total interior volume or total storage area, respectively, and the information can include, for example, the ratio of the unavailable (occupied) interior volume or the free (unoccupied) interior volume to the total volume of the interior. Alternatively, the information can include the ratio of the unavailable (occupied) storage area or the free (unoccupied) storage area to the total storage area within the interior.

[0181] The occupancy data containing information about the occupancy may also contain ID location data, class-related ID location data and / or individual-related ID location data, in this way making statements about the location resolution of the occupancy, i.e. the location of the occupancy within the interior or statements about the density distribution of objects within the interior, is possible.

[0182] It is possible and particularly preferred that the data processing device of the incubator is programmed to store information about the occupancy state of the incubator in the form of occupancy state data in a data storage device and to transfer it in particular to an external data processing device, in particular a laboratory device, a PC, or a mobile computer, in particular a tablet computer or a smartphone.

[0183] It is possible and particularly preferred that the data processing device of the incubator is programmed to display information about the occupancy state of the incubator on a screen of the incubator or of an external data processing device, which may be part of a laboratory device, a PC, a mobile computer, in particular a tablet computer, or a smartphone, in particular depending on occupancy data that can be retrieved from the data memory.

[0184] In a series of tests underlying an embodiment of the present invention, it was determined that the temperature change over time within an incubator chamber resulting from temperature control after opening the incubator door depends on the occupancy of the incubator chamber. When a large volume within the chamber is occupied by existing objects, the difference between the volume within the chamber and the volume occupied by the objects results in a small free volume within the chamber. In such a situation, a temperature control designed to control the entire internal volume can potentially result in other undesirable consequences. A rapid overshoot can occur, which may accelerate recovery to a target temperature of, for example, 37°C, shortening the recovery time. This is undesirable. The introduction of multiple new objects at temperatures lower than the target temperature can potentially delay recovery time, but knowledge of the newly introduced, lower temperatures can also be used to adapt the temperature control. Temperature control of the internal temperature of the incubator chamber depends on control parameters.

[0185] The electronic control device of the incubator is preferably configured or programmed so that at least one temperature control device of the incubator, arranged for temperature control of the incubator chamber, is driven with a power Ptemp(t) depending on the time t during temperature control. In particular, the incubator can be configured so that the temperature control device is driven by pulse width modulation (PWM) of the current. The amplitude of the current is preferably constant, so that the power is determined in particular by the duty cycle of the PWM. In particular, the above values may be temperature control variables, i.e., control parameters.

[0186] The incubator's electronic control device is preferably configured or programmed to adapt the temperature control or the incubator gas supply (e.g., CO, N, and / or O), particularly at least one control parameter, depending on the occupancy of the incubator. In this way, the influence of objects placed inside the incubator chamber on the response behavior of the controlled system can be taken into account, particularly in cases of high occupancy, thereby shortening recovery times.

[0187] The data processing device of the imaging system or other system is preferably separate from the first data processing device of the incubator. However, the data processing device may also be part of the incubator's control device (also referred to as the "first control device"), which controls the incubator's functions. The functions of the control device are implemented, in particular, by electronic circuits. The data processing device of the imaging system may have at least one CPU and, optionally, at least one GPU. The GPU can be provided for performing image processing or deep learning processes. As an alternative to a CPU or GPU, the data processing device may have a dedicated chip for performing image processing or deep learning processes, such as an NVIDIA Jetson, which can preferably be used in object tracking, particularly possible object class classification or object individual recognition. Such a dedicated chip can be added to the data processing device as a computation accelerator. GPUs are already present in many systems-on-chips (SoCs) (for graphics and video playback images). The Raspberry PI may also have a dedicated GPU unit as part of its SOC.

[0188] The object tracking system may have a control unit that can be provided separately from the first control device. In this specification, the terms "control device" and "control unit" are used synonymously. The control device may have a microprocessor that can include a data processing device. The microprocessor may be of the "Raspberry PI" type. The control device and / or the data processing device are preferably configured to execute control procedures, also called control software or control programs, associated with the incubator and / or the object tracking system, respectively. The functions of the incubator and / or the object tracking system and / or the control device and / or the data processing device can be described in method steps. The method steps can be implemented as components of the control program, in particular as subprograms of the control program.

[0189] Within the scope of the present invention, the control device, which is an optional component of the incubator system according to the present invention, generally comprises or is in particular a data processing device or data processing unit, in particular a processor (CPU) and / or a microprocessor for processing data. The control of the incubator can preferably also be configured to control the object tracking system.

[0190] The data processing device of the imaging system is preferably a device arranged outside the incubator chamber or the incubator, in particular optionally separately therefrom, and is also called an external device or external data processing device. The data processing device and the incubator are preferably in data connection and are preferably components of a network for data exchange.

[0191] At least one camera of the imaging system is preferably connected to the control device or data processing device of the imaging system via a cable connection. For this purpose, the incubator chamber has a through-hole (port) through which the cable of the cable connection passes. In this case, a seal, in particular a silicone seal, is preferably provided to seal the port so as to (almost) prevent it from affecting the atmosphere inside the incubator. Alternatively, the camera is connected to the control device or data processing device for wireless data exchange, for example via Bluetooth or WLAN.

[0192] The incubator may have a partial housing in which at least one control device (of the incubator and / or the object tracking system) is arranged, preferably at the rear side of the incubator, i.e. in particular opposite the incubator door.

[0193] The system, incubator, and / or imaging system and / or data processing device and / or control device are preferably configured to use the position data of at least one object or multiple objects to create an electronic file in which the location and / or movement of the objects and / or their time of residence and / or the identification data of the user who caused the movement within the incubator are recorded and filed. This file is then stored, in particular in a data storage device, and is preferably continuously updated. In this way, the "correct" handling of the objects according to standard protocols can be verified if necessary. On the other hand, deviations from standard protocols can be identified later and / or correlations of information can be determined. Knowledge of such data significantly improves the quality and reliability of cell-based experimental work and medical, biological, and pharmaceutical processes. It increases the reproducibility of cell-based experimental work and allows early detection of deviations from normal characteristics, allowing the user to correct or repeat the experiment early. This file can be provided by data exchange from the control device to the user or to an external data processing device. Such file storage is particularly useful in critical applications, such as forensic applications or applications in which cells of considerable value are cultured.

[0194] The present invention also relates to a retrofit system for cultivating live cell cultures, comprising an incubator for live cell culture, the incubator comprising: an incubator chamber for accommodating an object, in particular a cell culture vessel, the incubator chamber having opposing inner walls, a chamber opening for a user to place or remove an object, and at least one storage area extending between the opposing inner walls for storing the object; an incubator door for closing the chamber opening; and an imaging system for processing images configured to retrofit the incubator, the imaging system including a data processing device with a data memory, an illumination device, and a camera device configured to photograph at least one storage area of the incubator chamber; The imaging system, in particular the data processing device, illuminating a storage area of the incubator extending between the interior walls with a lighting device; capturing at least one image of a storage area extending between the interior walls with a camera device; Optionally, storing by the data processing device the at least one image in the form of image data in a data storage device; determining at least one occupancy ratio from the image data characterizing the occupancy of the at least one storage area; performing at least one mathematical comparison operation comparing the at least one occupancy rate to at least one occupancy criterion value; incorporating a result of the at least one mathematical comparison operation into at least one occupancy evaluation parameter; It is configured and programmed to store at least one occupancy assessment parameter in a data storage device.

[0195] The system is therefore based on an incubator that can be retrofitted with an imaging system, which imaging system is a fixed component of the incubator in claim 1, and the retrofit imaging system must be compatible with so-called "compatible incubators." This "retrofitting" preferably includes the following: the camera device can be suitably arranged or fixed in the incubator chamber; the lighting device can be suitably arranged or fixed in the incubator chamber; the data processing device, in particular with a data memory, can be suitably arranged or fixed inside or to the side of the incubator; the camera device and / or the lighting device and / or the data processing device, in particular with a data memory, can be connected or connectable to the data processing device of the incubator or an external computer for data transmission purposes; and the camera device and / or the lighting device and / or the data processing device, in particular with a data memory, can be connected or connectable to an energy supply that may be part of the imaging system or the incubator. In particular, alternatively or additionally, the camera device and / or the lighting device can be connected or connectable to the data processing device of the incubator for data transmission purposes, so that the data processing device of the imaging system is formed by the data processing device of the incubator.

[0196] The control device or data processing device of the incubator according to the invention or a compatible incubator, which in particular can also control the atmospheric parameters in the incubator chamber (temperature, partial gas pressures CO2, HO, etc.), is preferably set or programmed to determine at least one operating parameter of the incubator, in particular parameters for controlling the display of information on a screen of the incubator or parameters to be displayed on a screen of the incubator, depending on data from the imaging system, in particular position data or the final position of at least one object in the storage area, and in particular position data or the final position of the at least one object can be displayed on the screen.

[0197] Preferably, the system for cultivating live cell cultures comprises an external device, in particular a (particularly portable) user identification device and in particular a data exchange device, separate from the incubator and in data exchange connection with the incubator, by means of which the data processing device can exchange data with the external device, in particular by means of which user identification data can be determined.

[0198] The present invention also relates to a method for imaging in an incubator used for culturing live cell cultures, the incubator comprising: an incubator chamber for storing objects, particularly cell culture vessels, comprising opposing inner walls, a chamber opening for a user to place and remove objects, and at least one storage area extending between the opposing inner walls for storing the objects; an incubator door for closing the chamber opening; an imaging system, A lighting device; A camera device; a data processing device; The method comprises: illuminating a storage area extending between the interior walls with a lighting device; capturing, by a camera device, at least one image in the form of image data of the storage area extending between the interior walls while illuminated; determining from the image data at least one occupancy ratio characterizing the occupancy of the at least one storage area; performing at least one mathematical comparison operation comparing the at least one occupancy rate with at least one occupancy criterion value; incorporating a result of the at least one mathematical comparison operation into at least one occupancy evaluation parameter; and storing the at least one occupancy assessment parameter in a data storage device.

[0199] The invention also relates to an imaging system for processing images, in particular designed to be retrofitted to an incubator, said imaging system comprising: A lighting device; at least one camera device; a data processing device having a data memory, The imaging system is illuminating at least one storage area extending between the interior walls with a lighting device; capturing, by a camera device, at least one image in the form of image data of at least one storage area extending between the interior walls; determining at least one respective occupancy ratio characterizing the occupancy of at least one storage area of each incubator from the image data; performing at least one mathematical comparison operation comparing the at least one occupancy rate to at least one occupancy criterion value; incorporating a result of the at least one mathematical comparison operation into at least one occupancy evaluation parameter; The at least one occupancy assessment parameter is configured to be stored in a data storage device.

[0200] Further preferred configurations of the subject matter according to the invention, in particular the method according to the invention, will become apparent from the description of the system according to the invention with an incubator and its preferred embodiments. Further optional configurations of the invention will become apparent from the illustrated examples. Identical elements in the examples are provided with substantially the same reference numerals unless otherwise specified or otherwise clear from the context. [Brief explanation of the drawings]

[0201] [Figure 1] FIG. 1 is a perspective view of a system according to the invention having an incubator according to an embodiment. [Figure 2] FIG. 2 is a front view of the incubator of FIG. [Figure 3] FIG. 3 is a front view of the incubator of FIG. 1 with a graphical representation of the occupancy of the incubator chamber by objects highlighted with color codes associated with users. [Figure 4a]FIG. 4 a shows a smartphone with a camera and a display 63 as an external device that may be a component of a system 400 including the incubator 1 and smartphone 69 of FIG. [Figure 4b] FIG. 4b shows a legend of the color codes used to highlight objects associated with the user on the screen of FIG. [Figure 5a] FIG. 5a is a schematic side view of an imaging system that is a component of the incubator of FIGS. 1-4b, showing an example of a chamber having a single monitored containment plate. [Figure 5b] FIG. 5b is a schematic side view of an imaging system that is a component of the incubator of FIGS. 1-4b, illustrating an example of a chamber having multiple monitored storage plates. [Figure 5c] FIG. 5c is a perspective view of a storage area monitored by the object tracking system of FIGS. 5a and 5b, and showing the initial position P1, change in position dP, and final position P2 of the tracked object relative to the coordinate system. [Figure 5d] FIG. 5d is a digital image taken with the wide-angle fisheye camera of the imaging system used in FIGS. 5a and 5b, which appears distorted due to the optics. [Figure 5e] FIG. 5e is the image of FIG. 5d with the distortion removed by the imaging system using a correction algorithm. [Figure 5f] Figure 5f is a still image taken with the wide-angle fisheye camera of the imaging system used in Figures 5a and 5b for output on the incubator screen, showing the imaging system's bounding box, identification number, and color code identifying the user / owner. [Figure 5g] FIG. 5g is a possible screen content that can be displayed on the incubator screen to illustrate the screen shown in FIG. 5f. [Figure 6] FIG. 6 is a top view schematically showing the storage area of the incubator of FIGS. 1 to 5f, including an object placed within the imaging range of the camera of the imaging system. [Figure 7]FIG. 7 shows an image of an object newly placed in an incubator between two existing objects based on the imaging range of FIG. [Figure 8] FIG. 8 is a diagram showing a schematic sequence of an exemplary method according to the present invention. [Figure 9a] FIG. 9a is an image evaluation for performing a comparison operation by segmentation and determining parameters based on the occupancy of the storage area of FIG. 7 as an example of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0202] FIG. 1 shows an incubator 1 for housing experimental samples, more precisely, a CO2 incubator for housing live cell cultures in a defined atmosphere at a controlled temperature, e.g., 37°C. For this purpose, the incubator's chamber interior 5 is insulated and gas-tightly sealed from the environment; the internal gas composition is likewise controlled and can be changed via a gas connection 43. The incubator's chamber housing 2 rests on a base 44, encapsulating the interior 5 and opening at the incubator's front face 3. This front face has a chamber opening 4 through which the chamber interior 5 can be accessed. A transparent inner chamber door 6 serves to close the chamber opening when the chamber door is in its closed position. In the incubator 1, the chamber housing 2 is disposed inside an outer housing 40, which is spaced apart and insulated from one another. A shelf insert 45 and a humidification tray 46 are visible inside the chamber. In this example, the front face 3 of the chamber housing and the front face of the outer housing coincide.

[0203] The outer incubator door 41 and chamber door 6 are shown in the open position. The outer door 41 is hinged to the outer edge of the outer housing and has a full perimeter seal, in particular a silicone seal 42.

[0204] When the outer door 41 is opened, the inner chamber door 6 of the incubator is still closed for the time being. For this purpose, the closing device (10, 7a, 7b) is in operation. With the chamber door 6 closed, the user can first see the interior 5 through the transparent door wall and then open the door to insert or remove experimental samples. Nevertheless, opening the outer incubator door 41 is already an interference and can potentially damage the incubator atmosphere.

[0205] The incubator has an external camera 65 mounted on the door 41 and facing forward, the images of which can be evaluated by a suitably programmed data processing device of the incubator to identify the user, in particular by facial recognition, so that the external camera 65 connected to the data processing device can function as a user identification device 66. This can also be done by the camera of a smartphone 69.

[0206] In order to protect the experimental samples housed in the incubator, it is effective to minimize the time the interior of the incubator is exposed to the environment (open time). The present invention is based on the observation that the open time can be shortened by using an imaging system 200. The incubator 1 includes an imaging system (not shown in FIGS. 1 and 2).

[0207] As shown in FIG. 2, on the outside of the outer incubator door there is a first screen, a touch screen 61, on which the operating parameters of the incubator 1, such as the temperature of the incubator atmosphere or the partial pressure of the gas inside 5, are displayed.

[0208] On the outside of the outer incubator door 41, a second screen 62 is provided, which may be a touchscreen. However, instead of a second screen, all screen output can also be on one screen. The data processing unit (not shown) of the incubator 1 is programmed to display the occupancy of the interior of the incubator on the screen 62. The screen 62 serves as a "digital window" through which the user can (virtually) see the interior of the incubator. In this case, a graphic reproduction of the interior of the incubator or at least one storage area and its occupancy by existing objects can be programmed so that certain existing objects are graphically highlighted depending on certain criteria or condition parameters.

[0209] The system according to the invention essentially consists of an incubator 1 in which are mounted data storage and processing devices that perform the following functions: Determining at least one occupancy ratio from the image data characterizing occupancy of the at least one storage area. performing at least one mathematical comparison operation comparing the at least one occupancy rate to at least one occupancy criterion value; Recording the result of the at least one mathematical comparison operation in at least one occupancy evaluation parameter. Storing the at least one occupancy assessment parameter in a data storage device.

[0210] 3, the data processing device of the incubator 1 is again programmed to display one or more objects on the display 62 according to at least one condition parameter, here dependent on the user identification data, and according to their respective positions inside the incubator as determined by the imaging system. Existing objects associated with specific user identification data, each identifying a specific user, are highlighted with a user-specific color. A legend 61a for such color coding is shown to the user here in a sub-area 61a of the upper display 61. This legend 61a is shown enlarged in FIG. 4b. User identifiers "Jane", "Joe", etc. are assigned corresponding highlighting colors used in the displays 62, 63.

[0211] FIG. 4 shows that the output displays 61 and / or 62 may alternatively or additionally be external devices having a display 63 in data exchange connection with the incubator and functioning here as components of the incubator system, here a smartphone 69.

[0212] 5a is a schematic front view of shelf inserts 45a, 45b of incubator 1, arranged one above the other as storage plates for objects. The vertical spacing between such shelf inserts 45 in an incubator is typically not large, e.g., 10-40 cm, and in the case of incubator 1, approximately 15 cm. Therefore, multiple cameras must be used to capture the entire storage area 45, in this case the entire storage surface of shelf insert 45b and the "airspace" above it, up to shelf insert 45a. Camera 70 or camera device 70' is or includes a wide-angle or wide-angle fisheye lens camera with an image angle of approximately 200° measured diagonally.

[0213] FIG. 5a shows the imaging system 20 installed within the incubator 1 and is designated by the reference numeral 200 in case it is designed as a retrofit system. The imaging system 20 includes a wide-angle fisheye camera 70, which has a field of view or viewing angle 71a preferably between 160° and 220°, capturing the storage area of the shelf insert 45b located below the camera and the majority (approximately 80%) of the surface of the incubator interior wall portions 72a, 72b. The incubator interior wall portions 72a, 72b, together with the storage plates 45a, 45b extending therebetween, define the compartment 73 of the incubator chamber. Due to the wide viewing angle, a single camera can capture the entire storage area of the shelf insert 45b below it, particularly the (existing) objects 80′, 80, i.e., the stack 80′ of cell culture vessels and the airspace into which the cell culture vessels 80 extend. The nominal viewing angle of the wide-angle fisheye camera is here 200°, but preferably only the image range corresponding to a viewing angle taken from the range 160°-170° is evaluated.

[0214] The camera is positioned vertically above the geometric center of the storage surface of shelf insert 45b. Imaging system 20 also includes illumination device 90 and control device 23, which includes data processor 21 and data memory 22, other components of imaging system 20. Data processor 21 or control device 23 is connected to camera 70 and other cameras (not shown in FIG. 5a) via cable connections 25 that enter the incubator chamber through port 47 in the rear wall of the incubator chamber. Each of these cameras is positioned to monitor the entire storage area (the entire top surface of shelf insert 45; see FIG. 1). Furthermore, control device 23 includes a data interface 24 that allows data connections with components of the incubator equipment, for example, to output data or signals to displays 61, 62, and 63 of the incubator. An illumination device 90 comprising a number of LEDs 90', 90" is assembled above the storage plate 45b and connected to the control device 23 via connecting wires 25. Instead of the two LEDs shown, multiple LEDs can be provided. The storage area 45b can be illuminated by the optional illumination device 90 if appropriate for imaging purposes.

[0215] FIG. 5b is a schematic side view of an imaging system, a component of the incubator of FIGS. 1 to 4b, showing an example of a chamber with multiple monitored storage plates 45a, 45b, and 45c. This representation extends the principle of FIG. 5a, where the incubator chamber is divided into multiple compartments 5a, 5b, and 5c. These compartments are now arranged one above the other and are connected for gas exchange via holes in the storage plates 45a, 45b, and 45c. The storage area or storage plate 45a in compartment 5a is monitored by camera 70', the storage area or storage plate 45b in compartment 5b is monitored by camera 70, and the storage area or storage plate 45c in compartment 5c is monitored by camera 70'', with cameras 70' and 70'' designed and positioned similarly to camera 70 in FIG. 5a. All cameras are connected to the control unit 23 via a connecting cable bundle 26 inside the incubator chamber. The connecting cable bundle 26 transitions to the cable connection 25 already shown in Figure 5a and exits the incubator chamber through port 47 in the rear wall of the incubator chamber. The data processing device 21 of the control device 23 is set to monitor all objects in all three compartments 5a, 5b, 5c. The incubator imaging system 20 shown in Figure 5b now comprises three cameras 70, 70', 70", an illumination device 90, the data processing device 21, the data storage device 22, and connecting lines.

[0216] Figure 5c shows a perspective view of a compartment 5b or storage area 45b monitored by the imaging system of Figures 5a and 5b, and shows object positions P1, P2 relative to a Cartesian coordinate system (x, y, z). If the imaging system is configured as an object tracking system by video data acquisition and digital image processing means, it can also track the change in position dP of an object moving along a path of movement B. The origin of the coordinate system can be fixedly located at a corner of the compartment.

[0217] FIG. 5d shows a digital image taken with the wide-angle fisheye camera of the imaging system used in FIGS. 5a and 5b, which appears distorted due to the optics.

[0218] FIG. 5e shows the image of FIG. 5d after the distortion has been removed by the imaging system using a correction algorithm.

[0219] Figure 5f is a still image taken by the wide-angle fisheye camera of the imaging system used in Figures 5a and 5b and output to the incubator screen, showing the imaging system's bounding box, identification number, and color code identifying the user / owner.

[0220] Figure 5g shows possible screen content that can be displayed on the incubator screen to illustrate the screen shown in Figure 5f. In addition to identifying the object by its identification number, a color code identifying the user / owner and optionally the time the object was placed in the incubator chamber, as registered by the incubator, are also displayed.

[0221] FIG. 6 shows the storage area, i.e., the top surface of shelf insert 45b, in a bird's-eye or overhead view. Furthermore, the imaging range 71 captured by camera 70 is shown schematically. The imaging range 71 is the range captured by camera 70 in one or each image, since camera 70 does not change its viewing angle or position. Each image therefore shows this range 71. In the figure, the lower edge of range 71 represents the area close to incubator chamber opening 4. However, camera 70 and / or lighting device 90 with two light sources (LEDs) 90' and 90" can also be supported movably or displaceably by a transport device 95, here a motorized rail system.

[0222] The imaging systems 20 and 200 here include: a lighting device 90 for illuminating the storage area 49 extending between the interior walls 72a, 72b; capturing, by a camera device 70, at least one image of the storage area 49 extending between the interior walls 72a, 72b; The data processing device 21 is configured to store at least one image in the form of image data in the data storage device 22 .

[0223] The imaging system further comprises: illuminating at least two objects disposed in the storage area (49) with a lighting device (90); acquiring images of at least two objects 80, 80' on the storage area by the camera device 70, i.e., images showing these at least two objects; The image of the at least two objects 80, 80' and the storage area 49 is configured to be stored by the data processing device 21 in the data storage device 22 in the form of image data.

[0224] The data processing device here optionally comprises: evaluating the image data to distinguish between a first object 80 and a second object 80' appearing in the image (in particular to assign different identification data to the first object and the second object, to count the objects, to recognize object classes, to analyze, store and recognize individual features, and to track the objects in motion), and detecting the contours of the first object 80 and the second object 80' in the image using image processing algorithms, in particular; In particular, it is programmed to store information about the first object 80 and the second object 80', in particular the contours of the first object 80 and the second object 80', in the data storage device 22 in the form of object data.

[0225] The illumination device 90 is here optionally operable, and the data processing device is here optionally programmed to operate the illumination device 90 in at least two illumination modes, and the imaging system 20, 200 The lighting device 90 illuminates the storage area 49 of the incubator chamber. First, in the first lighting mode, illuminate with the active LED 90´ and the inactive LED 90″. Then, in a second lighting mode, specifically with the active LED 90" and the inactive LED 90', capturing at least one image of the storage area 49 with the camera device 70 while illuminated by both the first illumination mode and the second illumination mode; The at least one image is configured to provide in the form of image data including composite image information acquired during both the first illumination mode and the second illumination mode, and the data processing device is programmed to execute an image evaluation program to derive the composite image information from the image data.

[0226] The at least one image of the storage area 49 preferably includes at least one first image of the storage area 49 and a second image of the storage area 49 that is different therefrom, the first image being acquired in a first illumination mode and the second image being acquired in a second illumination mode, the first image being provided in the form of first image data and the second image being provided in the form of second image data, and the data processing device and the image evaluation program: combining the first image data and the second image data to obtain combined image data, in particular resulting from adding and / or averaging the first image data and the second image data; The program is programmed to derive the composite information from the composite image data.

[0227] The data processing unit of the imaging system 20, 200 is programmed to acquire and evaluate image data by the camera 70 during illumination in response to detecting a closed state of the outer incubator door 41. By comparing successive images in time, it can be determined whether a new object 81 has reached the imaging range 71.

[0228] 7, an image containing a contour 81a that has newly appeared in the imaging range 71 and can be assigned to an object 81 that has been placed therein is considered to be a modified image. Starting from this modified image, identification data is assigned to this newly appeared contour 81a, assuming that this is an object 81 that is newly placed in the incubator.

[0229] FIG. 8 shows the sequence of the method according to the invention, which has already been mentioned implicitly in the above description of the preceding figures.

[0230] The method 300 is for imaging in an incubator used to cultivate live cell cultures, an incubator chamber for receiving an object, particularly a cell culture vessel, having opposing inner walls, a chamber opening for a user to place or remove an object, and at least one storage area extending between the opposing inner walls for storing the object; an incubator door for closing the chamber opening; an imaging system, A lighting device; A camera device; a data processing device having a data memory, Method 300 includes the following program-controlled steps: illuminating (301) a storage area extending between interior walls with a lighting device; capturing (302) at least one image of a storage area extending between interior walls by a camera device while illuminated; Optionally, storing (303) at least one image in the form of image data in a data storage device; determining (310) at least one occupancy ratio characterizing the occupancy of at least one storage area from the image data; performing (311) at least one mathematical comparison operation comparing at least one occupancy rate with at least one occupancy criterion value; incorporating (312) the result of the at least one mathematical comparison operation into at least one occupancy evaluation parameter; and storing (313) the at least one occupancy assessment parameter in a data store.

[0231] In particular, step 310 for determining the occupancy rate A1 preferably includes at least one of the following steps, which may be performed repeatedly as necessary: Determining contiguous free surface segments 101 (surfaces not occupied by objects), optionally using computer-aided image processing, in particular segmentation, to determine a plurality of contiguous free surface segments. Area A1 of segment 101 (e.g. cm 2 determining the temperature (measured in units of Determining consecutive free surface segments 101 greater than area A2. Optionally using computer-aided image processing, in particular segmentation, to determine a plurality of consecutive free surface segments each greater than area A2. Determining contiguous free surface segments 101 whose shape conforms to predetermined dimensions and are larger than area A2, in particular a suitable dimensioned installation surface A2 for placing laboratory sample containers, where the suitable dimensions M can be selected from a group of various suitable dimensions M that can be stored in a data storage device, each suitable dimension being assigned to one of a plurality of laboratory sample containers, e.g., SBS standard microtiter plates, cell culture bottles of different sizes, and Petri dishes of various sizes.

[0232] The step 311 of performing a mathematical comparison operation to compare at least one occupancy rate with at least one occupancy criterion value includes in particular the following steps: Using a mathematical comparison function V=V(A1;A2), determine whether A1≧A2 holds for area A1 and area A2.

[0233] The step 312 of incorporating the result of the at least one mathematical comparison operation into at least one occupancy evaluation parameter includes, inter alia, the following steps: Determining an occupancy evaluation parameter PBB=V=(0 or 1), here Boolean, where PBB=1 if area A1 is greater than or equal to area A2, and PBB=0 if area A1 is less than area A2.

[0234] FIG. 9a shows the situation in image 71 of storage area 45b, evaluated in FIG. 7 using computer-aided image processing. Segmentation identifies a surface segment 101 that is not occupied by cell culture vessels 80′, 81, and 80. The area of this surface segment 101 is determined as A1. The total area of this storage area can be divided into a grid for computational purposes, and the positions of the grid points can be recorded in a Cartesian coordinate system (x;y). Grid regions or grid points can have an occupancy attribute BA (occupied or free). Occupied areas with specific dimensions can be assigned to objects. Free areas with specific dimensions A2 can be assigned to free installation areas 102. Free installation areas 102, 104 can be continuously compared by determining free segment surfaces 101, 101′ and comparing them with at least one value A2 or A4 and the appropriate dimensions (FIG. 9b). This automatic occupation continues until the area of storage area 45b is optimally utilized by installation areas 102, 104, etc. Such automatic planning can be used to implement a user guidance system.

[0235] Instead of dynamically allocating footprints 102, 104 to the dimensions of the storage area, it would be possible to use a grid with predetermined footprints, which are then imaged and identified as "vacant" or "occupied."

[0236] Preferably, the following steps are also performed in method 300: Monitoring the changes over time in the incubator chamber 2, 5 by at least one camera 70 of an incubator camera system 70', the camera 70 being positioned to capture an image of at least one storage area 49 inside the incubator chamber in which at least one object 80; 80'; 81 is placed (304). Assigning identification data to at least one object 80; 81, the object being captured in an image of the storage area 49 taken by at least one camera 70 after being positioned within the storage area (305). Storing (306) the position of at least one object 80;81 in the data memory as ID position data depending on the identification data of the at least one object.

[0237] Preferably, method 300 also includes the following steps: reading, by means of a user identification device (307), user identification data identifying the user of the incubator 1 who introduced at least one object 80; 81 into the incubator chamber, and storing said user identification data in the data memory of the incubator (308);

[0238] Preferably, method 300 also includes the following steps: Storing the ID location data as ID location data associated with the user according to the user identification data (309).

Claims

1. 1. A system for monitoring the occupancy of storage locations in at least one incubator for culturing live cell cultures, comprising: at least one incubator for culturing live cell cultures; The incubator comprises: an incubator chamber for storing objects, particularly cell culture vessels, comprising opposing inner walls, a chamber opening for a user to place and remove objects, and at least one storage area extending between the opposing inner walls for storing objects; an incubator door for closing the chamber opening; An imaging system; and The imaging system includes: A lighting device; at least one camera device; a data processing device; Equipped with The imaging system includes: illuminating at least one storage area extending between the interior walls with the lighting device; capturing, by said camera device, at least one image in the form of image data of at least one storage area extending between said interior walls; It is set as follows: the system includes a data storage device and a programmable data processing device; The programmable data processing device comprises: determining at least one occupancy ratio characterizing the occupancy of the at least one storage area from the image data; performing at least one mathematical comparison operation comparing the at least one occupancy rate to at least one occupancy criterion value; incorporating a result of the at least one mathematical comparison operation into at least one occupancy evaluation parameter; storing at least one occupancy assessment parameter in said data storage device; The system is programmed to do so.

2. 2. The system according to claim 1, wherein the data processing device is programmed to determine from the image data as an occupancy rate a preferably contiguous sub-area of at least one storage area, in particular a sub-area of the storage area that is not occupied by an object, and to determine the size of this free storage space.

3. 3. The system of claim 2, wherein the data processor is programmed to perform a mathematical comparison operation to compare the size of the free storage location with a reference value for free storage location.

4. 4. The system of claim 3, wherein the free storage location is a footprint of a location for laboratory sample containers, and the occupancy reference value is a predetermined comparison value appropriate for the footprint.

5. 5. The system of claim 4, wherein the data processing device is programmed to assign location ID data to the location that uniquely identifies the location from other locations.

6. 6. The system according to claim 4 or 5, wherein the data processing device is programmed to ascertain the availability of the installation location, i.e. to answer the question whether the installation location is vacant or occupied, depending on the occupancy assessment parameters, and in particular to store the answer to said question as occupancy assessment data.

7. The data processing device includes: determining the occupancy assessment data in dependence on the occupancy assessment parameters; outputting information regarding the occupancy evaluation data to a user via a user interface device; 7. The system of claim 1, wherein the system is programmed to:

8. The information regarding the occupancy assessment data includes: the number of free locations for laboratory sample containers on a given surface determined using a mathematical comparison operation; the number of occupied mounting locations for laboratory sample containers on a given mounting surface determined using a mathematical comparison operation; above or below a threshold characterizing an underutilization or overutilization rate of at least one storage area of the incubator; The system according to claim 7, wherein the information is any one of the above.

9. 9. The system of claim 1, wherein the system is programmed to generate culture report data for each sample contained in each laboratory sample container by assigning a sample ID to the laboratory sample container that uniquely identifies the laboratory sample container, i.e., by assigning a location ID to the laboratory sample container.

10. 10. The system according to any one of claims 1 to 9, comprising a camera device, which may in particular be the camera device of the imaging system, by means of which at least one image of at least one installation location of at least one storage area can be obtained from image data, in particular to obtain an image file of the installation location.

11. 11. The system according to claim 1, further comprising a positioning and guidance system, the positioning and guidance system comprising the lighting device capable of precisely illuminating an area or spot in a storage area, in particular the installation location.

12. 12. The system of claim 11, wherein the data processing device or data processing unit of the incubator is programmed to set a lighting target in response to the sample ID data and / or the location ID data and to apply directional lighting to the target or location.

13. a user guidance system, the user guidance system including the lighting device; 13. The system of claim 1, configured to assist in positioning laboratory sample containers on and / or between locations of the incubator by illuminating at least one location and / or laboratory sample containers located on the location according to a predetermined sequence plan.

14. Two or more incubators connected to each other for the purpose of data exchange, each incubator comprising: an incubator chamber for storing objects, particularly cell culture vessels, comprising opposing inner walls, a chamber opening for a user to place and remove objects, and at least one storage area extending between the opposing inner walls for storing objects; an incubator door for closing the chamber opening; an imaging system; The imaging system includes: A lighting device; at least one camera device; a data processing device; Equipped with The imaging system includes: illuminating at least one storage area extending between the interior walls with the lighting device; capturing, by said camera device, at least one image in the form of image data of at least one storage area extending between said interior walls; It is set as follows: the system includes a data storage device and a programmable data processing device; The programmable data processing device comprises: determining at least one respective occupancy ratio characterizing the occupancy of at least one storage area of each incubator from the image data; performing at least one mathematical comparison operation comparing the at least one occupancy rate to at least one occupancy criterion value; incorporating a result of the at least one mathematical comparison operation into at least one occupancy evaluation parameter; storing the at least one occupancy assessment parameter in a data storage device; 14. A system according to any one of claims 1 to 13, programmed to:

15. 1. A method for imaging in an incubator used for culturing live cell cultures, comprising: The incubator comprises: an incubator chamber for storing objects, particularly cell culture vessels, comprising opposing inner walls, a chamber opening for a user to place and remove objects, and at least one storage area extending between the opposing inner walls for storing objects; an incubator door for closing the chamber opening; An imaging system; It has The imaging system includes: A lighting device; A camera device; a data processing device; It is equipped with The method comprises: illuminating a storage area extending between the interior walls with a lighting device; capturing, by said camera device, in the form of image data, at least one image of a storage area extending between said interior walls during illumination; determining at least one occupancy ratio characterizing the occupancy of at least one storage area from the image data; performing at least one mathematical comparison operation comparing the at least one occupancy rate with at least one occupancy criterion value; incorporating a result of the at least one mathematical comparison operation into at least one occupancy evaluation parameter; storing the at least one occupancy assessment parameter in a data storage device; A method comprising:

Images