Classification system for items to be transported for a baggage transportation system, such baggage transportation system and method for classifying items to be transported

Abstract

A classification system (1) for a baggage transportation system (100) for an item (50) to be transported, the classification system (1) comprising at least one camera module (2) which captures the item (50) to be transported, comprising: a 3D image sensor (3) which generates 3D image data (5); and / or a 2D image sensor (4) which generates 2D image data (6). The camera module (2) transmits the 3D image data (5) and / or the 2D image data (6) to an evaluation module (15). The evaluation module (15) uses an artificial intelligence model (16) to determine at least one property (20) of the item (50). The evaluation module (15) classifies the item (50) by a rule-based comparison (17) using the determined at least one property (20) and at least one rule provision (18) to determine whether the item (50) can be transported on the baggage transportation system (100).

Description

Technical Field

[0001] This invention relates to a classification system for items to be transported in a baggage transport system, such a baggage transport system, and a method for classifying items to be transported. Background Technology

[0002] Inspection of items (such as luggage and mail) is essential in many industrial sectors, such as logistics or airport baggage inspection, to detect defective or dangerous goods. Such inspections are typically conducted by designated personnel. For example, professionals alongside airport conveyor belts inspect luggage items moving along the conveyor belts to identify and reject non-transportable items or special baggage.

[0003] A problem with inspecting transported items is that the speed and reliability of the inspection depend on the personnel performing it. This can lead to longer downtime for the baggage transport system than actually needed. Furthermore, defective or hazardous items may go undetected due to human error. If such items are not detected, it can damage the baggage transport system, potentially causing congestion and subsequent costs. Therefore, the baggage transport system becomes vulnerable and cannot be optimally utilized or operate at full capacity. Thus, technical and personnel redundancy must be implemented at each stage to enable rapid response to such issues. Summary of the Invention

[0004] Therefore, the objective of this invention may include: providing a classification system for items to be transported in a baggage transport system, as well as such a baggage transport system and a corresponding classification method for items to be transported.

[0005] This task is solved by the classification system according to claim 1, the baggage transportation system according to claim 14, and the classification method according to claim 15. Beneficial extensions to this classification system are given in claims 2 to 13.

[0006] The classification system according to the invention is used in baggage transportation systems for classifying items to be transported, such as luggage or mail items. Such baggage transportation systems can be used, for example, at airports. The classification system includes at least one camera module designed to acquire, particularly optically, at least one item to be transported. The at least one camera module includes, particularly, a time-of-flight (TOF) 3D image sensor. The 3D image sensor is designed to generate 3D image data of the item. Additionally or alternatively, the at least one camera module includes a 2D camera designed to generate 2D image data of the item. The classification system also includes an evaluation module, wherein the camera module is designed to transmit 3D image data and / or 2D image data to the evaluation module. The evaluation module is designed to use an artificial intelligence model to determine at least one attribute of the item. The evaluation module is also designed to classify the item by rule-based comparison, utilizing the determined at least one attribute and at least one rule, to determine whether the item can be transported on the baggage transportation system.

[0007] Using a camera module that includes a 3D image sensor and / or a 2D camera offers advantages. A 3D image sensor provides precise depth information, which is crucial for analyzing the spatial location and structure of an object. This, combined optimally with the high-resolution image information from a 2D camera, enables a more accurate understanding of the object's shape, size, and surface characteristics. The 2D camera provides detailed color and texture information, which, when combined with the 3D shape information, improves the determination of the object's properties. This is particularly useful in complex scenarios where objects may have similar shapes but differ in color or texture, such as luggage. 3D image sensors, especially those based on Time-of-Flight (ToF) or structured light technologies, can operate effectively regardless of lighting conditions. This complements the capabilities of 2D cameras, which provide optimal results under ideal lighting conditions.

[0008] Furthermore, and particularly preferably, the classification is conducted in two stages. In the first stage, at least one attribute of the item to be transported is determined. Using an artificial intelligence model is particularly preferred in this stage because it allows for the determination of at least one attribute of the item to be transported in a particularly efficient manner, applicable to, for example, millions of different items. In the second stage, this at least one attribute is compared using rule-based methods. In this rule-based comparison, for example, it is checked whether the attribute falls within the range permitted by the rules. Through this innovative two-stage approach, the classification results are traceable to the user. If errors exist in the classification, the source of the error can be traced. Each stage outputs partial results that are interpretable to the user. Subsequently, based on an abstract decision table, the so-called "business rules," these partial results are merged into an "overall classification." Each step is specifically designed to utilize existing data for processing. Furthermore, each step is traceable and verifiable to the user in terms of content. Therefore, feedback can be integrated, and requirements can be parameterized.

[0009] So-called 3D image data refers to image data generated by a 3D image sensor. Specifically, 3D image data contains depth information, such as the distance of an object represented in the image data relative to the 3D image sensor. In particular, the 3D information can be calculated in at least one camera module by a computing unit within that module, for example, by an ASIC (Application-Specific Integrated Circuit) such as an ISP (Image Signal Processor). Specifically, the 3D information allows for the precise determination of the three-dimensional dimensions of an object, such as its size, shape, and orientation. Specifically, 3D image sensors include time-of-flight image sensors, 3D stereo cameras, and / or SLS image sensors (structured light scanning image sensors).

[0010] 2D image data can be “regular” image data, representing conventionally captured two-dimensional images. For example, 2D image data can include color information, particularly high-resolution 2D-RGB information, and / or grayscale information for a large number of image pixels. Grayscale information can be generated, for example, by an infrared camera, particularly under active illumination, or by converting RGB information from an RGB camera to grayscale. Classification of item attributes can be performed using, particularly only, color information, and / or using, particularly only, grayscale information. In some cases, using color information may be more advantageous for classifying item attributes, such as in item segmentation; while in other cases, using grayscale information may be more advantageous, such as when the background has the same color as the item, or when classifying the material properties of the item, where the grayscale information includes, for example, an infrared grayscale image captured using infrared light and presented in grayscale levels.

[0011] In preferred extensions, at least one attribute exists in a human-readable form. This can be in text form and / or image form. Therefore, users can immediately verify whether the AI ​​model has correctly assigned the attribute to the item.

[0012] In a preferred extension, the evaluation module is designed to output at least one attribute to the user. This output can, for example, be displayed on a computer screen. Alternatively, the output can be performed on a web server, accessed by the user's computer, which then displays the attribute. This allows the user to immediately identify which attribute has been assigned to the item.

[0013] In a preferred extension, the evaluation module is designed to output at least one attribute to the user, along with the classification result and / or at least one rule specification. Such output can be, for example, displayed on a computer screen. Alternatively, the output can be delivered, for example, on a web server, which the user's computer accesses and displays the attribute. Thus, the user not only sees the attribute but also the result of a rule-based comparison or the corresponding rule specification by which the attribute is compared. The user can immediately determine whether the result of the rule-based comparison matches the determined attribute.

[0014] In particular, in one extension, the evaluation module is designed to output to the user only at least one attribute that fails to meet the rule requirements based on the rule-based comparison, specifically, along with the classification result and / or at least one rule requirement. Therefore, the user can immediately know the reasons why the evaluation module classifies the item as untransportable. For example, if the maximum length of the item is 150cm, while the maximum length of items that the baggage transport system can transport is 140cm, then only the attribute reflecting the length of the item will be output to the user.

[0015] In a preferred extension, an input unit is provided, designed to receive user input, wherein at least one attribute can be redefined based on the received user input. In this case, attributes that might have been incorrectly determined can be redefined. This is also referred to as a "mark".

[0016] In particular, in one extension, the evaluation module is designed to be able to redetermine the attribute based on user input, either before or after classification using rule-based comparisons.

[0017] In one extension, the AI ​​model can be trained using received user input. For example, if the AI ​​model assigns the attribute "suitcase" to an item that is actually a handbag, the user can redefine that attribute using their user input, thereby training the AI ​​model, especially in operation.

[0018] In one particular extension, the evaluation module is designed to utilize artificial intelligence models to determine at least two, three, four, five, six, or more attributes of an item. This allows for a significantly more accurate characterization of the item.

[0019] In particular, in one extension, the evaluation module is designed to compare each defined attribute against the corresponding rule specification in a rule-based comparison. For example, if there are five attributes, then there are at least five rule specifications.

[0020] In a preferred extension, if at least one previously determined attribute is determined with a probability below a threshold, the evaluation module will determine an additional attribute for the item. In such a case, during rule-based comparison, at least one additional attribute can be used to classify whether the item can be transported on the baggage transportation system. This takes into account a situation where an attribute cannot be reliably determined. In this case, other attributes will be introduced for decision-making. Conversely, if an attribute is determined with a probability above a threshold or above another threshold, then rule-based comparison can be initiated directly without determining the additional attribute. This saves time.

[0021] In the preferred extension, at least one attribute is selected from at least one of the following features:

[0022] a) Color; and / or

[0023] b) Shape, such as round, square, oblong, and / or flat; and / or

[0024] c) Materials, such as plastics, metals, cardboard, and / or fabrics; and / or

[0025] d) Dimensions, such as length, width, and / or depth; and / or

[0026] e) Texture, such as soft and / or hard; and / or

[0027] f) Categories, such as suitcases, handbags, mailed items, and / or sports equipment; and / or

[0028] g) Abnormal characteristics, such as protruding straps, pull-out handles, and / or plastic film packaging; and / or

[0029] h) weight; and / or

[0030] i) Weight distribution.

[0031] If such attributes are output, users can verify them directly. Weight or weight distribution can be determined using an additional scale.

[0032] The relevant rules may specify, for example, the maximum length, width, and / or depth an item can have. If, in a rule-based comparison, the attribute representing the maximum length, for example, exceeds the maximum allowed length specified by the rules, then the item can be rejected. Alternatively or additionally, this attribute, along with the violated rule, may be output to the user, if necessary.

[0033] In preferred extensions, a rule is a specification that an attribute must satisfy, or the specification is an extreme value or range that the attribute must conform to. Rule specifications can be customer-specific, application-specific, and / or general. Multiple rule specifications can exist. Therefore, an airport can issue general rule specifications applicable to all items. A single airline can also issue customer-specific rule specifications that apply only to its customers. For example, airline A may allow different sizes than airline B. Application-specific rule specifications may also be provided. For example, airline A's rule specifications for aircraft type X may differ from its rule specifications for aircraft type Y.

[0034] In one particular extension, the rules are presented in a user-readable format. Therefore, users can directly compare defined attributes with the rules.

[0035] In particular, in one extension, different rules may relate to the same attribute, where a general rule defines what requirements that attribute must at least meet. This general rule can be restricted by customer-specific rules, but cannot be extended. Customer-specific rules can be restricted by application-specific rules, but preferably, cannot be extended. This ensures that airport operator regulations are always followed, and therefore, the baggage handling system is not disrupted. Restrictions imposed by airlines or aircraft models will only result in more items being classified as non-transportable compared to situations where general rules apply. For example, if general rules only allow items with a maximum length of 140cm to be transported, customer-specific or application-specific rules can further restrict that maximum length, for example, to 130cm. Although an item with a maximum length of 130cm can be transported through the baggage handling system without issue, it will still be rejected because, for example, there is not enough space on the corresponding aircraft. Therefore, in principle, only items that meet at least one rule requirement can be transported through the baggage handling system. For example, if the specific rules for application are violated, the item may in principle be transported on the baggage transport system without damaging it, but it will still be classified as non-transportable because, for example, it may not be able to be loaded onto the aircraft.

[0036] In particular, in one extension, the evaluation module can be designed to invoke customer-specific rules and / or application-specific rules stored for the item during rule-based comparisons. For example, the item may have tags, such as barcodes, RFID chips, etc., in which the evaluation module uses the information provided by the tag to invoke the corresponding customer-specific rules and / or application-specific rules.

[0037] In particular, in one extension, the evaluation module is designed to determine the airline and / or aircraft model to which an item belongs, and based on the airline and / or aircraft model, to further invoke relevant customer-specific rules and / or relevant application-specific rules for rule-based comparison.

[0038] In a preferred extension, at least one second, one third, or at least one fourth camera module is provided, these camera modules having a 3D image sensor and / or a 2D camera, wherein the at least one attribute is achieved through:

[0039] a) 3D and 2D image data from only one or at least two or all camera modules are determined; or

[0040] b) 3D or 2D image data from only one or at least two or all camera modules is determined; or

[0041] Therefore, this attribute can be determined precisely because of the presence of additional image data.

[0042] In particular, in one extension, the evaluation module is designed to classify an item as non-transportable if its identified attributes do not meet only one rule requirement.

[0043] In one particular extension, different attributes are determined by different camera modules. Therefore, the AI ​​model is designed to determine certain attributes using only image data from a specific camera module.

[0044] In particular, in one extension, at least one specific attribute is determined solely from 3D image data of a 3D image sensor. Alternatively or additionally, at least one specific attribute is determined solely from 2D image data of a 2D camera. When using multiple camera modules, 3D or 2D image data from multiple camera modules can be used together to determine that specific attribute.

[0045] In a preferred extension, at least one second, one third, or at least one fourth camera module is provided, these camera modules having 3D image sensors and / or 2D cameras. These camera modules are arranged around the object to be captured at different orientations and / or angular positions, such that their respective 3D image data and / or 2D image data at least partially encompass different regions and / or sides of the object. Thus, the object can be captured with particularly high precision, allowing attributes to be determined very accurately, i.e., with extremely high probability, by an artificial intelligence model.

[0046] In particular, in one extension, the exposure time of the 2D camera in at least one camera module is adjusted according to the speed of the conveyor belt transporting the goods. Therefore, a longer exposure time can be used when the conveyor belt is moving more slowly compared to when it is moving faster.

[0047] In particular, in one extension, the at least one camera module is designed to take a picture of the item when it is on the conveyor belt and guided past it—that is, when it is in motion. Therefore, there is no need to stop the conveyor belt.

[0048] In particular, in one extension, the 3D image sensor and the 2D camera have substantially the same field of view, overlapping fields of view, or fields of view adjacent to each other. A field of view refers to the area in image data that is imageable and contains objects. Preferably, for example, at least 90% of the spatial angles of the fields of view of the 3D sensor and the 2D camera are the same. Preferably, the visual axes (i.e., orientations) of the 3D image sensor and the 2D camera are parallel.

[0049] In one extension, the AI ​​model is designed to use additional image data from other camera modules if at least one attribute to be determined is determined with a probability below a threshold. This improves the accuracy of attribute determination.

[0050] In the preferred extension, the evaluation module is designed to:

[0051] a) 3D image data from multiple camera modules; and / or

[0052] b) 2D image data from multiple camera modules; and / or

[0053] c) 3D and 2D image data from a camera module; and / or

[0054] d) 3D and 2D image data from multiple camera modules;

[0055] In particular, fusion is performed at least partially, largely, or completely. Based on 3D and 2D image data from each camera module, multi-view panoramic images of the environment and / or objects can be acquired, and thereby, in particular, more accurate or complete images of reality can be created. During fusion, for example, depth information can be additionally appended to the 2D image data.

[0056] In particular, in one extension, 3D image data and 2D image data are aligned (koregistriert). Specifically, this alignment already exists when the respective image data is generated, because the 3D and 2D image data are generated within the same component, i.e., within at least one camera module. Specifically, "alignment" refers to the spatial and temporal alignment of the 3D and 2D image data to achieve accurate and consistent rendering. Specifically, there is no need for the cumbersome synchronization of 3D and 2D image data as is required in image data from, for example, different camera modules.

[0057] In particular, in one extension, the evaluation module is designed to perform time synchronization of the individual 2D image data and / or individual 3D image data from the camera modules. Therefore, the evaluation module is designed as a central control unit and evaluation unit. Specifically, the images captured by the individual camera modules, i.e., the 2D and / or 3D image data used for item classification, can be coordinated temporally with respect to their respective time points, and in particular, have the same timestamp.

[0058] In particular, in one extension, the at least one camera module is fixedly set during runtime.

[0059] In particular, in one extension, the at least one camera module does not include a display, and more specifically, does not include a touch-sensitive display.

[0060] In particular, in one extension, the at least one camera module is designed to capture objects even without user interaction.

[0061] In particular, in one extension, the at least one camera module is designed to switch from an operating state to a standby state. In the operating state, objects are captured; in the standby state, the at least one camera module does not generate 3D image data and / or 2D image data, nor does it transmit 3D image data and / or 2D image data to the evaluation module. This reduces energy consumption.

[0062] In particular, in one extension, the camera module is designed to recognize motion, especially the motion of objects on a conveyor belt, and to switch from a standby state to an operating state when such motion is recognized.

[0063] In one extension, motion recognition is performed solely using a 3D image sensor or a 2D camera. Alternatively, this can be achieved by reducing the sampling rate and / or the resolution.

[0064] In particular, in one extension, the apparatus of the at least one camera module also includes a light source, thereby enabling the 2D camera to provide optimal results. Specifically, the light source is designed to generate light with a specific wavelength or wavelength range and to emit the light onto an object, which can be received and processed by the 2D camera. If the 2D camera operates in the infrared (IR) band, the light source also generates infrared light.

[0065] In particular, in one extension, the 2D and 3D image data comprise images taken sequentially over time. In this case, the 2D and 3D image data contain images arranged in chronological order. By using such chronologically arranged images, the artificial intelligence model is able to determine at least one attribute more accurately, i.e., with a higher probability.

[0066] In one particular extension, the 3D image sensor and 2D camera are integrated into a single component, namely, a camera module. In this case, the camera module has a very compact structure. In many situations, such as installing such cameras in baggage check reader lanes, installation space is a crucial factor. Especially in hard-to-reach areas, the integration of the camera module can be significantly simplified. Furthermore, the alignment of the 3D image sensor and 2D camera can be verified during the manufacturing of the camera module. Therefore, cumbersome on-site installation and calibration are unnecessary.

[0067] In one particular extension, the classification system comprises only one camera module. This keeps the installation space required for the classification system low. In principle, the 3D image sensor and the 2D camera can also be mounted in separate housings.

[0068] In particular, in one extension, the 2D camera is a color camera. Additionally or alternatively, the 3D image sensor is a 3D depth sensor. This improves the accuracy of determining at least one attribute of the article.

[0069] In particular, in one extension, the 2D camera and / or 3D image sensor have a high image refresh rate, a large field of view, and / or high resolution.

[0070] In one extension, the evaluation module includes an artificial intelligence model. Alternatively, the artificial intelligence model can be centrally located, allowing different evaluation modules to use the same model.

[0071] In particular, in one extension, the 2D camera includes a computing unit designed to dynamically, and especially automatically, adjust the image capture parameters of the 2D camera. This computing unit is, for example, a smart chip, such as an ISP. In other words, the 2D camera has automatic adjustment functions, such as automatic gain control, automatic white balance, and automatic exposure. Specifically, the automatic adjustment can be performed within preset limits for various applications. For example, the maximum exposure time can be preset to prevent or at least reduce motion blur.

[0072] In one particular extension, at least one camera module and an evaluation module are interconnected via a cable connection (e.g., an electrical or optical connection). Wireless connectivity can also be used in principle.

[0073] In particular, in one extension, the cable connection is designed as a coaxial cable. Preferably, at least one camera module is powered via this coaxial cable.

[0074] In particular, in one extension, at least one camera module and an evaluation module are interconnected via a single connecting cable. In this case, only a single cable is needed to electrically connect at least one camera module to the evaluation module (i.e., for data connection and power supply), significantly simplifying the connection between the at least one camera module and the evaluation module. This is especially beneficial in hard-to-reach areas, where connecting at least one camera module can be significantly simplified.

[0075] In particular, in one extension, the 3D image sensor includes an integrated processing unit, such as a DSP, which can calculate depth information based on raw 3D data (i.e., phase information of the emitted light and the light subsequently scattered back). The raw 3D data can (initially) be 3D image data. Furthermore, the processing unit can filter out invalid pixel information according to variable criteria and perform preprocessing steps (before conversion to depth information) and post-processing steps, which can be parameterized, in particular, through an evaluation module. The processing unit can add state information about the pixel data (e.g., metadata, confidence data) to the 3D image data.

[0076] In particular, in one extension, the 3D image sensor is a TOF (Time-of-Flight) sensor or an iTOF (Indirect Time-of-Flight) sensor, especially a laser scanner or LIDAR (Light Detection and Ranging). Specifically, the 3D image sensor may include an emitting light source that emits outgoing light within a monitored area. Within the monitored area, the outgoing light can illuminate an object, which reflects, i.e., scatters back, the outgoing light in the direction of the 3D image sensor. The scattered outgoing light captured by the 3D image sensor can be used (directly or indirectly) to assess the time of flight of the light to determine the distance to the object. The outgoing light can be emitted to different ranges within the monitored area to generate a depth image of the monitored area with a large number of image points.

[0077] In particular, in one extension, the 2D camera is a monochrome or color camera, and preferably has a resolution of at least 4 megapixels, 8 megapixels, or 12 megapixels. Specifically, the 2D camera may have optics and an image sensor located behind the optics. An image of the monitored area is projected onto the image sensor through the optics. The image sensor may have the aforementioned resolution of at least 4 megapixels, 8 megapixels, or 12 megapixels, and may be designed, for example, as a CCD sensor or a CMOS sensor.

[0078] Therefore, the classification system can be adapted and / or expanded according to application and needs, allowing for flexible use. High-precision classification can be achieved by combining, in particular, high-resolution multi-view 2D and 3D image data, as different viewpoints can be acquired in detail. This allows for reliable object detection and / or classification, as even tiny objects or anomalies can be accurately identified. In particular, the evaluation module can reduce the collected image data, i.e., the 2D and / or 3D image data from the camera module, to basic information for further processing, especially for object classification.

[0079] In particular, in one extension, the AI ​​model can be trained, or trained entirely, on 3D and / or 2D example image data. The example image data comprises 2D and / or 3D image data acquired by corresponding camera modules, in which one or more objects are displayed, wherein the corresponding objects are displayed for corresponding attributes. Specifically, the attributes of the objects can be pre-given and therefore can be labeled data. Additionally or alternatively, the example image data can also include data fused by the camera modules, with the AI ​​model trained using this fused data. Furthermore, the AI ​​model can be trained using image data based on different lighting conditions, particularly light intensity, exposure time, and / or different ambient light, to make the AI ​​model less sensitive to lighting conditions. This makes the system particularly stable.

[0080] The artificial intelligence model can be, for example, an artificial neural network, particularly a convolutional neural network (CNN). For instance, the AI ​​model can be executed by an evaluation module or an external computing device connected to the evaluation module, and is responsible for determining the attributes of the item. Preferably, the attribute can represent characteristics of the item, such as different sizes, different types and / or shapes, different values ​​and / or states. Additionally or alternatively, the item can be classified by the AI ​​model into two categories, and more specifically, only two categories, where one category represents a "normal" or "transferable" state, and the other represents an "abnormal" or "non-transferable" state.

[0081] Furthermore, the hardware and software can be adapted and optimized for the use of artificial intelligence models; in particular, the hardware can be designed to process large amounts of raw data. For example, the hardware can be designed, in particular, to process large amounts of raw data from the camera module at processing speeds up to 40 Gbit / s. Thus, the hardware may include, for example, GPU-accelerated chips or computing devices optimized for artificial intelligence applications. This enables real-time classification of items, wherein, in particular, the classification time is less than 1 second, less than 0.5 seconds, and preferably less than 0.1 seconds.

[0082] In one extension, the AI ​​model is trained to perform rule-based comparisons.

[0083] In one particular extension, the AI ​​model comprises an input layer, optional one or more intermediate layers, and an output layer. The input layer has numerous nodes, which can be fed 3D and 2D image data from at least one camera module. The output layer contains at least one node, at which the properties of an object can be output.

[0084] In particular, in one extension, the artificial intelligence model comprises at least one first neural network and a second neural network, wherein the first neural network is designed and used to determine a first attribute of an item, and wherein the second neural network is designed and used to determine a second attribute of the item. Specifically, the first neural network is a different neural network from the second neural network. Preferably, the first and second neural networks are trained on completely different or largely different data.

[0085] In particular, in one extension, the evaluation module is designed so that the first and second neural networks run in parallel with each other in the artificial intelligence model.

[0086] In particular, in one extension, the artificial intelligence model includes a separate neural network for most or all attributes.

[0087] In a preferred extension, rule-based comparison includes multiple attributes, each with at least one rule specifying it. Furthermore, rule-based comparison establishes a hierarchy among the attributes, where if an attribute, along with its rule specifying it, classifies an item as non-transportable, and if that attribute follows another attribute, along with its rule specifying it as transportable, in the hierarchy, the item is still considered transportable. For example, if the first attribute is that the item's length is 220cm, and the corresponding rule specifies that only items shorter than 150cm are transportable, and the second attribute specifies the item type as a surfboard, and the corresponding rule specifies that surfboards are transportable, then if the second attribute precedes (i.e., precedes) the first attribute in the hierarchy, the item is classified as transportable through rule-based comparison.

[0088] In particular, in one extension, the evaluation module is designed to take this hierarchy into account.

[0089] In particular, in one extension, an input unit is provided, which is designed to receive user input, wherein the evaluation module is designed to determine and / or change the hierarchy based on the received user input.

[0090] In a preferred extension, the evaluation module is designed such that, in rule-based comparisons, a first comparison result of a first attribute (and at least one rule specifying the first attribute) is interoperated with at least one second comparison result of a second attribute (and at least one rule specifying the second attribute) through at least one logical connection. Based on the result of this logical connection, the item is classified to determine whether it can be transported in the baggage transport system. Thus, more complex rules can be established, indicating whether an item can be transported in the baggage transport system. For example, "(Item type = ski) and (Length > 150cm)" can indicate that if the item's length is greater than 150cm, and the item is a ski, then the item is classified as transportable. Other items exceeding 150cm in length are classified as non-transportable.

[0091] In particular, in one extension, logical connection refers to logical operations, especially AND, OR, NOT, XOR, NAND, or XNOR operations.

[0092] In particular, in one extension, an input unit is provided, which is designed to receive user input, wherein the evaluation module is designed to extract logical operations from the user input.

[0093] In particular, in one extension, an input unit is provided, designed to receive user input, wherein the evaluation module is designed to select a first attribute based on the user input, and optionally, to select at least one corresponding rule for the first attribute. Additionally or alternatively, the evaluation module is also designed to select a second attribute based on the user input, and optionally, to select at least one corresponding rule for the second attribute.

[0094] In particular, in one extension, the camera module includes at least two 3D image sensors arranged spaced apart from each other and designed to generate 3D image data of the object from different angular positions. Additionally or alternatively, the camera module also includes at least two 2D cameras arranged spaced apart from each other and designed to generate 2D image data of the object from different angular positions or viewpoints. This improves the accuracy for determining the object's properties. Furthermore, if one of the at least two 3D image sensors fails, and / or one of the at least two 2D cameras fails, the object's properties can still be determined.

[0095] In particular, in one extension, 3D image data from at least two 3D image sensors are input into the same neural network of the artificial intelligence model or a different neural network of the artificial intelligence model. Additionally or alternatively, 2D image data from at least two 2D cameras are input into the same neural network of the artificial intelligence model or a different neural network of the artificial intelligence model.

[0096] In particular, in one extension, the AI ​​model is designed to determine all attributes from all 3D image data and / or all 2D image data. Additionally or alternatively, the AI ​​model is designed to determine at least one attribute, but not all attributes, from 3D image data, but not from 2D image data. Alternatively or alternatively, the AI ​​model is designed to determine at least one attribute, but not all attributes, from 2D image data, but not from 3D image data.

[0097] In one extension, the AI ​​model includes at least two neural networks designed to determine the same attribute, with the attribute assigned to the neural network having the highest confidence value. The confidence value in a neural network is a numerical value representing the probability or certainty with which the neural network makes a particular prediction. Confidence values ​​typically range from 0 to 1 (i.e., 0% to 100%), with values ​​closer to 1 indicating high certainty and values ​​closer to 0 indicating low certainty. If the attribute is the "type" of an item, and the first neural network assigns the value "suitcase" to the attribute with a 90% confidence value, while the second neural network assigns the value "handbag" to the attribute with a 50% confidence value, then the value "suitcase" will be assigned to the attribute.

[0098] The classification system can be designed such that the camera module contains only a 3D image sensor or a 2D camera. In this case, only 3D or 2D image data is generated, and the generated data is transmitted to the evaluation module. Furthermore, the evaluation module is designed to use an artificial intelligence model to determine at least one attribute of the item. Additionally, the evaluation module is designed to classify the item using rule-based comparisons, leveraging the determined attribute and at least one rule, to determine whether the item is suitable for transport in the baggage transportation system.

[0099] The baggage transport system according to the invention includes the aforementioned classification system. The baggage transport system is designed such that when an item to be transported, particularly luggage or mail items, is classified by the classification system as untransportable by the baggage transport system, the item is rejected. In particular, the baggage transport system includes a conveyor, such as a conveyor belt. By rejecting these items, it can be ensured that the baggage transport system and the corresponding items are not damaged.

[0100] The classification method according to the present invention is used in a baggage transportation system for classifying items to be transported. In a first method step, the items to be transported are captured by at least one camera module, which includes a 3D image sensor and / or a 2D camera. Furthermore, 3D image data and / or 2D image data are generated. In a second method step, the 3D image data and / or 2D image data are transmitted to an evaluation module. In a third method step, the evaluation module, using an artificial intelligence model, determines at least one attribute of the captured item based on the 3D image data and / or 2D image data. In a fourth method step, the evaluation module, using the determined at least one attribute and at least one rule, classifies at least one item according to a rule-based comparison to determine whether the at least one item can be transported on the baggage transportation system. Attached Figure Description

[0101] The invention will now be described illustratively with reference to the accompanying drawings.

[0102] Figure 1 An embodiment of a baggage transport system according to the invention, having a classification system according to the invention, is shown.

[0103] Figure 2 Another embodiment of the baggage transport system according to the invention is shown, wherein the classification system includes multiple camera modules.

[0104] Figure 3 An embodiment is shown, which describes in more detail the operating principle of the classification system.

[0105] Figure 4 An example is shown illustrating the results of item classification.

[0106] Figure 5 A flowchart is shown illustrating the method for classifying items to be transported in a baggage transportation system.

[0107] List of reference numerals

[0108] 1. Classification System

[0109] 2 camera modules

[0110] 33D image sensor

[0111] 42D camera

[0112] 53D image data

[0113] 62D image data

[0114] 7 First Objective

[0115] 8 Second objective lens

[0116] 9 light sources

[0117] 10 communication devices

[0118] 11 connections

[0119] 15 Assessment Modules

[0120] 16 Artificial Intelligence Models

[0121] 17 Rule-based comparisons

[0122] Rule 18 stipulates

[0123] 20 attributes

[0124] 25 Computer Systems

[0125] 50 items

[0126] 51 External Events

[0127] 100 baggage transportation system

[0128] 101 Conveyor Belt

[0129] 102 Upper-level control system

[0130] S1, S2, S3, S4 method steps Detailed Implementation

[0131] Book Figure 1 A baggage transport system 100 is shown, which includes a sorting system 1 for items 50 to be transported. The baggage transport system 100 also includes a conveyor belt 101 on which items 50 can be transported.

[0132] The classification system 1 includes at least one camera module 2 designed for visual acquisition of the item 50 to be transported. Camera module 2 includes a 3D image sensor 3 and a 2D camera 4. The 3D image sensor 3 is designed to generate 3D image data 5. The 2D camera 4 is designed to generate 2D image data 6. Camera module 2 also includes a first objective lens 7 for the 3D image sensor 3 and a second objective lens 8 for the 2D camera 4. Camera module 2 also includes a light source 9 that emits light towards the item 50.

[0133] The camera module 2 also includes a communication device 10, which is designed to transmit 3D image data 5 and 2D image data 6 to the evaluation module 15 of the classification system 1 via a connection 11. Preferably, the connection 11 uses exactly one connection cable.

[0134] The evaluation module 15 is designed to use an artificial intelligence model 16 to determine at least one attribute 20 of the item 50.

[0135] The evaluation module 15 is also designed to classify the item 50 using at least one attribute 20 and at least one rule specification 18 through rule-based comparison 17 to determine whether the item 50 can be transported on the baggage transport system 100.

[0136] The results of this classification can be displayed to the user in computer system 25.

[0137] Figure 2 Another embodiment of the baggage transport system 100 is shown. In this case, the sorting system 1 includes a plurality of camera modules 2 arranged around the item 50 to be detected at different orientations and / or angular positions, such that the respective 3D image data 5 and 2D image data 6 at least partially contain different regions and / or sides of the item 50.

[0138] Evaluation module 15 design: Using these additional 3D image data 5 and 2D image data 6 through artificial intelligence model 16, at least one or more attributes 20 are determined.

[0139] Each camera module 2 is connected to the evaluation module 15 via connection 11. This connection 11 is a parallel connection 11. In principle, each camera module 2 can also be connected to the evaluation module 15 via a serial connection, especially a daisy-chain connection.

[0140] Figure 3 An embodiment is shown, which describes the operation of the classification system in more detail. The classification system 1 includes multiple camera modules 2, which capture 3D image data 5 and 2D image data 6 of an item 50, and transmit these 3D image data 5 and 2D image data 6 to an evaluation module 15.

[0141] The evaluation module 15 is designed to use an artificial intelligence model 16 to determine at least one attribute 20 of the item 50. Preferably, the artificial intelligence model 16 is a component of the classification system 1. Subsequently, a rule-based comparison 17 is performed using the determined at least one attribute 20 and at least one rule specification 18 to classify the item 50 and determine whether it can be transported on the baggage transportation system 100.

[0142] The diagram also shows that the classification results are output in a user-readable, i.e., understandable, format. In such cases, a "checkmark" or "X" is used to indicate whether object 50 is transportable. For example, Figure 3Four objects 50 are displayed on the right. The first object 50, located in the upper right corner, is transportable. Rule-based comparison 17 shows that all four defined attributes 20 conform to rule 18. Therefore, all attributes are marked with a checkmark. The second object 50, displayed below the first object 50, is not transportable. Rule-based comparison 17 shows that one defined attribute 20 does not conform to rule 18. This attribute is marked with an X. The third object 50, displayed below the second object 50, is not transportable. Rule-based comparison 17 shows that one defined attribute 20 does not conform to rule 18. This attribute is marked with an X. The fourth object 50, displayed below the third object 50, is transportable. Rule-based comparison 17 shows that all four defined attributes 20 conform to rule 18. In this case, both the defined attributes 20 and the information regarding whether these defined attributes 20 conform to rule 18 are presented in a user-readable, i.e., understandable, form. In principle, the corresponding rule 18 can also be displayed next to the defined attribute 20 in a user-readable, i.e., understandable form.

[0143] Figure 4 An embodiment is shown that illustrates in detail the results of classifying object 50, as in... Figure 3 As shown in the diagram, the evaluation module 15 is designed to determine four attributes 20 of object 50. In rule-based comparison 17, each attribute 20 is compared with at least one rule provision 18. The first, second, and fourth attributes 20 all satisfy the corresponding rule provision 18. This result is displayed next to attribute 20 by an indicator (e.g., a checkmark). The third attribute 20 does not satisfy the corresponding rule provision 18. Therefore, another indicator, in this case, in the form of an X, is displayed next to attribute 20. The user can immediately identify which attribute 20 does not satisfy the corresponding rule provision 18. In this case, this refers to the pull handle on the item 50 to be transported, which is in the form of a suitcase.

[0144] Figure 5A flowchart illustrating a method for classifying items 50 to be transported in a baggage transport system 100 is shown. In a first method step S1, the item 50 to be transported is captured by at least one camera module 2, which includes a 3D image sensor 3 and / or a 2D camera 4. Furthermore, 3D image data 5 and / or 2D image data 6 are generated. In a second method step S2, the 3D image data 5 and / or 2D image data 6 are transmitted to an evaluation module 15. In a third method step S3, the evaluation module 15, using an artificial intelligence model 16, determines at least one attribute 20 for the captured item 50 based on the 3D image data 5 and / or 2D image data 6. In a fourth method step S4, the evaluation module 15, using the determined at least one attribute 20 and at least one rule specification 18, classifies at least one item 50 according to a rule-based comparison 17 to determine whether the at least one item 50 can be transported on the baggage transport system 100.

[0145] This invention is not limited to the embodiments described. Within the scope of this invention, all described and / or drawn features can be combined arbitrarily.

Claims

1. A classification system (1) for a baggage transportation system (100) for items (50) to be transported. The classification system (1) includes: At least one camera module (2) designed to capture images of the item (50) to be transported, wherein at least one camera module (2) comprises: a) A 3D image sensor (3), said 3D image sensor (3) is designed to generate 3D image data (5), and / or b) A 2D camera (4), which is designed to generate 2D image data (6); and The classification system (1) further includes an evaluation module (15), wherein the camera module (2) is designed to transmit the 3D image data (5) and / or the 2D image data (6) to the evaluation module (15). The evaluation module (15) is designed to use an artificial intelligence model (16) to determine at least one attribute (20) of the item (50). The evaluation module (15) is also designed to classify the item (50) by means of rule-based comparison (17), using at least one of the determined attributes (20) and at least one rule provision (18), to determine whether the item (50) can be transported on the baggage transport system (100).

2. The classification system (1) according to claim 1, in, At least one of the properties (20) exists in a human-readable form.

3. The classification system (1) according to claim 1 or 2. in, The evaluation module (15) is designed to output at least one of the attributes (20) to the user.

4. The classification system (1) according to any one of the preceding claims. in, The evaluation module (15) is designed to output at least one of the attributes (20) to the user, as well as the classification result and / or at least one of the rules (18).

5. The classification system (1) according to any one of the preceding claims. in, An input unit is provided, wherein the input unit is designed to receive user input, wherein at least one of the attributes (20) is re-determined based on the received user input.

6. The classification system (1) according to any one of the preceding claims. in, If at least one of the previously determined attributes (20) is determined with a probability below a threshold, the evaluation module (15) determines additional attributes (20) for the item (50).

7. The classification system (1) according to any one of the preceding claims. in, The at least one attribute (20) is selected from at least one of the following features: a) Color; and / or b) Shape; and / or c) Materials; and / or d) Dimensions; and / or e) Texture; and / or f) Types; and / or g) Abnormal characteristics; and / or h) weight; and / or i) Weight distribution.

8. The classification system (1) according to any one of the preceding claims. in, The rule (18) is a rule that the attribute (20) must satisfy, or the rule is an extreme value or a range that the attribute (20) must comply with.

9. The classification system (1) according to any one of the preceding claims. in, The rule-based comparison (17) includes a plurality of the attributes (20), each of the attributes (20) having at least one rule provision (18), and the rule-based comparison (17) establishes a hierarchy among the plurality of the attributes (20), wherein if an attribute (20) together with its rule provision (18) classifies the item (50) as non-transportable, and if the attribute (20) is located after another attribute (20) together with its rule provision (18) in the hierarchy, while the other attribute (20) classifies the item (50) as transportable, then the item (50) is still considered transportable.

10. The classification system (1) according to any one of the preceding claims. in, The evaluation module (15) is designed to perform mutual operations in the rule-based comparison (17) by at least one logical connection, combining the first comparison result of the first attribute (20) with at least one rule provision (18) of the first attribute (20) with at least one second comparison result of the second attribute (20) with at least one rule provision (18) of the second attribute (20), so as to classify the item (50) according to the result of the logical connection and determine whether the item (50) can be transported on the baggage transportation system (100).

11. The classification system (1) according to any one of the preceding claims. in, At least one second camera module (2), one third camera module (2), or at least one fourth camera module (2) are provided, wherein the at least one second camera module (2), the one third camera module (2), or the at least one fourth camera module (2) has a 3D image sensor and / or a 2D camera, wherein at least one of the attributes (20) is: a) The 3D image data (5) and the 2D image data (6) of only one or at least two or all of the camera modules (2) are determined; or b) The 3D image data (5) or the 2D image data (6) of only one or at least two or all of the camera modules (2) is determined.

12. The classification system (1) according to claim 11. in, The camera module (2) is arranged around the item (50) to be captured at different orientations and / or angles, such that the respective 3D image data (5) and / or 2D image data (6) at least partially contain different regions and / or sides of the item (50).

13. The classification system (1) according to claim 11. in, The evaluation module (15) is designed to: a) the 3D image data (5) of the plurality of said camera modules (2); and / or b) the 2D image data (6) of the plurality of said camera modules (2); and / or c) the 3D image data (5) and the 2D image data (6) of one of the camera modules (2); and / or d) The 3D image data (5) and the 2D image data (6) of the plurality of camera modules (2); They merged with each other.

14. A baggage transport system (100) having a classification system (1) according to any one of the preceding claims. in, The baggage transport system (100) is designed such that when an item (50) to be transported is classified by the classification system (1) as being unable to be transported by the baggage transport system (100), the item (50) to be transported is removed.

15. A classification method for a baggage transport system (100) for items to be transported (50), said classification method comprising: The item to be transported (50) is acquired (S1) by at least one camera module (2), the camera module (2) including a 3D image sensor (3) and / or a 2D camera (4), and 3D image data (5) and / or 2D image data (6) are generated. Transmit the 3D image data (5) and / or the 2D image data (6) to the evaluation module (15) (S2); Using the evaluation module (15) and an artificial intelligence model (16), at least one attribute (20) of the collected item (50) is determined (S3) based on the 3D image data (5) and / or the 2D image data (6). By means of the evaluation module (15), using at least one of the determined attributes (20) and at least one rule provision (18), at least one of the items (50) is classified (S4) according to a rule-based comparison (17) to determine whether at least one of the items (50) can be transported on the baggage transport system (100).

Images