Method of detecting a fire hazard in a processing plant and a system for the same
The method uses heat information data and machine learning to enhance fire hazard detection in processing plants, improving accuracy and reducing risks by classifying and responding to fire hazards effectively.
Patent Information
- Application Number
- PCT/EP2025/068331
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-07-10
- Filing Date
- 2025-06-27
- Publication Date
- 2026-01-15
AI Technical Summary
Current methods for detecting fire hazards in processing plants are time-consuming and prone to human error, leading to potential fires, downtime, and increased risk of damage, with insufficient accuracy in identifying fire hazards.
A method utilizing heat information data in conjunction with detection zone sensor data to classify and detect fire hazards, assisted by machine learning models like neural networks, to improve detection accuracy and reduce false positives/negatives.
Enhances fire hazard detection accuracy, reducing the risk of fires and downtime by accurately identifying potential hazards, thereby improving safety and processing efficiency.
Smart Images

Figure EP2025068331_15012026_PF_FP_ABST
Abstract
Description
[0001] METHOD OF DETECTING A FIRE HAZARD IN A PROCESSING PLANT AND A SYSTEM FOR THE SAME
[0002] Technical Field
[0003] The present disclosure generally relates to a method of fire hazard plant control of a processing plant, a control device for a processing plant configured to perform said method, and a processing plant comprising the same.
[0004] Background of the Invention
[0005] Machinery, equipment, appliances, tools, or human behavior provide many possible hazards within a processing plant. One option is to use machine learning to identify fire hazards. US2023115475 AA discloses a method for managing plant safety using machine learning, wherein a hazard map is generated, which hazard map identifies a plant hazard, such as a fire hazard, and the hazard rate in association with a plant area of the processing plant.
[0006] However, fire hazards can also be introduced in the form of items to be processed in the processing plant, such as lithium batteries. Potential fire hazards can be identified by inspecting a material batch before being placed in the processing plant for processing. However, this approach is time consuming and human error can result in fire hazards not being accurately identified, thereby introducing a risk of fire in the processing plant.
[0007] Thus, there is a risk with current solutions that fires may start and spread throughout a processing plant, thereby endangering human lives. It is therefore of particular interest to further develop current solutions so as to improve safety of human lives.
[0008] Further, when detecting a fire hazard in a processing plant, and depending on the severity of the fire hazard, it may be necessary to halt operations of the processing plant so as to handle the fire hazard, thereby resulting in downtime of the processing plant, which is undesirable from an productivity perspective. If failing to detect a fire hazard, fire may break out, thereby introducing risk of damages to the processing plant and costly repair and maintenance. It is therefore important to provide a solution for detecting fire hazards, both reliably and accurately, while keeping the number of false positives and the number of false negatives at acceptable levels.
[0009] In view of the above, it is of interest to provide a solution which alleviates problems of the prior art.
[0010] Summary of the Invention
[0011] In view of the above, it is an object of the present disclosure to provide an improved method of fire hazard plant control of a processing plant, a control device configured to perform said method, and a processing plant comprising the same. This, and other objects, which will become apparent in the following, is accomplished by means of the solutions defined in the accompanying claims.
[0012] The present disclosure is based on the inventors’ realization that a method of fire hazard plant control of a processing plant, said method including a step of detecting fire hazard in a processing plant and a step of initiating suitable fire hazard response actions, can be improved by applying various detection methods based on, or assisted by, heat information data to more reliably detect fire hazards in a flow of matter at one or more locations in a processing plant. Depending on the type of fire hazard that is detected and / or any circumstance(s) thereof, a suitable fire hazard response action based on said detection of fire hazard may be initiated, thereby reducing risk of, or improving suppression of, a fire in a processing plant. Thus, both safety of human lives and processing efficiency is improvable by a solution implementing the concept of the above-mentioned fire hazard plant control.
[0013] According to a first aspect of the invention, a method of fire hazard plant control of a processing plant is provided. The processing plant is provided with a transporting arrangement for transporting items to a detection zone and a classification arrangement. Said classification arrangement is configured to classify items present in said detection zone based on at least one feature of said items and optionally to sort items based on item classification. The method comprises: transporting, by means of the transport arrangement, a material batch as a flow of matter to at least one detection zone of the classification arrangement. The method comprises: providing, by means of the classification arrangement, detection zone sensor data of matter captured when said matter was present in one of the at least one detection zone. The method comprises: classifying, based on said detection zone sensor data, at least one item in the flow of matter as an item of at least one item classification. The method comprises: providing classification data comprising item classification of said at least one item or information from which item classification for said at least one item is derivable. The method comprises: establishing, based at least on said classification data and / or said detection zone sensor data, item data of said at least one item. The method comprises: detecting, if said item data is determined to satisfy at least one fire hazard condition, said at least one item as a fire hazard. Said detection of said at least one item as a fire hazard is based on, or assisted by, heat information data relating to: i) the same at least one item, and / or ii) items at least classifiable as one of said at least one item classification of said at least one item. The method further comprising: controlling, in response to a detection of a fire hazard, the processing plant to initiate at least one fire hazard response action.
[0014] Heat information data may serve as additional information beyond said detection zone sensor data, and in particular said classification data, for detecting potential fire hazards. Thereby, fire hazards among items being processed by the processing plant may be detected. Furthermore, using heat information data as additional information for detecting fire hazards may facilitate detection of fire hazards which are difficult to detect based on nonheat information data alone. In addition, it may improve accuracy in detecting fire hazards based on classification data. Thus, the number of false negatives of fire hazards may be reduced. This yields in improved safety of human lives, since the risk of fire is reduced due to fewer fire hazards going unnoticed. Likewise, the number of false positives of fire hazards may be reduced. This yields in improved processing efficiency, since fire hazard response actions are more prone to be initiated when an item detected as a fire hazard is in fact more likely be a fire hazard, thereby justifying initiation of fire hazard response actions which, depending on the nature of the fire hazard response action selected to be initiated, may limit or pause processing at the processing plant. Thus, corroborating classification data with heat information data may improve safety and processing efficiency may be improved.
[0015] In the context of the application, said classification arrangement being configured to classify items present in said detection zone based on at least one feature of said items refers to one or more features which may be used to characterize an item in terms of classification and that said classification arrangement is configured to detect said one or more features and identify an item classification which is determined to match with said at least one or more feature. Information of said at least one or more feature may be provided by said detection zone sensor data. Said one or more features may include at least one material property selected from one or more material property sets. In addition, one or more of said at least one feature may be provided or inferred by other means, such as by manual input or by a preceding detection step (magnetic detection) or discrimination step (e.g., size discrimination).
[0016] In the context of the application, a fire hazard refers to a threat to fire safety. Thus, an item detected as a fire hazard is an item that introduces at least an increased threat to fire safety. As a non-limiting example, an item detected as a fire hazard may be a high priority fire hazard or low priority fire hazard. A high priority fire hazard may be a fire hazard which require immediate attention and should preferably be handled with priority over other impacted processes in the processing plant. A low priority fire hazard may be a fire hazard which does not require immediate attention but further monitoring is advised to monitor if the low priority fire hazard develops into a high priority fire hazard. As a non-limiting example, a high priority fire hazard may be a lithium battery which has ignited. As a non-limiting example, a low priority fire hazard may be a lithium battery having an abnormal temperature but has not yet ignited. The method may comprise a step of assigning a fire hazard priority to a detected fire hazard with one of at least two fire hazard priorities, wherein a first fire hazard priority is said high priority and a second fire hazard priority is said low priority. The selection of, and timing of initiation of, a fire hazard response action may be determined based on fire hazard priority. As a non-limiting example, said detection of said at least one item as a fire hazard is based on heat information data. In this particular context, based on refers to said heat information data being for said detection of fire hazards. As a non-limiting example, a thermal sensor system may be arranged to provide said heat information data during normal operations, either continuously or when prompted for instance if an indicated temperature or a measured temperature exceeds a temperature threshold, thus providing the option if needed to use heat information data in detecting fire hazards. Said heat information data may relate to the same at least one item and / or items at least classifiable as one of said at least one item classification of said at least one item. The latter may be information of items having said at least one feature from which items may be classified to an item classification of said at least item. Alternatively, said items may be already classified in terms of item classification and it is only necessary to compare item classification of said at least one item with item classifications of said item.
[0017] As a non-limiting example, said detection of said at least one item as a fire hazard is assisted by heat information data. In this particular context, assisted by refers to using heat information data indirectly in said detection of fire hazards. As a non-limiting example, heat information data may be used indirectly in detection of fire hazards if the heat information data is used to train a neural network associated with said classification arrangement to detect said at least one item as a fire hazard based on said detection zone sensor data. In other words, heat information data could for instance be used only during a training phase of said classification arrangement to improve detection of fire hazards. Thus, said heat information data may relate to items at least classifiable as one of said at least one item classification of said at least one item.
[0018] As a non-limiting example, said detection of said at least one item as a fire hazard is based on heat information data and assisted by heat information data. For instance, said detection of said at least one item as a fire hazard is based on a first set of heat information data and said detection of said at least one item as a fire hazard is further assisted by a second set of heat information data at least partially different from said first set of heat information data. Thus, heat information may be used to train a neural network associated with said classification arrangement to detect said at least one item as a fire hazard based on said detection zone sensor data and also provide heat information data for facilitating detection of fire hazards not only during a training phase but also during normal operations.
[0019] As a non-limiting example, any training of said neural network may be distributed to at least one further neural network of a respective further classification arrangement for detecting fire hazards, said further classification arrangement arranged to monitor matter and provide detection zone sensor data of matter in a further detection zone located upstream or downstream relative to said at least one detection zone, or located to receive a parallel stream of said matter. As an alternative, the neural network itself may be distributed to at least one further classification arrangement to detect fire hazards.
[0020] In the context of this application, matter refers to material being received by an arrangement of the present disclosure for processing. Processing may include events such as detection, classification, transportation, and / or sorting.
[0021] A material batch may have a limited quantity of matter including items. A material batch may be provided e.g., by means of a container, as a plastic bale, or may be provided from a warehouse storing a material batch or a plurality of material batches. Said matter including items may be provided in form of whole items, damaged items, fragmented items and / or the like. A plurality of material batches may be provided so as to provide a continuous stream of items. The method and arrangement of the present disclosure may be adapted for processing a continuous stream of material batches and / or a continuous stream of matter.
[0022] A detection zone according to the present disclosure may be a region into which items may be provided, wherein said detection zone may be inspected by one or more detectors of said classification arrangement so as to provide detection zone sensor data indicating at least one feature of at least one item within the detection zone during the moment when said inspection by said classification arrangement is made. Item classification refers to classifying an item based on at least one feature of the item into at least one item classification of at least one item classification. An item may be classified into one or more item classifications depending on said at least one feature.
[0023] The processing plant may include means to provide items in at least one stream of matter. The processing plant may include means to process items provided in said at least on stream of matter. Such means may include any one of, or any combination of: a transport arrangement for transporting a flow of material to at least one detection zone; at least one classification arrangement for providing detection zone sensor data of matter in one of said at least one detection zone and for classifying items of said matter into at least one item classification of at least one item classification; a sorting arrangement for sorting items based on item classification; a receiving arrangement for receiving sorted items and optionally any unsorted items, a containment arrangement for containing items detected as fire hazards, a monitoring arrangement for monitoring detection zone sensor data, item data, heat information data, one or more statuses of the processing plant, a control device for controlling one or more of said selection of arrangements, fire suppression means, etc. Said control device may be operable from on-site and / or off-site (i.e. , remotely). As a non-limiting example, said control device may be operable from a control room, which control room is provided with said monitoring arrangement, preferably including at least display means and / or control interface means. As a non-limiting examples, display means includes at least one display and / or at least one status light for indicating a status of the processing plant. As a non-limiting examples, control interface means includes at least one input device. Input device may for example be a computer keyboard, a touch panel, a touch screen (including a touch panel and a display), or communication means for communicating with an external input device, such as a mobile device, a portable computer, either by wired connection or wirelessly.
[0024] According to a second aspect of the invention, a method of fire hazard plant control of a processing plant is provided. The processing plant is provided with a transporting arrangement for transporting items to a detection zone and a classification arrangement. Said classification arrangement is configured to classify items present in said detection zone based on at least one feature of said items and optionally to sort items based on item classification. The method comprises: transporting, by means of the transport arrangement, a material batch as a flow of matter to at least one detection zone of the classification arrangement. The method comprises: providing, by means of the classification arrangement, detection zone sensor data of matter captured when said matter was present in one of the at least one detection zone. The method comprises: classifying, based on said detection zone sensor data, at least one item in the flow of matter as an item of at least one item classification. The method comprises: providing classification data comprising item classification of said at least one item or information from which item classification for said at least one item is derivable. The method comprises: establishing, based at least on said classification data and / or said detection zone sensor data, item data of said at least one item. The method comprises: detecting, if said item data is determined to satisfy at least one fire hazard condition, said at least one item as a fire hazard. Said detection of said at least one item as a fire hazard is based on, or assisted by, heat information data relating to: i) the same at least one item, and / or ii) items at least classifiable as one of said at least one item classification of said at least one item. The method further comprising: controlling, in response to a detection of a fire hazard, the processing plant to initiate at least one fire hazard response action.
[0025] Said assistance of detecting said at least one item as a fire hazard includes: using a machine learning model associated with the classification arrangement to detect said at least one item as a fire hazard based on said detection zone sensor data or a combination of said detection zone sensor data and said heat information data, wherein said machine learning model is trained at least on using said heat information data.
[0026] By using a machine learning model to detect said at least one item as a fire hazard as described above, it may advantageously improve detection of fire hazards using detection zone sensor data without necessarily relying on heat information data which may require an additional thermal sensor system, which may add cost and need of maintenance. Rather, said thermal sensor system may be momentarily provided to provide said heat information data for a training phase of said classification arrangement. Thus, the machine learning model may be trained to detect at least one item as a fire hazard based on the detection zone sensor data as input. In other words, the machine learning model may have been trained to correlate items as detected and classified based on the detection zone sensor data with the heat information data, thereby allowing detection of an item as a fire hazard. As a non-limiting example, the detection zone sensor data may allow detection and classification of batteries, wherein the heat information data may indicate that batteries are fire hazards. Thus, batteries as detected based on detection zone sensor data may be detected as fire hazards as such, without referring to heat information data. Said training and / or trained neural network may be distributed to at least one further classification arrangement.
[0027] Anything disclosed in context of the method according to the first aspect may apply to the method according to the second aspect and vice versa. The following embodiments, optional features, and other disclosure may apply to or be combined with the method according to the first aspect or the method according to the second aspect.
[0028] According to one embodiment, said machine learning model comprises a neural network. The use of a neural network provides significant technical advantages in detecting fire hazards in a processing plant environment. Neural networks may be capable of automatically learning complex, nonlinear relationships within the heat information data and / or between heat information data and detection zone sensor data, whereby a likelihood of a fire hazard may be determined. This may enable more accurate and robust detection compared to fixed-rule or threshold-based systems. Neural networks may enable the system to recognize early or subtle indicators of fire hazards. Furthermore, once trained, such models can operate in real time, including on resource-constrained hardware, making them suitable for continuous monitoring applications. Neural networks may be retrained or finetuned over time, allowing neural networks to adapt to changes in the monitored environment, thereby enhancing long-term performance and safety.
[0029] According to one embodiment, said assistance of detecting said at least one item as a fire hazard includes: using said heat information data to train a neural network associated with the classification arrangement to improve detection of said at least one item as a fire hazard based on said detection zone sensor data. This advantageously improves detection of fire hazards using detection zone sensor data without necessarily relying on heat information data which may require an additional thermal sensor system, which may add cost and need of maintenance. Rather, said thermal sensor system may be momentarily provided to provide said heat information data for a training phase of said classification arrangement. Said training and / or trained neural network may be distributed to at least one further classification arrangement.
[0030] According to one embodiment, said method comprises: further comprising: providing, by means of a thermal sensor system, heat information data of matter when present in one of at least one detection zone, said heat information data including heat information data of items at least classifiable as one of said at least one item classification of said at least one item. Optionally, said thermal sensor system includes a microbolometer. Said heat information data may include heat image data. Said heat information data may provide, or enable establishing of, a temperature of at least one item of said flow of matter and / or of the detection zone. Said temperature may be indicated, i.e. , the heat information provides data from which said temperature may be derived. Said temperature may be measured by means of a suitable sensor arrangement, such as a microbolometer. A microbolometer may refer to a specific type of bolometer used as a detector in a thermal image system (e.g., a thermal camera). A microbolometer may operate as follows: infrared radiation with wavelengths between 7.5-14 pm strikes a detector material of the microbolometer, heating it, and thus changing its electrical resistance. This change in electrical resistance is measured and processed into temperatures which can be used to create an image. Unlike other types of infrared detecting equipment, microbolometers may advantageously operate without active cooling. Moreover, microbolometers provide a cost-efficient solution for providing said heat information data. However, the present disclosure is not limited to the use of microbolometer, rather, the present disclosure may be realized with any other type of infrared detecting equipment known by a person skilled in the art.
[0031] According to one embodiment, the method comprises: providing, by means of a thermal sensor system or said thermal sensor system, heat information data of matter when present in one of at least one detection zone, said heat information data at least relating to the same at least one item, said heat information data comprising at least one of: an indicated temperature and / or a measured temperature associated with said at least one item and / or said at least one detection zone; information from which said indicated temperature and / or measured temperature is derivable. One of said at least one fire hazard condition includes said indicated temperature and / or said measured temperature exceeding a temperature threshold indicating an item as a potential fire hazard. Optionally, said thermal sensor system includes a microbolometer or said microbolometer.
[0032] If said heat information data is determined to satisfy said at least one fire hazard condition, the item associated with said heat information data is detected as a potential fire hazard. As a non-limiting example, if said heat information data satisfies at least one fire hazard condition, such as said indicated temperature or said measured temperature of said item exceeding said temperature threshold, it may indicate that said indicated temperature or said measured temperature is an abnormal temperature. The abnormal temperature may be such that it is appropriate to initiate a potential fire hazard response action to handle said item.
[0033] The method may apply any one of said fire hazard conditions depending on desired performance and / or item classification. With regards to the latter, it may be that certain items may be more easily or more difficulty detectable as fire hazardous item depending on the fire hazard condition used. By adjusting the fire hazard condition with respect for item classification and / or said at least one feature, the method may advantageously reduce the number of false negatives and false positives in terms of fire hazard detections, thus providing a more accurate detection of fire hazards overall. As a result, safety is improved and processing efficiency is increased. Naturally, if sufficient processing power is available, all fire hazard conditions may be implemented and checked if satisfied for said at least one item.
[0034] In the context of this application, an indicated temperature or a measured temperature associated with at least one item of said flow of matter and / or said at least one detection zone may be an average temperature over a region or a maximum temperature in a region, whether said region is identified to belong to a region of an item in the detection zone or in a region of a boundary of the detection zone itself.
[0035] In the context of this application, heat information data may include time-specific and / or pixel-specific heat information data. A thermal sensor system may be arranged to provide heat information data of a detection zone or of matter or any item therein. The thermal sensor system may have a field of view attributed with a spatial resolution, wherein each pixel of said spatial resolution maps to a specific direction and / or a specific location within the field of view of the thermal sensor system. Thus, the heat information data may include for each pixel and for each timepoint for which sensor data is captured, time-specific and pixel-specific heat information data wherefrom an indicated or measured temperature about at least a region of an item and / or said detection zone associated with said each pixel and each timepoint.
[0036] As a non-limiting example, the heat information data may include heat image data from which it is possible to at least derive said indicated temperature. Alternatively, the heat information data may provide a measured temperature. As a non-limiting example, the heat information data may include sensor data captured by means of a microbolometer arranged to provide detection zone sensor data of items present in said detection zone. Some microbolometers available today are configured to provide temperature data of items in a field of view of the microbolometer. In the context of this application, providing temperature data of items within a field of view of a sensor arrangement such as a microbolometer is equivalent to providing a temperature measurement, i.e. , providing a measured temperature. However, it should be understood that such arrangements may need calibration so as to provide more accurate temperature data. Such calibration may include providing at least one black reference and / or a white reference in the field of view of the sensor arrangement. Such calibration may include calibrating settings of the sensor arrangement to provide more accurate temperature data of items depending on measurement circumstances.
[0037] As an alternative to providing a measured temperature associated with an item, the method may provide an indicated temperature or a measured temperature of said at least one detection zone. This may allow inferring at least an indicated temperature associated with said item by principle of analyzing deviations. As a non-limiting example, the method may provide a measured temperature of the detection zone, in which the item results in a deviation from said measured temperature. This deviation may then be used to infer an indicated temperature of said item.
[0038] Moreover, the method may implement the above detection strategies in combination, in other words, the method may include providing heat information data wherefrom an indicated or measured temperature associated with at least one item of said flow of matter as well as an indicated or measured temperature associated with said at least one detection zone. By said combination, an adaptive temperature threshold may be used for determining an abnormal temperature. The ambient temperature in the processing plant may fluctuate throughout the day, thus consequently impacting the temperature associated with said at least one detection zone. Likewise, it may impact temperature of items being processed. This may in effect shift what constitutes as an abnormal temperature. Thus, an adaptive temperature threshold may compensate for ambient temperature fluctuations. However, the present disclosure is not limited to an adaptive temperature threshold; in fact a predetermined temperature threshold which may be valuedependent based on at least one parameter (such as time of day, weather, and / or climate) may likewise be used, or may be constant over time. The latter may be preferable if the processing plant is configured with air conditioning means so as to reduce ambient temperature fluctuations. According to one embodiment, said thermal sensor system includes a microbolometer. The microbolometer may be as set forth in the present disclosure.
[0039] According to one embodiment, the method further comprises: storing said item data and / or said classification data and / or said detection zone sensor data and / or said heat information data on a memory storage device if said item data satisfies said at least one fire hazard condition. Said memory storage device may comprise one or more storage modules for storing said information. Said one or more storage modules may be adapted with storage capacity in light of the amount of data that needs to be stored. The data stored by means of the memory storage device may be cleared, on demand or periodically, so as to maintain sufficient amounts of free space for storing the above mentioned data, or any other data. Each data may comprise other forms of data entries such as location, time, etc. As a further option, the data on said memory storage device may be transmitted to a remote device for storing said data, such as a remote server. This may allow safekeeping of data in case the memory storage device fails or if a fire breaks out in the processing plant, thus acting as a black box. As an alternative, or in combination, a plurality of memory modules may be provided, or a plurality of memory storage devices, so as to provide data redundancy.
[0040] According to one embodiment, said fire hazard response action is selected from a group of fire hazard response actions at least including: providing a fire hazard alert indicating a presence of a fire hazard; verifying, by automatic and / or manual inspection of said item data, if said item detected as a fire hazard is an actual fire hazard; monitoring an item detected as a fire hazard in a subsequent detection zone downstream from the detection zone in which detection zone sensor data was provided for detecting said item as a fire hazard; sorting said item detected as a fire hazard into a transport path for receiving said item; sorting said item detected as a fire hazard into a fire proof container; determining, by flame detection and / or smoke detection, whether said fire hazard is burning and / or glowing; suppressing a fire using a sprinkler system and / or a foam suppression system and / or a ventilation system and / or a gaseous cleaning agent; allocating memory storage in said memory storage device based on file size of at least said detection zone sensor data of said item detected as a fire hazard before receiving said at least said detection zone sensor data of said item; outputting at least said detection zone sensor data of said item detected as a fire hazard for review on a display device; transmitting at least said detection zone sensor data of said item detected as a fire hazard to an auxiliary memory storage device provided at a fire proof location at the processing plant and / or an auxiliary memory storage device provided at a remote location. The method may advantageously initiate one or more of said fire hazard response actions depending on the type of fire hazard detected, thus providing great flexibility in handling fire hazards in a manner which maintains a satisfactory level of safety of humans lives while also allowing an improved processing efficiency.
[0041] As a non-limiting example, the group of fire hazard response actions may include: providing a fire hazard alert indicating a presence of a fire hazard. The fire hazard alert may be provided upon detecting an item as a fire hazard. The fire hazard alert may be a system level alert, i.e. , an alert triggering and / or prompting another action, such as storing item data and any other mentioned data by means of the memory storage device for future reference. The fire hazard alert may be an audio- and / or visual alert for alerting personnel of a presence of an item being detected as a fire hazard. Such audio- and / or visual alert may be provided by an alarm system of the processing plant (warning siren or speaker system) and / or be displayed by one or more display devices of a monitoring arrangement, such as one or more monitors or portable devices (smart devices).
[0042] As a non-limiting example, the group of fire hazard response actions may include: verifying, by automatic and / or manual inspection of said item data, if said item detected as a fire hazard is an actual fire hazard. This may reduce the number of false positive and reduce the number of false negatives. Manual inspection may be performed by personnel provided with said item data or any other relevant data to assess if said item detected as a fire hazard is an actual fire hazard. For instance, is a lithium-battery detected as a fire hazard actually, for the present moment, an actual fire hazard. Automatic inspection may be performed by a monitoring arrangement configured with processing means to perform instructions for initiating a predetermined verification process corresponding or at least mirroring in part said manual inspection in terms of which features are assessed to determine if said item detected as a fire hazard is an actual fire hazard.
[0043] As a non-limiting example, the group of fire hazard response actions may include: monitoring an item detected as a fire hazard in a subsequent detection zone downstream from the detection zone in which detection zone sensor data was provided for detecting said item as a fire hazard. A subsequent or a further classification arrangement may be arranged to provide detection zone sensor data of said subsequent detection zone. Thus, the method advantageously enables monitoring whether an item detected as a fire hazard develops from a low priority fire hazard to a high priority fire hazard, for which other fire hazard response actions may be preferred to initiate.
[0044] As a non-limiting example, the group of fire hazard response actions may include: sorting said item detected as a fire hazard into a transport path for receiving said item. This may advantageously remove an item detected as a fire hazard from said flow of matter. Said sorting may be performed by a sorting arrangement. Said sorting arrangement may include a claw, a robot arm, an ejection sorter (sorting arrangement configured to sort items using a flow of fluid media, gaseous and / or liquid). Items detected as fire hazards which are sorted from said flow of matter may be transported by means of said transport arrangement in order to move along said transport path. A plurality of sorting arrangements may be provided at various locations in the processing plant. The transport arrangement may be configured to collect sorted fire hazard into a single receiving zone or a plurality of receiving zones depending on configuration of said processing plant and / or the type of item detected as fire hazard and / or fire hazard priority.
[0045] As a non-limiting example, the group of fire hazard response actions may include: sorting said item detected as a fire hazard into a fire proof container. Said fire proof container may be provided as one of said receiving zones. By means of a fire proof container, high priority fire hazards, for instance items that are currently burning, may be contained and prevent fire from spreading to other yet unaffected parts of the processing plant.
[0046] As a non-limiting example, the group of fire hazard response actions may include: determining, by flame detection and / or smoke detection, whether said fire hazard is burning and / or glowing. Flame detection may be provided by means of flame detection units. Flame detection units may be optical devices, designed to identify flams at one or more locations along said stream(s) of matter or at any of said at least one detection zones. Smoke detection may be provided by means of one or more smoke detection units. In response to a smoke detection, the arrangement may be initiated into an alerted state wherein said step of detecting an item as a fire hazard is made more strict. This may be advantageous if a current setting of the classification arrangement fails to detect an item burning and / or glowing as a fire hazard, and a revised, more strict setting of the classification arrangement does allow for said item to be detected as a fire hazard.
[0047] As a non-limiting example, the group of fire hazard response actions may include: suppressing a fire using a sprinkler system and / or a foam suppression system and / or a ventilation system and / or a gaseous cleaning agent. This is advantageous so as to suppress an ongoing fire or preventing a glowing item from igniting. Said sprinkler system, or any of said other mentioned systems, may be arranged at a plurality of locations of the processing plant, or along specific transportation paths along which items detected as fire hazards may be transported, or in said fire proof container. Thereby, items detected as fire hazards may be handled while maintaining normal operations of the processing plant, or at least at a reduced level of operation.
[0048] As a non-limiting example, the group of fire hazard response actions may include: allocating memory storage in said memory storage device based on file size of at least said detection zone sensor data of said item detected as a fire hazard before receiving said at least said detection zone sensor data of said item. The method may advantageously prepare itself to receive large amounts of data if needed, thereby allowing in-depth review of item data and other data relating to a detected fire hazard. As a non-limiting example, the group of fice hazard response actions may include: outputting at least said detection zone sensor data of said item detected as a fire hazard for review on a display device. The method may advantageously output said detection zone sensor data, said item data, said heat information data, and any other data, for manual- and / or automatic inspection. The method may output said data in response to said fire hazard alert.
[0049] As a non-limiting example, the group of fice hazard response actions may include: transmitting at least said detection zone sensor data of said item detected as a fire hazard to an auxiliary memory storage device provided at a fire proof location at the processing plant and / or an auxiliary memory storage device provided at a remote location. The method may advantageously output said detection zone sensor data, said item data, said heat information data, and any other data to said auxiliary memory storage device. The method may advantageously protect data in case of a local fault of said memory storage device, or if a majority of the processing plant bums down as a result of a fire hazard starting a fire. Thus, in principle, said auxiliary memory device may serve as a black box.
[0050] According to one embodiment, wherein said at least one fire hazard condition includes: an item classification of said item and at least one fire hazard in an item classification list consisting of at least one fire hazard item is determined to match, and optionally, said item classification list includes any of the following fire hazards: batteries, such as lithium-based batteries; lighters; spray bottles; gas tanks; items comprising flammable material. Alternative, on in combination, said at least one fire hazard condition includes: at least one image of said item is determined to match with any of at least one fire hazard reference image by means of an image comparison algorithm and a matching criteria, each fire hazard reference image indicating a visual appearance of a fire hazard. Alternative, on in combination, said at least one fire hazard condition includes: at least one image of said item is determined to match with any of said at least one fire hazard by means of an image classification engine trained to detect at least one fire hazard, wherein the image classification engine is trained using labeled image data indicating at least one fire hazard and / or using unlabeled image data in combination with fire hazard detection feedback. Thus, the method may use a number of different fire hazard conditions in order to detect an item as a fire hazard. This allows for adjusting settings of one or more arrangements so as to improve detection of fire hazard for a given detection specification. By detection specification, it may mean achieving a certain detection accuracy of not exceeding a particular number of false positives and / or false negatives for a certain processing throughput.
[0051] According to one embodiment, said step of detecting a fire hazard includes: determining if said item data satisfies at least one potential fire hazard condition indicating an item as a fire hazard, and if said item is determined as a potential fire hazard, reducing said temperature threshold. Said at least one potential fire hazard condition includes at least one of the following conditions: i) an item classification of said item and at least one potential fire hazard in said item classification list is determined to match, and optionally, said item classification list includes any of the following item categories: batteries, such as lithium-based batteries; lighters; spray bottles; gas tanks; items comprising flammable material; and / or ii) at least one image of said item is determined to match with any of at least one fire hazard reference image by means of an image comparison algorithm and a matching criteria, each fire hazard reference image indicating a visual appearance of a fire hazard; iii) at least one image of said item is determined to match with any of said at least one fire hazard by means of an image classification engine trained to detect at least one fire hazard, wherein the image classification engine is trained using labeled image data indicating at least one fire hazard and / or using unlabeled image data in combination with fire hazard detection feedback. As a non-limiting example, the temperature threshold may be reduced whenever a lithium battery is detected as a potential fire hazard. The same principle applies to any other potentially fire hazardous items, so the method may apply said potential fire hazard condition, and which type, depending on the item classification, since certain items may be more easily or more difficulty detectable as potentially fire hazardous items. By adjusting the potential fire hazard condition with respect for item classification and / or said at least one feature, the method may advantageously reduce the number of false negatives and false positives in terms of fire hazard detections, thus providing a more accurate detection of fire hazards overall. As a result, safety is improved and processing efficiency is increased. Naturally, if sufficient processing power is available, all potential fire hazard conditions may be implemented and checked if satisfied for said at least one item.
[0052] According to one embodiment, said step of classifying, based on said detection zone sensor data, at least one item in the flow of matter in one of at least one item classifications includes: determining a first material property set and / or said second material property from said detection zone sensor data; comparing whether said first material property set and / or said second material property set of said item is associated with any one item classification of a list of item classifications including at least a first item classification defined at least in terms of one or more material properties of said first material property set and / or in terms of one or more material properties of said second material property set; classifying said item as an item of a particular item classification based on said comparison. Optionally, if said item cannot be classified as an item of any item classification of the list of item classifications, adaptively update the list of item classifications to include a new item classification for said item, which new item classification is defined in terms of one or more material properties of said first material property set and / or one or more material properties of said second property set of said item. The method may thus advantageously classify items as items of at least one item classification. Any one item may be classified as an item with two or more item classifications. For instance, an item may be classified as a metal item and an item having the color gray. It should be understood that items may be classified as items of many different types and variations thereof. Thus, the method as such is not limited to any particular types of items with respect to item classification. Said classification may implement or be based on any of, or any combination of the following non-exhaustive list: object detection; object recognition; instance segmentation; semantic segmentation.
[0053] According to one embodiment, the first property set is indicative of at least one of a spectral response of the matter, a material type of the matter, a color of the matter, a fluorescence of the matter, a ripeness of the matter, a dry matter content of matter, a water content of the matter, a fat content of the matter, an oil content of the matter, a calorific value of the matter, a presence of bones or fishbones of the matter, a presence of pest of the matter, a mineral type of the matter, an ore type of the matter, a defect level of the matter, a detection of hazardous biological materials of the matter, a presence of matter, a non-presence of matter, a detection of multilayer materials of the matter, a detection of fluorescent markers of the matter, a quality grade of the matter, a physical structure of the surface of the matter and molecular structure of the matter.
[0054] According to one embodiment, the second property set is indicative of at least one of a height of the matter, a height profile of the matter, a 3D map of the matter, an intensity profile of reflected and / or scattered light, a volume center of the matter, an estimated mass center of the matter, an estimated weight of the matter, an estimated material of the matter, a presence of matter, a non-presence of matter, a detection of isotropic and anisotropic light scattering of the matter, a structure and quality of wood, a surface roughness and texture of the matter and an indication of presence of fluids in the matter.
[0055] According to one embodiment, wherein said heat information data includes at least one thermal image of said item, the method comprising: providing a black body reference and / or a white body reference within a field of view of the thermal sensor system, and adjusting said thermal image and / or said indicated temperature and / or said measured temperature of said item based on the black body reference and / or the white body reference, wherein the black body reference and / or the white body reference is / are stationary within said field of view or moveably into and out of said field of view. The method may thus advantageously provide more usable thermal images for detection of fire hazards. Moreover, by providing a moveable white / black body reference, it may be ensured that said white / black body reference does not provide obstruction to flow of matter outside of a calibration phase and / or a training phase.
[0056] According to one embodiment, the method comprises: adjusting said at least one thermal image and / or said indicated temperature and / or said measured temperature of said item based on a material emission coefficient associated with a material of said item. The method may thus advantageously provide more usable thermal images for detection of fire hazards.
[0057] According to one embodiment, the method comprises: adjusting said at least one thermal image, by means of image processing, to compensate for motion blur. A disadvantage of some thermal sensor systems commercially available today is their relatively slow measurement speed, which, depending on items being processed and the transport speed provided by the transport arrangement, may result in less accurate detection of fire hazards, for instance resulting in a too high number of false positives and / or a too high number of false negatives in terms of detection of fire hazards. One option to solve this particular problem with some thermal sensor systems commercially available today is to process said thermal image by means of image processing to compensate for motion blur. By this, said at least one thermal image may advantageously be clearer and serve as more reliable data to base detection of fire hazards upon. Motion blur is caused by relative motion between a sensor arrangement, such as said classification arrangement and / or said thermal sensor system, and an item during an exposure window of said sensor arrangement. To restore an image that was degraded by motion blur, a motion path of said item must be estimated. One option is to apply motion estimation.
[0058] According to one embodiment, the method comprises a step of estimating motion of items in the flow of matter. Estimating motion of items, i.e. , motion estimation, refers to the concept of estimating at least a future position of an item to be sorted based at least on one or more current and / or pre-current position of said items, wherein pre-current position refers to a position where an item was located before being located at the current position. As a first non-limiting example, a future position may be estimated based on a current position and a motion vector. As a second non-limiting example, a future position may be estimated based on a current position and a pre-current position, from which a motion vector is estimated indicating said at least a future position at a future timepoint from said current position at a current timepoint. Motion estimation may be further enhanced by implementing other motion parameters such as acceleration (due to gravity or otherwise) and / or motion relative a transport arrangement. By determining item position by means of motion estimation, position of items may be more accurately determined, thereby allowing clearer thermal images to be provided if said motion estimation is applied to compensate for motion blur. Moreover, a future position at a future timepoint may be compared to a current position at said future timepoint. This allows for estimating an error in position due to one or more parameters used in said motion estimation. Motion estimation may be further enhanced by adjusting said one or more parameters so as to reduce said error in position.
[0059] According to one embodiment, said temperature threshold is a baseline temperature threshold, said baseline temperature for said baseline temperature threshold being one of: an ambient temperature; an average temperature of the flow of matter as a whole; an average temperature associated with said item classification. An ambient temperature may be a temperature inside a processing plant, preferably in the vicinity of said at least one detection zone. An average temperature may be a temperature based on an average (mean or median or the like) of a plurality of indicated temperatures and / or measured temperatures. It may be an average taken over number of data points provided or over a time period during which said data points were provided. Said data points for said indicated temperatures and / or said measured temperatures may correspond in time to regular time intervals. Said different temperatures may thus provide said baseline temperature (reference temperature) for the flow of matter as a whole or for items of any specific item classification. By this, an abnormal temperature of any one item may be easily detected as a deviation from said reference temperature.
[0060] According to one embodiment, said average temperature of said flow of matter as a whole or said average temperature associated with said item classification is provided by a thermal background model, said thermal background model being constructed based on at least one thermal image of the flow of matter obtained by said thermal sensor system. As a non-limiting example, said thermal background model is based on thermal images taken at regular intervals, being the average of said thermal images. As a nonlimiting example, said thermal background model may indicate a time-specific and / or a pixel specific temperature. Said pixel specific temperature may be correlated to items moving within the field of view. Said thermal background model may be adapted to provide a reference temperature for the flow of matter as a whole or for items of any specific item classification. By this, an abnormal temperature of any one item may be easily detected as a deviation from said thermal background model.
[0061] According to one embodiment, the method comprises: continuously adapting the thermal background model based on thermal images of the flow of matter obtained by said thermal sensor system. The method may advantageously improve the model over time, so as to adjust for change of time-of-day, change in weather condition, change in season, etc.
[0062] According to one embodiment, said step of classifying said at least one item in terms of item classification is performed based on any one of, or any combination of sensor data including: hyperspectral sensor data, such as VIS / NIR hyperspectral sensor data, and / or camera-based sensor, and / or laser-based sensor data, and / or X-ray-based sensor data. The method may use other types of sensor arrangements for providing said different types of sensor data or other types of sensor data not specifically mentioned herein but known by persons skilled in the art. Said sensor arrangement may include any one, or any combination of: i) a laser triangulation arrangement; ii) a time- of-flight detection arrangement; iii) a stereo vision detection arrangement; iv) structured light detection arrangement; v) sequence-of-light-pattern detection arrangement; an imaging device, such as a RGB camera; X-ray sensor arrangement; VIS and / or NIR spectroscopy system. Said spatial information of said item includes: 3D information, and / or height information, and / or footprint area, and / or position, and / or shape, and / or volume, and / or weight, and / or density, and / or relative distances to nearby items to be sorted. According to one embodiment, the classification arrangement comprises two or more different sensor system to provide any one of, or any combination of said sensor measurements, the method comprising: registering two or more sensor systems of the classification arrangement, or all sensor systems of the classification arrangement, to the same coordinate system; receiving detection zone sensor data from at least two or more sensor systems of the classification arrangement in said memory storage device, and correlating in time and space said detection zone sensor data from said at least two or more sensor systems of the classification arrangement.
[0063] According to one embodiment, the method comprises: sorting, by means of a sorting arrangement, items into at least two separate locations based on item classification. The method may advantageously actively sort items into at least two separate locations by means of the same sorting arrangement. This may allow a first item detected as a fire hazard with a low priority and a second item detected as a fire hazard with a high priority to be sorted differently by the same sorting arrangement. Thus, items detected as fire hazard with a high priority may be handled via an express transport path to a fire proof container whereas items detected as fire hazard with a low priority may be handled via a different transport path for further monitoring and or other form of handling.
[0064] According to one embodiment, said item classification and / or the detection zone sensor measurement are / is optionally provided to a memory storage device for storing said item classification and said detection zone sensor data at least temporarily
[0065] According to one embodiment, said heat information data is provided to said memory storage device for storing said heat information data at least temporarily.
[0066] According to one embodiment, the machine learning model comprises a model selected from the group comprising or consisting of: a decision tree, a random forest, a support vector machine (SVM), a k-nearest neighbors (k- NN) model, a gradient boosted tree model, a clustering model, an autoencoder, a convolutional neural network (CNN), and a recurrent neural network (RNN).
[0067] According to a third aspect of the present disclosure, a control unit for a processing plant is provided. The control unit is adapted to communicatively couple to a transport arrangement and a classification arrangement of said processing plant, said control unit configured to perform the method according to the first aspect, second aspect, or any embodiments thereof. Said control unit may comprise means such as processing means, and / or memory storage means, and / or wired and / or wireless communication means so as to be able to communicate with anyone arrangement of the processing plant and perform said method and / or any individual steps thereof.
[0068] According to a fourth aspect, a processing plant is provided. The processing plant comprises: a classification arrangement, which classification arrangement is optionally a classification and sorting arrangement; a transport arrangement configured to transport a material batch as a flow of matter to at least one detection zone of a classification arrangement; wherein the classification arrangement is configured to provide detection zone sensor data of the matter in one of the at least one detection zone; wherein the processing plant further comprises: a control unit according to said third aspect or any embodiments thereof.
[0069] According to one embodiment, an item classification list consisting of at least one potential fire hazard item is provided, and said item data includes item classification of said item, and said at least one fire hazard condition includes at least a condition that said item classification of said item and said an item classification list is determined to match; and wherein said item classification list of at least one potential fire hazard item optionally includes any of the following item categories: batteries, such as lithium-based batteries; lighters; spray bottles; gas tanks, items comprising flammable material.
[0070] According to one embodiment, the classification arrangement is adapted with an artificial neural network configured to classify items as fire hazards based on said detection zone sensor data. According to one embodiment, at least one of said at least one detection zone is provided at least partly in one of a free fall area, a conveyor belt area, or a chute area, or any combination thereof. Any of said at least one detection zone may be provided at least partly in one of a free fall area, a conveyor belt area, or a chute area.
[0071] According to one embodiment, the transportation arrangement comprises at least a first transport arrangement module. The transport arrangement may comprise a plurality of transport arrangement modules. The transport arrangement modules of the transport arrangement may be arranged so as to transport items at least partially in any of the following configurations: from a material input zone to a detection zone; from a detection zone to a sorting zone; from a sorting zone to a receiving zone; and / or from a receiving zone to a further receiving zone. One or more transport arrangement modules of the transport arrangement may include any of the following: a conveyer belt for transporting the material flow with items to be sorted; a chute arranged for transporting the material flow with items to be sorted; a free-falling segment. Said free-falling segment may provide said free falling area. Said chute may provide said chute area. Said conveyer belt may provide said conveyor belt area.
[0072] According to one embodiment, said at least a first sorting arrangement may be configured to sort items into at least a first item classification, said first item classification based on a first material property set and / or a second material property set.
[0073] According to one embodiment, the sensor arrangement comprises: a spectroscopy system including a spectrometer, wherein the spectroscopy system is adapted to receive and analyse light reflected and / or scattered by items in the detection zone. The arrangement may be configured to determine item instance segmentation based on analysis provided by the spectroscopy system.
[0074] The spectroscopy system may include near-infrared, NIR, spectroscopy system. NIR spectroscopy may advantageously enable detection of characteristics of surfaces of TLF-feed material. In the context of the application, NIR refers to near-infrared region of the electromagnetic spectrum. As a non-limiting example, near-infrared region of the electromagnetic spectrum is in the interval of 780 nm to 2500 nm. As an alternative, or in combination, such a spectroscopy system may be configured to use region of the electromagnetic spectrum outside NIR, such as the visible region of electromagnetic spectrum or medium infrared. The spectroscopy system may be configured to use both NIR spectroscopy and X- ray spectroscopy. The spectroscopy system may be a VIS / NIR spectroscopy system configured to detect visible spectrum and / or near-infrared spectrum.
[0075] The spectroscopy system may be configured to analyse light in the wavelength interval 400 - 1000 nm. The spectroscopy system may be configured to analyse light in the wavelength interval 500 - 1000 nm. The spectrometer may be configured to analyse light in the wavelength interval 1000 - 1900 nm. The spectroscopy system may be configured to analyse light having a wavelength above 900 nm. The spectroscopy system may be configured to analyse light in the wavelength interval 1900 - 2500 nm. The spectroscopy system may be configured to analyse light in the wavelength interval 2700 - 5300 nm. The spectroscopy system may be configured to analyse light in the wavelength interval 900 - 1700 nm. The spectroscopy system may be configured to analyse light in the wavelength interval 700 - 1400 nm. The spectroscopy system may analyse visible light. The spectroscopy system may analyse NIR light. The spectroscopy system may analyse IR light. Different types of spectroscopy system may be used depending on characteristics of the matter to be detected.
[0076] According to a fifth aspect, a computer program is provided. The computer program comprises instructions which, when the program is executed by a computer, cause the computer to carry out the method according to the first aspect or the second aspect, or any embodiments thereof.
[0077] According to a sixth aspect, a computer-readable storage medium is provided. The computer-readable storage medium comprises instructions which, when executed by a computer, cause the computer to carry out the method according to the first aspect or the second aspect, or any embodiments thereof.
[0078] Effects and features of the second and third and fourth and fifth and sixth aspects are largely analogous to those described above in connection with the first aspect. Embodiments mentioned in relation to the first aspect are largely compatible with the second and third and fourth and fifth and sixth aspects. It is further noted that the inventive concepts relate to all possible combinations of features unless explicitly stated otherwise.
[0079] The invention is defined by the appended independent claims, with embodiments being set forth in the appended dependent claims, in the following description and in the drawings. It is to be understood that this disclosure is not limited to the particular component parts of the device described or steps of the methods described as such device and method may vary. It is also to be understood that the terminology used herein is for purpose of describing particular embodiments only, and is not intended to be limiting. It must be noted that, as used in the specification and the appended claims, the articles "a", "an", "the", and "said" are intended to mean that there are one or more of the elements unless the context clearly dictates otherwise. Thus, for example, reference to "a unit" or "the unit" may include several devices, and the like. Furthermore, the words "comprising", "including", "containing" and similar wordings do not exclude other elements or steps.
[0080] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to “a / an / the [element, device, component, means, step, etc.]” are to be interpreted openly as referring to at least one instance of said element, device, component, means, step, etc., unless explicitly stated otherwise.
[0081] Brief description of the drawings
[0082] The aspects of the present inventive concept, including its particular features and advantages, will be readily understood from the following detailed description and the accompanying drawings. The figures are provided to illustrate the general structures of the present inventive concept.
[0083] Fig. 1 illustrates a selection of elements of a processing plant according to one embodiment of the present disclosure;
[0084] Fig. 2 illustrates a flow chart of a method according to one embodiment of the present disclosure;
[0085] Fig. 3 illustrates a flow chart of a method according to one embodiment of the present disclosure;
[0086] Figs. 4a-4b illustrate selection of elements of a processing plant according to one embodiment of the present disclosure;
[0087] Figs. 5a-5b illustrate various schematics of sorting potential fire hazards according to one method of the present disclosure;
[0088] Fig. 6 illustrates a classification arrangement according to one embodiment of the present disclosure;
[0089] Fig. 7a-7b illustrate classification arrangement according to different embodiments of the present disclosure;
[0090] Fig. 8 illustrates a classification arrangement according to one embodiment of the present disclosure.
[0091] All figures are schematic, not necessarily to scale, and generally only show parts which are necessary in order to elucidate the disclosure, wherein other parts may be omitted or merely suggested. Throughout the figures, the same reference signs designate the same, or essentially the same features.
[0092] Detailed description of the drawings
[0093] In the following description, the present inventive concept is described with reference to the accompanying drawings, in which currently preferred variants of the inventive concept are shown. This inventive concept may, however, be implemented in many different forms and should not be construed as limited to the variants set forth herein; rather, these variants are provided for thoroughness and completeness, and fully convey the scope of the present inventive concept to the skilled person. Features illustrated in the attached drawings or described in the following as part of one embodiment may be used with another embodiment to yield still a further embodiment. In the interest of clarity, not all features of an actual implementation may be described in this specification. Various structures, arrangements and systems are schematically depicted in the drawings for purposes of explanation only and so as to not obscure the description with details that are well known to those skilled in the art.
[0094] The words and phrases used herein should be understood and interpreted to have a meaning consistent with the understanding of those words and phrases by those skilled in the relevant art. No special definition of a term or phrase, i.e. , a definition that is different from the ordinary and customary meaning as understood by those skilled in the art, is intended to be implied by consistent usage of the term or phrase herein. To the extent that a term or phrase is intended to have a special meaning, i.e., a meaning other than that understood by skilled artisans, such a special definition is included herein in a definitional manner that directly and unequivocally provides the special definition for the term or phrase.
[0095] Fig. 1 illustrates a schematic side view of selected features of a processing plant 100. The processing plant 100 may be configured to receive a material batch including matter. The matter comprises items 110, which may be whole objects, worn-out objects, and / or pieces of objects. In the figures, items are represented by boxes. However, it should be understood that these boxes are merely representations of items 110 and invention(s) according to the present disclosure are applicable for items of any type. The processing plant 100 comprises a transport arrangement 101 for transporting items to an detection zone IZ. As a non-limiting example, the transport arrangement 101 comprises at least one transport arrangement module. As a non-limiting example, a transport arrangement module may include a conveyor belt, a chute or a free falling zone. The transport arrangement 101 may include a plurality of transport arrangement modules, configured to cooperate so as to transport items 110 from a material batch input to a desired destination. As a non-limiting example, a first transport arrangement module may include a first conveyor belt arrangement for transporting items through an detection zone IZ of a classification arrangement 102, and a second transport arrangement module may include a second conveyor belt arrangement arranged downstream from the first transport arrangement module, wherein the second conveyor belt arrangement is arranged for transporting items downstream the first transport arrangement module. As a non-limiting example, a chute may be arranged to transport items from the first conveyor belt arrangement to the second conveyor belt arrangement.
[0096] The processing plant 100 comprises a classification arrangement 102 configured to classify items 110 present in said detection zone based on at least one feature of said item 110. As will be exemplified in later parts of this disclosure, the classification arrangement 102 may optionally be configured to sort items based on item classification, in which case the classification arrangement may be referred to as a classification and sorting arrangement. The processing plant 100 is adapted with means to detect one or more items 110 as potential fire hazards FH. By such a configuration, a method of fire hazard plant control of a processing plant is enabled.
[0097] In reference to Fig. 1 and Fig. 2, a method 1000 of hazard plant control is detailed. The method 1000 comprises a step 1001 of transporting, by means of the transport arrangement 101 , a material batch as a flow of matter to at least one detection zone IZ of the classification arrangement 102. The processing plant may comprise a plurality of classification arrangements 102, each of which are arranged to inspect a respective detection zone through which matter is provided. The material batch is typically received as an input stream SO of matter. Matter including items leaving the detection zone IZ may be provided as an output stream S1 . The method comprises a step 1002 of providing, by means of the classification arrangement 102, detection zone sensor data of matter captured when said matter was present in one of the at least one detection zone IZ. Based on said detection zone sensor data, items may be classified into at least a first item classification. Moreover, based on said detection zone sensor data, items may be detected as a potential fire hazard. The method 1000 comprises a step 1003 of classifying, by means of the classification arrangement 102, detection zone sensor data, at least one item 110 into at least a first item classification. Item categories may be defined based on one or more parameters, such as material, size, color, condition, etc. At least one of said at least a first item classification may be based on a first material property set and / or a second material property set. The present disclosure lists various examples of the first material property set and the second material property set in the Summary of the present disclosure.
[0098] The method comprises a step 1004 of providing, by means of the classification arrangement 102, classification data comprising item classification and / or the detection zone sensor data associated with said at least one item 110. Classification data may indicate what item classification an item is classified as, or may be classified as, once said classification data is processed.
[0099] The processing plant 100 may comprise a control unit 103. The control unit 103 may be configured to receive one or more signals from one or more elements of the processing plant 100. The control unit 103 may be configured to process said one or more input signals. The control unit 103 may be configured to transmit one or more output signals. Said one or more input signals may include signals carrying information pertaining to any of the aforementioned sensor data and / or information pertaining to operational status of one or more elements of the processing plant 100. Said one or more output signals may include signals carrying information pertaining to any of the aforementioned sensor data and / or information pertaining to instructions to any of the one or more elements of the processing plant 100. For instance, the control unit 103 may receive an input signal indicating a material batch is received and in response to this input signal, transmit a first output signal to a transport arrangement 101 , which first output signal carries information pertaining to instructions of a transport speed of the transport arrangement, and transmit a second output signal to a classification arrangement 102, which second output signal carries information pertaining to instructions for the classification arrangement 102 in terms of sensor settings.
[0100] The control unit 103 may relay data, such as said item classification and / or detection zone sensor data, to a memory storage device 104 for storing said data. Said storing may be at least temporary, e.g., for at least 24 hours. However, the memory storage device 104 may be adapted in memory storage size so as to be able to store data for longer term storage. The control unit 103 may be adapted to communicatively couple to a transport arrangement and a classification arrangement of said processing plant. The control unit may be configured for carrying out one or more steps of the method 1000 of the present disclosure.
[0101] The method comprises a step 1005 of establishing, based at least on said classification data and / or said detection zone sensor data, item data of said at least one item 110.
[0102] The method comprises a step 1006 of detecting, if said item data is determined to satisfy at least one fire hazard condition, said at least one item 110 as a fire hazard FH, wherein said detection of said at least one item 110 as a fire hazard FH is based on, or assisted by, heat information data relating to: i) the same at least one item, and / or ii) items at least classifiable as one of said at least one item classification of said at least one item. The method further comprises: controlling 1008, in response to a detection of a fire hazard, the processing plant 100 to initiate at least one fire hazard response action.
[0103] In some embodiments, said assistance of detecting said at least one item 110 as a fire hazard FH includes: using said heat information data to train a neural network associated with the classification arrangement 102 to improve detection of said at least one item 110 as a fire hazard FH based on said detection zone sensor data.
[0104] Further, the method comprises a step 1007 of providing, by means of a thermal sensor system, heat information data of matter when present in one of at least one detection zone, said heat information data including heat information data of items at least classifiable as one of said at least one item classification of said at least one item; wherein, optionally, said thermal sensor system includes a microbolometer or said microbolometer.
[0105] The heat information data may such that an indicated or measured temperature associated with said flow of matter and / or said at least one detection zone may be derived, at least when said indicated or measured temperature exceeds a predetermined threshold. Said heat information data may include a thermal image which may indicate or specify temperature of the scenery within the image, in particular any items depicted. However, the heat information data may as an alternative include information derived from RGB images by means of a machine learning algorithm, such as a deep learning algorithm, to identify or measure temperature based on visually identifiable features. Said heat information data may optionally also be provided to said memory storage device 104 for storing said heat information data at least temporarily.
[0106] As a further option, the method comprises a step 1007 of providing, by means of a thermal sensor system or said thermal sensor system, heat information data of matter when present in one of at least one detection zone, said heat information data at least relating to the same at least one item, said heat information data comprising at least one of: i) an indicated temperature and / or a measured temperature associated with said at least one item and / or said at least one detection zone, ii) information from which said indicated temperature and / or measured temperature is derivable; wherein one of said at least one fire hazard condition includes said indicated temperature and / or said measured temperature exceeding a temperature threshold indicating an item as a fire hazard, wherein, optionally, said thermal sensor system includes a microbolometer.
[0107] As an example, said method may comprise a step of assessing said provided classification data, and said heat information data when present, to detect a potential fire hazard. The step of detecting a potential fire hazard may be implemented in a variety of ways.
[0108] As a first non-limiting example, said item data includes at least an indicated or measured temperature of said item, and said at least one fire hazard condition includes at least a condition that said temperature of said item exceeds a temperature threshold indicating an item as a potential fire hazard. Thus, whenever an indicated or measured temperature of an item 110 exceeds said temperature threshold, said item is detected as a potential fire hazard. As an example, a lithium battery may be provided to an detection zone and the classification arrangement inspects the lithium battery. From said inspection, it is determined that said lithium battery has an abnormal temperature, as indicated by said indicated or measured temperature exceeding said temperature threshold. Thus, once detecting an item as a potential fire hazard, the processing plant 100 may take a fire hazard response action.
[0109] As a second non-limiting example, an item classification list consisting of at least one potential fire hazard item is provided, and said item data includes item classification of said item, and said at least one fire hazard condition includes at least a condition that said item classification of said item. Said at least one fire hazard condition includes a condition that said item classification of said item and at least one potential fire hazard in said item classification list is determined to match. For instance, said item classification list may specify lithium batteries as a potential fire hazard, and when an item classification for an item is determined, said item classification is compared with the item classification list. If the item classification of said item 110 matches with an item classification listed as a potential fire hazard, then said item 110 may be detected as a potential fire hazard. Said item classification list of at least one potential fire hazard item optionally includes any of the following item categories: batteries, such as lithium-based batteries; lighters; spray bottles; gas tanks; items comprising flammable material.
[0110] As a third non-limiting example, said item data includes at least one image of said item 110, and said at least one fire hazard condition includes at least a condition that said at least one image is determined to match with any of at least one fire hazard reference image by means of an image classification engine trained to detect at least one fire hazard. The image classification engine is trained using labeled image data indicating at least one fire hazard and / or using unlabeled image data in combination with fire hazard detection feedback. For instance, one reference image may be an image of a lithium battery. Thus, the image classification engine may compare an image of an item 110 with said at least one fire hazard reference image. If said image of said item 110 is detected to match with said reference image of the lithium battery, then said item may be detected as a potential fire hazard. As a fourth non-limiting example, the classification arrangement is adapted with an artificial neural network configured to classify items as fire hazards based on said detection zone sensor data.
[0111] In addition, two or more of the four non-limiting examples may be combined for the purpose of increasing the confidence of an item detecting as a potential fire hazard in fact really is a potential fire hazard. For instance, if an item is detected as a potential fire hazard by one or more of said detection examples, said temperature threshold may be reduced. For instance, if a lithium battery is provided, then a purely image-based detection method may detect the lithium-battery as a potential fire hazard, and since we are provided with information that an item is a lithium-battery, a stricter temperature threshold may be implemented so as to identify more accurately a condition of the lithium-battery and determine if said lithium-battery is a fire risk or is not in danger of igniting.
[0112] The present disclosure also addresses how to handle items detected as a potential fire hazard. Thus, the method 1000 comprises a step 1008 of controlling, in response to a detection of a potential fire hazard FH, the processing plant 100 to initiate a fire hazard response action. Which fire hazard response action is chosen may depend on the type of potential fire hazard detected, and optionally one other parameters such as available fire hazard response action, time of day, safety aspects, etc. As a non-limiting example, one fire hazard response action may be to sort a potential fire hazard to a first container; however, should this first container be full, missing, or otherwise not able to receive said item detected as a potential fire hazard, another fire hazard response action may be chosen instead, such as monitoring the item detected as a potential fire hazard until said first container is ready to receive said item detected as a potential fire hazard. As a nonlimiting example, time of day may also impact the choice of fire hazard response action taken, for instance, if the processing plant is operating during night time, during which time most personnel of the processing plant is off- work, it may be preferably to choose fire hazard response actions which represent more strict response actions so as to improve or at least maintain a certain safety level even with the reduced number of personnel present. Said step 1006 of assessing said classification data and said heat information data when present to detect a fire hazard includes: a step 1007 of establishing, based on said classification data, and said heat information data when present, item data of said item; a step 1008 of determining if said item data or said heat information data satisfies at least one fire hazard condition; a step 1009 of detecting, if said item data or said heat information data is determined to satisfy said at least one fire hazard condition, the item 110 associated with said item data or said heat information as a potential fire hazard.
[0113] Fig. 3 illustrates a wide selection of a non-exhaustive list of possible fire hazard response action. The method 1000 is adapted so that said fire hazard response action is selected from a group of fire hazard response actions at least including: providing 2001 a fire hazard alert indicating a presence of a fire hazard; verifying 2002, by automatic and / or manual inspection of said item data, if said item detected as a fire hazard is an actual fire hazard; monitoring 2003 an item detected as a fire hazard in a subsequent detection zone downstream from the detection zone in which detection zone sensor data was provided for detecting said item as a fire hazard; sorting 2004 said item detected as a fire hazard into a transport path for receiving said item; sorting 2005 said item detected as a fire hazard into a fire proof container; determining 2006, by flame detection and / or smoke detection, whether said fire hazard is burning and / or glowing; suppressing 2007 a fire using a sprinkler system and / or a foam suppression system and / or a ventilation system and / or a gaseous cleaning agent; allocating 2008 memory storage in said memory storage device based on file size of at least said detection zone sensor data of said item detected as a fire hazard before receiving said at least said detection zone sensor data of said item; outputting 2009 at least said detection zone sensor data of said item detected as a fire hazard for review on a display device; transmitting 2010 at least said detection zone sensor data of said item detected as a fire hazard to an auxiliary memory storage device provided at a fire proof location at the processing plant and / or an auxiliary memory storage device provided at a remote location. The method may comprise a step of assessing said classification data to detect whether a fire hazard alarm is to be issued, in which case the method may comprise: determining if said item data satisfies at least one potential fire hazard condition indicating an item as a potential fire hazard, if said item data is determined to satisfy said at least one potential fire hazard condition, reducing said temperature threshold, wherein said item data includes item classification of said item, and said at least one potential fire hazard condition includes at least a condition that said item classification of said item is in a list of at least one potential fire hazard, wherein each potential fire hazard in the list of at least one potential fire hazard is identified by item classification, and / or said item data includes at least one image of said item, and said at least one potential fire hazard condition includes at least one condition wherein said at least one image is determined to match, using an image comparison algorithm and a matching criteria, with any of at least one potential fire hazard reference image, each potential fire hazard reference image indicating a visual appearance of a potential fire hazard, and / or said item data includes at least one image of said item, and said at least one potential fire hazard condition includes at least one condition wherein said at least one image is determined to match, using an image classification engine trained to detect at least one potential fire hazard, with any of said at least one potential fire hazard, wherein the image classification engine is trained using labeled image data indicating at least one potential fire hazard and / or using unlabeled image data in combination with potential fire hazard detection feedback.
[0114] As an example, said step of detecting at least one item as a fire hazard includes: determining 1010 if said item data satisfies at least one potential fire hazard condition indicating an item as a fire hazard, if said item is determined as a potential fire hazard, reducing 1011 said temperature threshold. Said at least one potential fire hazard condition includes at least one of the following conditions: i) an item classification of said item and at least one potential fire hazard in said item classification list is determined to match, and optionally, said item classification list includes any of the following item categories: batteries, such as lithium-based batteries; lighters; spray bottles; gas tanks; items comprising flammable material; and / or ii) at least one image of said item is determined to match with any of at least one fire hazard reference image by means of an image comparison algorithm and a matching criteria, each fire hazard reference image indicating a visual appearance of a fire hazard; iii) at least one image of said item is determined to match with any of said at least one fire hazard by means of an image classification engine trained to detect at least one fire hazard, wherein the image classification engine is trained using labeled image data indicating at least one fire hazard and / or using unlabeled image data in combination with fire hazard detection feedback.
[0115] In one exemplary embodiment, said step 1003 of classifying, based on said detection zone sensor data, at least one item in the flow of matter in one of at least one item classifications includes: determining 1012 a first material property set and / or said second material property from said detection zone sensor data; comparing 1013 whether said first material property set and / or said second material property set of said item is associated with any one item classification of a list of item classifications including at least a first item classification defined at least in terms of one or more material properties of said first material property set and / or in terms of one or more material properties of said second material property set; classifying 1014 said item as an item of a particular item classification based on said comparison; optionally, if said item cannot be classified as an item of any item classification of the list of item classifications, adaptively updating 1015 the list of item classifications to include a new item classification for said item, which new item classification is defined in terms of one or more material properties of said first material property set and / or one or more material properties of said second property set of said item.
[0116] As an option, said heat information data includes at least one thermal image of said item, the method comprising: providing 1016 a black body reference and / or a white body reference within a field of view of the thermal sensor system; adjusting 1017 said thermal image and / or said indicated temperature and / or said measured temperature of said item based on the black body reference and / or the white body reference, wherein the black body reference and / or the white body reference is / are stationary within said field of view or moveably into and out of said field of view.
[0117] The method may comprises: adjusting 1018 said at least one thermal image and / or said indicated temperature and / or said measured temperature of said item based on a material emission coefficient associated with a material of said item.
[0118] The method may comprise: adjusting 1019 said at least one thermal image, by means of image processing, to compensate for motion blur.
[0119] As an option, said temperature threshold is a baseline temperature threshold, said baseline temperature for said baseline temperature threshold being one of: an ambient temperature; an average temperature of the flow of matter as a whole; an average temperature associated with said item classification. Optionally, said average temperature of said flow of matter as a whole or said average temperature associated with said item classification is provided by a thermal background model, said thermal background model being constructed based on at least one thermal image of the flow of matter obtained by said thermal sensor system. Optionally, the method further comprising: continuously adapting (1020) the thermal background model based on thermal images of the flow of matter obtained by said thermal sensor system.
[0120] In one exemplary embodiment, said step of classifying 103 said at least one item in terms of item classification is performed based on any one of, or any combination of sensor measurements includes: hyperspectral sensor data, such as VIS and / or NIR hyperspectral sensor data, and / or camera-based sensor, and / or laser-based sensor data, and / or X-ray-based sensor data. Optionally, the classification arrangement comprises two or more different sensor system to provide any one of, or any combination of said sensor measurements, the method comprises: registering 1021 two or more sensor systems of the classification arrangement, or all sensor systems of the classification arrangement, to the same coordinate system; receiving 1022 detection zone sensor data from at least two or more sensor systems of the classification arrangement in said memory storage device, and correlating 1023 in time and space said detection zone sensor data from said at least two or more sensor systems of the classification arrangement.
[0121] Said list of at least one fire hazard or said list of at least one potential fire hazards includes any of the following item categories: batteries, such as lithium-based batteries; lighters; spray bottles; gas tanks, items comprising flammable material.
[0122] Said step of classifying, based on said detection zone sensor data, said at least one item in terms of item classification comprises: determining a first material property set and / or said second material property from said detection zone sensor data; comparing whether said first material property set and / or said second material property set of said item corresponds to any one item classification of a list of item categories including at least a first item classification defined at least in terms of one or more material properties of said first material property set and / or in terms of one or more material properties of said second material property set; classifying said item as an item of a particular item classification based on said comparison; optionally, if said item cannot be classified as an item of any item classification of the list of item categories, adaptively update the list of item categories to include a new item classification for said item, which new item classification is defined in terms of one or more material properties of said first material property set and / or one or more material properties of said second property set of said item.
[0123] As an option, said classification arrangement 102 includes a thermal sensor system configured to provide thermal images of matter in one of the at least one detection zone, wherein said step of determining, based on said heat information data, item data of said item includes: providing, by means of said thermal sensor system, heat information data including at least one thermal image of said item; based on said at least one thermal image, determining said an estimated temperature of said item; wherein, optionally, said thermal sensor system is a microbolometer sensor system, wherein, optionally, said thermal sensor system is configured to provide thermal images of matter in one of the at least one detection zone which overlaps with or is at least partially offset or completely separated from one other detection zone of said at least one detection zone associated with a different sensor system for providing detection zone sensor data different from said thermal images.
[0124] Further, the method may comprise: a step of sorting, by means of a sorting arrangement, items into at least two separate locations based on item classification. The sorting arrangement may be a claw, a robot arm, an ejection sorter, i.e. , a sorting arrangement configured to sort items using a flow of fluid media (gaseous and / or fluid). This is exemplified in Figs. 4a wherein items detected as a fire hazard FH are ejected into a second stream S2 whereas items not sorted pass on into stream S1 . The inputs of the two streams are separated by a splitter 104. The splitter may be adjustable in terms of position and / or orientation (not shown). However, adjusting the splitter may facilitate sorting of certain items which are too heavy to be ejection sorted when the splitter is in some positions. Fig. 4b illustrate a sorting arrangement, in particular two ejection sorters 103, 105 which are arranged to ejection sort items into different streams. The input of said streams are separated by splitters 104, 106 respectively.
[0125] Fig. 5a illustrates a classification arrangement 102 which is adapted with means to also sort items based on item classification. When detecting an item as a fire hazard, the classification and sorting arrangement 102 sorts fire hazards into a first stream S1 transported towards a fire hazard receiving zone FHZ1 whereas all other items not detected as fire hazards are sorted into a second stream S2 transported towards a receiving zone RZ. A shown in Fig. 5b, a second classification arrangement 102 may be arranged to receive the stream S1 of items comprising fire hazards. The second classification arrangement 102’ may inspect the fire hazards received and sort into different streams ST, S1” depending on how severe of a risk the potential fire hazards represents which may be characterized in terms of fire hazard priority as discussed in the present disclosure.
[0126] Fig. 6 illustrates an example embodiment of a sensor arrangement 12 according to the present disclosure. The classification arrangement 102 may incorporate said sensor arrangement 12. The sensor arrangement 12 may generally be based on the apparatus as disclosed in WO23104834 A1 which is herein incorporated in its entirety. As a non-limiting example, the sensor arrangement 12 is configured to: detect items using item detection and / or semantic segmentation. Alternatively, or in combination, the sensor arrangement 12 is configured to: use instance segmentation to add at least a boundary indicating a shape of a detected item. Moreover, according to one embodiment, the sensor arrangement 12 is configured to: determine, by means of instance segmentation, at least a boundary of an item to be sorted; based on said at least boundary and item classification of said item to be sorted, determine an item weight of the item to be sorted and optionally a center of gravity of the item to be sorted; wherein the ejection estimation is established based on item weight and optionally center of gravity of the item to be sorted. In addition, according to one exemplary embodiment, the sensor arrangement 12 is further configured to: based on said at least one feature detected of said item, classify said item as an item of said item classification of said at least one item classification. Said at least one feature of said item may include any one of, or any combination of: at least one visual feature, material information, material density, material shape, material weight, color, condition (whole, damaged, clean, etc.), consistence. The sensor arrangement 12 may comprise: an optical sensor arrangement configured to receive and analyze light reflected and / or scattered by items in a detection zone through which the items are provided, wherein the arrangement is configured to determine item positions of said items and / or material information and / or item classification of said items based on an analysis provided by the optical sensor arrangement. The sensor arrangement 12 may include any one, or any combination of: i) a laser triangulation arrangement 12a, 12c; ii) a time-of-flight detection arrangement; iii) a stereo vision detection arrangement 12b; iv) structured light detection arrangement; v) sequence-of-light-pattern detection arrangement; an imaging device, such as a RGB camera; X-ray sensor arrangement; VIS and / or NIR spectroscopy system 12d. Said spatial information of said item includes: 3D information, and / or height information, and / or footprint area, and / or position, and / or shape, and / or volume, and / or weight, and / or density, and / or relative distances to nearby items to be sorted.
[0127] The classification arrangement includes at least a first sensor arrangement. The sensor arrangement may include a thermal sensor system, as shown in Fig. 7a-7b. As a non-limiting example, the thermal sensor system is a microbolometer sensor system. A sensor head 12e of the thermal sensor system, i.e. , the thermal sensor head, may be arranged at one of a plurality of positions. As a non-limiting example, the thermal sensor head 12e is arranged to inspect an area which is at least partially upstream of the detection zone as shown in Fig. 7b. As a non-limiting example, the thermal sensor head 12e is arranged to inspect an area which is at least partially downstream of the detection zone as shown in Fig. 7a. As a non-limiting example, the thermal sensor head is arranged to substantially inspect an area overlapping the detection zone.
[0128] Fig. 8 illustrates a perspective schematic view of an arrangement 700 according to one embodiment of the present disclosure. The arrangement 700 is fed with a material batch including items 710. The items 710 is conveyed through a detection zone 720. However, the items may be provided through the detection zone by any suitable means or manually without any technical means. A light source arrangement 730 and a NIR spectroscopy system 222 are provided. The NIR spectroscopy system is adapted to receive and analyze 732, from the light source arrangement 730, which is reflected and / or scattered by the pieces light source arrangement 730, which is reflected and / or scattered by the items in the detection zone 720. Hence, the NIR spectroscopy system 222 typically acquires a spectrum of the Items from the material batch stream that is conveyed through the detection zone 720. The NIR spectroscopy system 222 of the arrangement 700 is configured to discriminate items from other material and / or items based on the acquired spectrum. In other words, the NIR system 222 is typically set up such that a specific type of items is discriminated form other types of the items owing from its spectrum. The arrangement 700 may further comprise a spectroscopy system 760, such as a NIR spectroscopy system, configured to acquire a spectrum of the Items originating from the material batch stream that is conveyed through the detection zone 720. The ejection unit 224 of the arrangement 700 may be further configured to divert said Items originating from the material batch stream based on the acquired spectrum thereby sorting the Items based on color. The spectroscopy system 760 may through the acquired spectrum determine the different colors of the Items that is conveyed through the detection zone 720. An advantage of determining the colors of the items is that it may be sorted into different fractions.
[0129] The arrangement 700 may further comprise a laser triangulation system 740 configured to determine height information of Items that is conveyed through the detection zone 720. The ejection unit 224 of said at least one arrangement may be further configured to divert said items based on the determined height information. The laser triangulation system 740 is typically configured to emit a line of laser light 742 towards the detection zone 720. The depicted laser triangulation system 740 includes a camera-based sensor arrangement 744 configured to receive and analyze light 746 which is reflected and / or scattered by the Items in the detection zone 720. By means of the laser triangulation system 740 the arrangement 700 may be able to detect Items that the NIR spectroscopy system 222 have difficulties to detect. By detecting height differences on a conveyor belt used to convey the material batch being sorted, the arrangement may combine such height information with the acquired information from the NIR spectroscopy system 222 to determine if there is any items that is hard to detect on the conveyor belt. Accordingly, further items may be recycled.
[0130] The arrangement 700 may further comprise a camera 750 configured to acquire images of items originating from the material batch stream that is conveyed through the detection zone 720. The arrangement 700 may comprise an artificial neural network in combination with the camera 750. Such artificial neural network may be configured to detect different characteristics of items that is conveyed through the detection zone 720 based on images acquired by the camera 750. The ejection unit 224 of the arrangement 700 may in this case be further configured to divert Items that is conveyed through the detection zone 720 based on the detected characteristics of Items. In other words, characteristics of Items may thus be determined by the artificial neural network from the, by the camera, acquired images. The characteristics may be a shape, a color, features at the surface or anything in the visual appearance of the Items that may be determined and classified by the artificial neural network. The camera 750 may provide the possibility to further sort Items into different fractions. With the help of the artificial neural network it may be possible to sort Items of the same material composition into different fractions depending on quality and origin.
[0131] In a first non-limiting example, the processing plant is configured to process general waste, which may include a mixture of items such as plastics, metals, organic matter, and electronic waste, including batteries (such as lithium-ion batteries). The items are transported via at least one conveyor belt to one or more detection zones. When items are in each detection zone, an RGB camera captures detection zone sensor data comprising two-dimensional images of each item. A classification arrangement is communicatively connected to each RGB camera and processes the detection zone sensor data to classify individual items into one of several predefined item classifications, including: “non-electronic waste”, “metallic item”, “undamaged battery”, and “damaged battery”.
[0132] To determine whether any classified item is a fire hazard, the classification arrangement uses a machine learning model, specifically a neural network, trained at least on heat information data. The heat information data indicates a surface temperature for each item. The heat information data is obtained using a thermal imaging sensor, such as a microbolometer, positioned to monitor items when present in one of the detection zones.
[0133] The neural network used for fire hazard detection in the processing plant comprises an input layer, at least one hidden layer, and an output layer.
[0134] The input layer is configured to receive a feature vector representing each item. The feature vector comprises: i) a set of numerical features extracted from RGB images capturing the item’s visual characteristics (e.g., dimensions, color characteristics, surface texture); ii) a numerical identifier representing the item type (for example, 2 for “damaged battery”, 1 for “undamaged batteries” and 0 for non-battery items); and iii) optionally, a surface temperature value obtained from the heat information data for that item. In a first non-limiting embodiment, the vector omits the surface temperature value (i.e. , heat information data is only used for fire hazard detection during training). In a second non-limiting embodiment, the vector comprises the surface temperature value (i.e., heat information data is also used for fire hazard detection after training).
[0135] The at least one hidden layer is configured to receive data from the input layer, which has processed the feature vector. Each hidden layer comprises a plurality of neurons. Each neuron receives inputs from one or more neurons in a prior layer, with each connection associated with a respective weight. These weights determine how much influence each input has on a respective neuron's output. Each neuron computes a weighted sum of its inputs, applies an activation function (such as Rectified Linear Unit, RLU), and transmits the result to one or more neurons in the next layer. In this way, the at least one hidden layer analyzes inputs based on the feature vector to detect patterns indicative of fire hazards, such as visual signs of battery damage supported by elevated temperatures. Multiple hidden layers may be employed to capture both low-level visual features and high-level contextual relationships, thereby improving classification accuracy.
[0136] The output layer processes results from the at least one hidden layer to produce a single output value between 0 and 1 , which may represent a probability that an analyzed item is a fire hazard. A value exceeding a predetermined threshold, such as 0.5, classifies the item as a fire hazard, while a lower value classifies it as safe.
[0137] The neural network is trained using a training method. The training method involves using a labeled dataset comprising a plurality of training samples. Each training sample comprises: i) at least one RGB image of an item; ii) a surface temperature for the item (e.g., obtained from heat information data for the item); iii) a classification of the item (e.g., “undamaged battery” or “damaged battery”), and iv) a ground-truth label indicating whether the item is an actual fire hazard or not. Each training sample is encoded as a feature vector and passed through the network during training. Training continues until a prediction error is reduced, preferably minimized, across the labeled dataset. In other words, the neural network is for example trained on the heat information data to associate undamaged batteries with no elevated surface temperature and damaged batteries with an elevated temperature. As a result, the classification arrangement may detect damaged batteries as fire hazards and undamaged batteries as no fire hazards.
[0138] For example, consider a training sample where:
[0139] - the item is a damaged lithium-ion battery;
[0140] - an RGB image indicates the battery has a deformed shape;
[0141] - the heat information data indicates an elevated temperature (e.g., 85°C);
[0142] - the item is labeled in the training set as a confirmed fire hazard (label = 1 ).
[0143] This input vector is processed by the network through the input layer and one or more hidden layers. Initially, the weights associated with each connection between neurons in adjacent layers may be random. As the input propagates through the neural network, the output layer may produce a value such as 0.3, indicating the neural network currently predicts a low probability that this item is a fire hazard. However, as the ground-truth label is 1 , the actual item is indeed a fire hazard. The neural network calculates an error, for example, using binary cross-entropy, which in this case will be relatively large because the prediction (0.3) is far from the correct label (1 ).
[0144] To reduce this error, the network performs backpropagation: the error is propagated backward through the network, and each weight is adjusted slightly in a direction that reduces the error on the next pass. The adjustment is computed using a gradient of the error function with respect to each weight, which informs the neural network how much a small change in that weight will affect the final output of the neural network.
[0145] For instance, suppose a neuron in a hidden layer is highly activated when detecting round shapes and shiny surfaces (common for batteries), and its output is strongly weighted toward a neuron that contributes to the “not a fire hazard” prediction. If the item is in fact a fire hazard, then during backpropagation, the weight between these two neurons is reduced to lower the influence of that misleading signal. Conversely, a neuron that responds to high surface temperature values or visual signs of deformation might have its weight increased, strengthening its contribution to the fire hazard prediction.
[0146] As a result of this iterative adjustment of weights, the neural network “learns” to associate certain combinations of visual and heat-related features with a higher probability of fire hazard. For example, it will eventually predict a value closer to 0.9 or 1 .0 for damaged, hot batteries, and values near 0.0 for intact, cool ones.
[0147] If the classification arrangement determines that a damaged battery is a fire hazard, for example due to shape deformation, a fire hazard response is automatically triggered. This may include stopping the transport arrangement, diverting the item to a safe container, or activating a suppression mechanism. This enables early, automated identification of battery-related fire hazards in heterogeneous waste environments, improving safety and operational continuity.
[0148] In a second non-limiting example, the processing plant is configured as in the first non-limiting example, i.e. , to process general waste, which may include a mixture of items such as plastics, metals, organic matter, and electronic waste, including batteries (such as lithium-ion batteries). In this example, a fire hazard detection method of the present disclosure comprises: determining if item data satisfies at least one potential fire hazard condition indicating an item as a fire hazard, and if said item is determined as a potential fire hazard, reducing said temperature threshold. For instance, the temperature threshold may be reduced whenever a lithium battery is detected as a potential fire hazard. By reducing the temperature threshold, a more responsive handling of fire hazards is enabled.
[0149] The at least one potential fire hazard condition includes at least one of a plurality of conditions.
[0150] In a first condition, the classification arrangement of the processing plant is configured to determine whether an item classification of an item belongs to a predefined fire hazard classification list, which includes item types known to be associated with fire risks, such as lithium-based batteries, lighters, spray bottles, gas tanks, or items comprising flammable materials. If the item is classified, for example, as a “damaged battery”, and said classification is included in the fire hazard classification list, then the item classification will indeed be determined to match an item classification included in the item classification list, thereby satisfying the first potential fire hazard condition. For instance, the item classification may be provided as a first string and each item classification in the item classification list may be also provided as strings. Thereby, a match may be determined based on string comparison.
[0151] In a second condition, the captured RGB image of the item is compared to one or more potential fire hazard reference images, which visually represent known hazardous states or potentially hazardous states of items (e.g., deformation, leakage, bum marks). An image comparison algorithm, such as one based on structural similarity index (SSIM), is used to determine whether the item image matches any of the reference images according to a predefined matching criteria. As a non-limiting example, the predefined matching criteria is an SSIM score between the item image and a reference image exceeds a threshold value, such as 0.85. If a match is detected above the criteria threshold, the second potential fire hazard condition is satisfied.
[0152] In a third condition, an image classification engine, such as a neural network, trained to recognize visual indications of potential fire hazards processes the image data of the item, which may be RGB images. The neural network may be trained as in the first non-limiting example. Although the above examples illustrate the present disclosure using batteries, RGB cameras, neural networks, and surface temperature measurements, it should be understood that the scope of the present disclosure is not limited to these specific features. Variations and modifications may be implemented using other item types, sensor technologies, classification models, or input data, without departing from the underlying principles of the disclosure.
[0153] While the foregoing is directed to embodiments of the disclosure, other and further embodiments may be devised without parting from the inventive concept discussed herein.
[0154] ITEMIZED LIST OF EMBODIMENTS Method of fire hazard plant control of a processing plant (100) provided with a transporting arrangement (101 ) for transporting items (110) to a detection zone (IZ) and a classification arrangement (102), said classification arrangement (102) configured to classify items (110) present in said detection zone (IZ) based on at least one feature of said items and optionally to sort items (110) based on item classification, the method comprising:
[0155] - transporting (1001 ), by means of the transport arrangement (101 ), a material batch as a flow of matter to at least one detection zone (IZ) of the classification arrangement (102),
[0156] - providing (1002), by means of the classification arrangement (102), detection zone sensor data of matter captured when said matter was present in one of the at least one detection zone (IZ);
[0157] - classifying (1003), based on said detection zone sensor data, at least one item (110) in the flow of matter as an item (110) of at least one item classification;
[0158] - providing (1004) classification data comprising item classification of said at least one item (110) or information from which item classification for said at least one item is derivable;
[0159] - establishing (1005), based at least on said classification data and / or said detection zone sensor data, item data of said at least one item (110);
[0160] - detecting (1006), if said item data is determined to satisfy at least one fire hazard condition, said at least one item (110) as a fire hazard (FH), wherein said detection of said at least one item (110) as a fire hazard (FH) is based on, or assisted by, heat information data relating to: i) the same at least one item, and / or ii) items at least classifiable as one of said at least one item classification of said at least one item; the method further comprising:
[0161] - controlling (1008), in response to a detection of a fire hazard, the processing plant (100) to initiate at least one fire hazard response action. Method according to item 1 , wherein said assistance of detecting said at least one item (110) as a fire hazard (FH) includes:
[0162] - using said heat information data to train a neural network associated with the classification arrangement (102) to improve detection of said at least one item (110) as a fire hazard (FH) based on said detection zone sensor data. Method according to item 2, further comprising:
[0163] - providing (1007), by means of a thermal sensor system, heat information data of matter when present in one of at least one detection zone, said heat information data including heat information data of items at least classifiable as one of said at least one item classification of said at least one item;
[0164] - wherein, optionally, said thermal sensor system includes a microbolometer or said microbolometer. Method according to any of items 1 -3, further comprising:
[0165] - providing (1007), by means of a thermal sensor system or said thermal sensor system, heat information data of matter when present in one of at least one detection zone, said heat information data at least relating to the same at least one item, said heat information data comprising at least one of: o an indicated temperature and / or a measured temperature associated with said at least one item and / or said at least one detection zone, o information from which said indicated temperature and / or measured temperature is derivable; wherein one of said at least one fire hazard condition includes said indicated temperature and / or said measured temperature exceeding a temperature threshold indicating an item as a fire hazard, wherein, optionally, said thermal sensor system includes a microbolometer. Method according to any of items 1-4, further comprising:
[0166] - storing (1009) said item data and / or said classification data and / or said detection zone sensor data and / or said heat information data on a memory storage device if said item data satisfies said at least one fire hazard condition. Method according to any of the preceding items, wherein said fire hazard response action is selected from a group of fire hazard response actions at least including:
[0167] - providing (2001 ) a fire hazard alert indicating a presence of a fire hazard;
[0168] - verifying (2002), by automatic and / or manual inspection of said item data, if said item detected as a fire hazard is an actual fire hazard;
[0169] - monitoring (2003) an item detected as a fire hazard in a subsequent detection zone downstream from the detection zone in which detection zone sensor data was provided for detecting said item as a fire hazard;
[0170] - sorting (2004) said item detected as a fire hazard into a transport path for receiving said item;
[0171] - sorting (2005) said item detected as a fire hazard into a fire proof container;
[0172] - determining (2006), by flame detection and / or smoke detection, whether said fire hazard is burning and / or glowing;
[0173] - suppressing (2007) a fire using a sprinkler system and / or a foam suppression system and / or a ventilation system and / or a gaseous cleaning agent; - allocating (2008) memory storage in said memory storage device based on file size of at least said detection zone sensor data of said item detected as a fire hazard before receiving said at least said detection zone sensor data of said item;
[0174] - outputting (2009) at least said detection zone sensor data of said item detected as a fire hazard for review on a display device;
[0175] - transmitting (2010) at least said detection zone sensor data of said item detected as a fire hazard to an auxiliary memory storage device provided at a fire proof location at the processing plant and / or an auxiliary memory storage device provided at a remote location. Method according to any of the preceding items, wherein said at least one fire hazard condition includes:
[0176] - an item classification of said item and at least one fire hazard in an item classification list consisting of at least one fire hazard item is determined to match, and optionally, said item classification list includes any of the following fire hazards: batteries, such as lithium- based batteries; lighters; spray bottles; gas tanks; items comprising flammable material; and / or
[0177] - at least one image of said item is determined to match with any of at least one fire hazard reference image by means of an image comparison algorithm and a matching criteria, each fire hazard reference image indicating a visual appearance of a fire hazard; and / or
[0178] - at least one image of said item is determined to match with any of said at least one fire hazard by means of an image classification engine trained to detect at least one fire hazard, wherein the image classification engine is trained using labeled image data indicating at least one fire hazard and / or using unlabeled image data in combination with fire hazard detection feedback. Method according to any of the preceding items in combination with item 4, wherein said step of detecting at least one item as a fire hazard includes:
[0179] - determining (1010) if said item data satisfies at least one potential fire hazard condition indicating an item as a fire hazard,
[0180] - if said item is determined as a potential fire hazard, reducing (1011) said temperature threshold, wherein said at least one potential fire hazard condition includes at least one of the following conditions:
[0181] - an item classification of said item and at least one potential fire hazard in said item classification list is determined to match, and optionally, said item classification list includes any of the following item categories: batteries, such as lithium-based batteries; lighters; spray bottles; gas tanks; items comprising flammable material; and / or
[0182] - at least one image of said item is determined to match with any of at least one fire hazard reference image by means of an image comparison algorithm and a matching criteria, each fire hazard reference image indicating a visual appearance of a fire hazard;
[0183] - at least one image of said item is determined to match with any of said at least one fire hazard by means of an image classification engine trained to detect at least one fire hazard, wherein the image classification engine is trained using labeled image data indicating at least one fire hazard and / or using unlabeled image data in combination with fire hazard detection feedback. Method according to any of the preceding items, wherein said step of classifying (1003), based on said detection zone sensor data, at least one item in the flow of matter in one of at least one item classifications includes: determining (1012) a first material property set and / or said second material property from said detection zone sensor data; - comparing (1013) whether said first material property set and / or said second material property set of said item is associated with any one item classification of a list of item classifications including at least a first item classification defined at least in terms of one or more material properties of said first material property set and / or in terms of one or more material properties of said second material property set;
[0184] - classifying (1014) said item as an item of a particular item classification based on said comparison;
[0185] - optionally, if said item cannot be classified as an item of any item classification of the list of item classifications, adaptively updating (1015) the list of item classifications to include a new item classification for said item, which new item classification is defined in terms of one or more material properties of said first material property set and / or one or more material properties of said second property set of said item. . Method according to any preceding items, wherein
[0186] - the first property set is indicative of at least one of a spectral response of the matter, a material type of the matter, a colour of the matter, a fluorescence of the matter, a ripeness of the matter, a dry matter content of matter, a water content of the matter, a fat content of the matter, an oil content of the matter, a calorific value of the matter, a presence of bones or fishbones of the matter, a presence of pest of the matter, a mineral type of the matter, an ore type of the matter, a defect level of the matter, a detection of hazardous biological materials of the matter, a presence of matter, a nonpresence of matter, a detection of multilayer materials of the matter, a detection of fluorescent markers of the matter, a quality grade of the matter, a physical structure of the surface of the matter and molecular structure of the matter, and / or - the second property set is indicative of at least one of a height of the matter, a height profile of the matter, a 3D map of the matter, an intensity profile of reflected and / or scattered light, a volume centre of the matter, an estimated mass centre of the matter, an estimated weight of the matter, an estimated material of the matter, a presence of matter, a non-presence of matter, a detection of isotropic and anisotropic light scattering of the matter, a structure and quality of wood, a surface roughness and texture of the matter and an indication of presence of fluids in the matter. . Method according to any preceding items, wherein said heat information data includes at least one thermal image of said item, the method comprising:
[0187] - providing (1016) a black body reference and / or a white body reference within a field of view of the thermal sensor system;
[0188] - adjusting (1017) said thermal image and / or said indicated temperature and / or said measured temperature of said item based on the black body reference and / or the white body reference, wherein the black body reference and / or the white body reference is / are stationary within said field of view or moveably into and out of said field of view, and / or the method comprising:
[0189] - adjusting (1018) said at least one thermal image and / or said indicated temperature and / or said measured temperature of said item based on a material emission coefficient associated with a material of said item, and / or the method comprising:
[0190] - adjusting (1019) said at least one thermal image, by means of image processing, to compensate for motion blur. . Method according to any of the preceding items in combination with item 4, wherein said temperature threshold is a baseline temperature threshold, said baseline temperature for said baseline temperature threshold being one of:
[0191] - an ambient temperature;
[0192] - an average temperature of the flow of matter as a whole;
[0193] - an average temperature associated with said item classification wherein, optionally, said average temperature of said flow of matter as a whole or said average temperature associated with said item classification is provided by a thermal background model, said thermal background model being constructed based on at least one thermal image of the flow of matter obtained by said thermal sensor system, wherein, optionally, the method further comprising:
[0194] - continuously adapting (1020) the thermal background model based on thermal images of the flow of matter obtained by said thermal sensor system. . Method according to any preceding items, wherein said step of classifying (103) said at least one item in terms of item classification is performed based on any one of, or any combination of sensor measurements including:
[0195] - hyperspectral sensor data, such as VIS and / or NIR hyperspectral sensor data, and / or
[0196] - camera-based sensor, and / or
[0197] - laser-based sensor data, and / or
[0198] - X-ray-based sensor data, wherein, optionally, the classification arrangement comprises two or more different sensor system to provide any one of, or any combination of said sensor measurements, the method comprising:
[0199] - registering (1021 ) two or more sensor systems of the classification arrangement, or all sensor systems of the classification arrangement, to the same coordinate system; - receiving (1022) detection zone sensor data from at least two or more sensor systems of the classification arrangement in said memory storage device, and
[0200] - correlating (1023) in time and space said detection zone sensor data from said at least two or more sensor systems of the classification arrangement. . Control unit (103) for a processing plant (100), adapted to communicatively couple to a transport arrangement (101 ) and a classification arrangement (102) of said processing plant (100), said control unit (103) configured to perform the method (1000) according to any preceding items. . Processing plant (100), comprising:
[0201] - a classification arrangement (102), which classification arrangement is optionally a classification and sorting arrangement;
[0202] - a transport arrangement (101 ) configured to transport a material batch as a flow of matter to at least one detection zone (IZ) of the classification arrangement (102); wherein the classification arrangement (102) is configured to provide detection zone sensor data of the matter in one of the at least one detection zone (IZ); wherein the processing plant (100) further comprises:
[0203] - a control unit (103) according to item 14.
Claims
CLAIMS1 . Method of fire hazard plant control of a processing plant (100) provided with a transporting arrangement (101 ) for transporting items (110) to a detection zone (IZ) and a classification arrangement (102), said classification arrangement (102) configured to classify items (110) present in said detection zone (IZ) based on at least one feature of said items and optionally to sort items (110) based on item classification, the method comprising:- transporting (1001 ), by means of the transport arrangement (101 ), a material batch as a flow of matter to at least one detection zone (IZ) of the classification arrangement (102),- providing (1002), by means of the classification arrangement (102), detection zone sensor data of matter captured when said matter was present in one of the at least one detection zone (IZ);- classifying (1003), based on said detection zone sensor data, at least one item (110) in the flow of matter as an item (110) of at least one item classification;- providing (1004) classification data comprising item classification of said at least one item (110) or information from which item classification for said at least one item is derivable;- establishing (1005), based at least on said classification data and / or said detection zone sensor data, item data of said at least one item (110);- detecting (1006), if said item data is determined to satisfy at least one fire hazard condition, said at least one item (110) as a fire hazard (FH), wherein said detection of said at least one item (110) as a fire hazard (FH) is based on, or assisted by, heat information data relating to: i) the same at least one item, and / or ii) items at least classifiable as one of said at least one item classification of said at least one item;the method further comprising:- controlling (1008), in response to a detection of a fire hazard, the processing plant (100) to initiate at least one fire hazard response action, wherein said assistance of detecting said at least one item (110) as a fire hazard (FH) includes:- using a machine learning model associated with the classification arrangement (102) to detect said at least one item (110) as a fire hazard (FH) based on said detection zone sensor data or a combination of said detection zone sensor data and said heat information data, wherein said machine learning model is trained at least on using said heat information data.
2. Method according to claim 1 , wherein said machine learning model comprises a neural network.
3. Method according to any one of claims 1-2, further comprising:- providing (1007), by means of a thermal sensor system, heat information data of matter when present in one of at least one detection zone, said heat information data including heat information data of items at least classifiable as one of said at least one item classification of said at least one item.
4. Method according to any of claims 1-3, further comprising:- providing (1007), by means of a thermal sensor system or said thermal sensor system, heat information data of matter when present in one of at least one detection zone, said heat information data at least relating to the same at least one item, said heat information data comprising at least one of: o an indicated temperature and / or a measured temperature associated with said at least one item and / or said at least one detection zone,o information from which said indicated temperature and / or measured temperature is derivable; wherein one of said at least one fire hazard condition includes said indicated temperature and / or said measured temperature exceeding a temperature threshold indicating an item as a fire hazard.
5. Method according to any one of claims 3-4, wherein said thermal sensor system includes a microbolometer.
6. Method according to any of claims 1-5, further comprising:- storing (1009) said item data and / or said classification data and / or said detection zone sensor data and / or said heat information data on a memory storage device if said item data satisfies said at least one fire hazard condition.
7. Method according to any of the preceding claims, wherein said fire hazard response action is selected from a group of fire hazard response actions comprising or consisting of:- providing (2001 ) a fire hazard alert indicating a presence of a fire hazard;- verifying (2002), by automatic and / or manual inspection of said item data, if said item detected as a fire hazard is an actual fire hazard;- monitoring (2003) an item detected as a fire hazard in a subsequent detection zone downstream from the detection zone in which detection zone sensor data was provided for detecting said item as a fire hazard;- sorting (2004) said item detected as a fire hazard into a transport path for receiving said item;- sorting (2005) said item detected as a fire hazard into a fire proof container;- determining (2006), by flame detection and / or smoke detection, whether said fire hazard is burning and / or glowing;- suppressing (2007) a fire using a sprinkler system and / or a foam suppression system and / or a ventilation system and / or a gaseous cleaning agent;- allocating (2008) memory storage in said memory storage device based on file size of at least said detection zone sensor data of said item detected as a fire hazard before receiving said at least said detection zone sensor data of said item;- outputting (2009) at least said detection zone sensor data of said item detected as a fire hazard for review on a display device;- transmitting (2010) at least said detection zone sensor data of said item detected as a fire hazard to an auxiliary memory storage device provided at a fire proof location at the processing plant and / or an auxiliary memory storage device provided at a remote location.
8. Method according to any of the preceding claims, wherein said at least one fire hazard condition includes:- an item classification of said item and at least one fire hazard in an item classification list consisting of at least one fire hazard item is determined to match, and optionally, said item classification list includes any of the following fire hazards: batteries, such as lithium- based batteries; lighters; spray bottles; gas tanks; items comprising flammable material; and / or- at least one image of said item is determined to match with any of at least one fire hazard reference image by means of an image comparison algorithm and a matching criteria, each fire hazard reference image indicating a visual appearance of a fire hazard; and / or- at least one image of said item is determined to match with any of said at least one fire hazard by means of an image classification engine trained to detect at least one fire hazard, wherein the image classification engine is trained using labeled image data indicatingat least one fire hazard and / or using unlabeled image data in combination with fire hazard detection feedback.
9. Method according to any of the preceding claims in combination with claim 4, wherein said step of detecting at least one item as a fire hazard includes:- determining (1010) if said item data satisfies at least one potential fire hazard condition indicating an item as a fire hazard,- if said item is determined as a potential fire hazard, reducing (1011 ) said temperature threshold, wherein said at least one potential fire hazard condition includes at least one of the following conditions:- an item classification of said item and at least one potential fire hazard in said item classification list is determined to match, and optionally, said item classification list includes any of the following item categories: batteries, such as lithium-based batteries; lighters; spray bottles; gas tanks; items comprising flammable material; and / or- at least one image of said item is determined to match with any of at least one fire hazard reference image by means of an image comparison algorithm and a matching criteria, each fire hazard reference image indicating a visual appearance of a fire hazard;- at least one image of said item is determined to match with any of said at least one fire hazard by means of an image classification engine trained to detect at least one fire hazard, wherein the image classification engine is trained using labeled image data indicating at least one fire hazard and / or using unlabeled image data in combination with fire hazard detection feedback.
10. Method according to any of the preceding claims, wherein said step of classifying (1003), based on said detection zone sensor data, at leastone item in the flow of matter in one of at least one item classifications includes:- determining (1012) a first material property set and / or said second material property from said detection zone sensor data;- comparing (1013) whether said first material property set and / or said second material property set of said item is associated with any one item classification of a list of item classifications including at least a first item classification defined at least in terms of one or more material properties of said first material property set and / or in terms of one or more material properties of said second material property set;- classifying (1014) said item as an item of a particular item classification based on said comparison;- optionally, if said item cannot be classified as an item of any item classification of the list of item classifications, adaptively updating (1015) the list of item classifications to include a new item classification for said item, which new item classification is defined in terms of one or more material properties of said first material property set and / or one or more material properties of said second property set of said item.11 . Method according to any preceding claims, wherein- the first property set is indicative of at least one of a spectral response of the matter, a material type of the matter, a colour of the matter, a fluorescence of the matter, a ripeness of the matter, a dry matter content of matter, a water content of the matter, a fat content of the matter, an oil content of the matter, a calorific value of the matter, a presence of bones or fishbones of the matter, a presence of pest of the matter, a mineral type of the matter, an ore type of the matter, a defect level of the matter, a detection of hazardous biological materials of the matter, a presence of matter, a nonpresence of matter, a detection of multilayer materials of the matter,a detection of fluorescent markers of the matter, a quality grade of the matter, a physical structure of the surface of the matter and molecular structure of the matter, and / or- the second property set is indicative of at least one of a height of the matter, a height profile of the matter, a 3D map of the matter, an intensity profile of reflected and / or scattered light, a volume centre of the matter, an estimated mass centre of the matter, an estimated weight of the matter, an estimated material of the matter, a presence of matter, a non-presence of matter, a detection of isotropic and anisotropic light scattering of the matter, a structure and quality of wood, a surface roughness and texture of the matter and an indication of presence of fluids in the matter.
12. Method according to any preceding claims, wherein said heat information data includes at least one thermal image of said item, the method comprising:- providing (1016) a black body reference and / or a white body reference within a field of view of the thermal sensor system;- adjusting (1017) said thermal image and / or said indicated temperature and / or said measured temperature of said item based on the black body reference and / or the white body reference, wherein the black body reference and / or the white body reference is / are stationary within said field of view or moveably into and out of said field of view, and / or the method comprising:- adjusting (1018) said at least one thermal image and / or said indicated temperature and / or said measured temperature of said item based on a material emission coefficient associated with a material of said item, and / or the method comprising:- adjusting (1019) said at least one thermal image, by means of image processing, to compensate for motion blur.
13. Method according to any of the preceding claims in combination with claim 4, wherein said temperature threshold is a baseline temperature threshold, said baseline temperature for said baseline temperature threshold being one of:- an ambient temperature;- an average temperature of the flow of matter as a whole;- an average temperature associated with said item classification wherein, optionally, said average temperature of said flow of matter as a whole or said average temperature associated with said item classification is provided by a thermal background model, said thermal background model being constructed based on at least one thermal image of the flow of matter obtained by said thermal sensor system, wherein, optionally, the method further comprising:- continuously adapting (1020) the thermal background model based on thermal images of the flow of matter obtained by said thermal sensor system.
14. Method according to any preceding claims, wherein said step of classifying (103) said at least one item in terms of item classification is performed based on any one of, or any combination of sensor measurements including:- hyperspectral sensor data, such as VIS and / or NIR hyperspectral sensor data, and / or- camera-based sensor, and / or- laser-based sensor data, and / or- X-ray-based sensor data, wherein, optionally, the classification arrangement comprises two or more different sensor system to provide any one of, or any combination of said sensor measurements, the method comprising:- registering (1021 ) two or more sensor systems of the classification arrangement, or all sensor systems of the classification arrangement, to the same coordinate system;- receiving (1022) detection zone sensor data from at least two or more sensor systems of the classification arrangement in said memory storage device, and- correlating (1023) in time and space said detection zone sensor data from said at least two or more sensor systems of the classification arrangement.
15. Method according to any one of the preceding claims, wherein the machine learning model comprises a model selected from the group comprising or consisting of: a decision tree, a random forest, a support vector machine, SVM,, a k-nearest neighbors, k-NN, model, a gradient boosted tree model, a clustering model, an autoencoder, a convolutional neural network, CNN, and a recurrent neural network, RNN.
16. Control unit (103) for a processing plant (100), adapted to communicatively couple to a transport arrangement (101 ) and a classification arrangement (102) of said processing plant (100), said control unit (103) configured to perform the method (1000) according to any preceding claims.
17. Processing plant (100), comprising:- a classification arrangement (102), which classification arrangement is optionally a classification and sorting arrangement;- a transport arrangement (101 ) configured to transport a material batch as a flow of matter to at least one detection zone (IZ) of the classification arrangement (102);wherein the classification arrangement (102) is configured to provide detection zone sensor data of the matter in one of the at least one detection zone (IZ); wherein the processing plant (100) further comprises:- a control unit (103) according to claim 16.
18. Method of fire hazard plant control of a processing plant (100) provided with a transporting arrangement (101 ) for transporting items (110) to a detection zone (IZ) and a classification arrangement (102), said classification arrangement (102) configured to classify items (110) present in said detection zone (IZ) based on at least one feature of said items and optionally to sort items (110) based on item classification, the method comprising:- transporting (1001 ), by means of the transport arrangement (101 ), a material batch as a flow of matter to at least one detection zone (IZ) of the classification arrangement (102),- providing (1002), by means of the classification arrangement (102), detection zone sensor data of matter captured when said matter was present in one of the at least one detection zone (IZ);- classifying (1003), based on said detection zone sensor data, at least one item (110) in the flow of matter as an item (110) of at least one item classification;- providing (1004) classification data comprising item classification of said at least one item (110) or information from which item classification for said at least one item is derivable;- establishing (1005), based at least on said classification data and / or said detection zone sensor data, item data of said at least one item (110);- detecting (1006), if said item data is determined to satisfy at least one fire hazard condition, said at least one item (110) as a fire hazard (FH), wherein said detection of said at least one item (110)as a fire hazard (FH) is based on, or assisted by, heat information data relating to: i) the same at least one item, and / or ii) items at least classifiable as one of said at least one item classification of said at least one item; the method further comprising:- controlling (1008), in response to a detection of a fire hazard, the processing plant (100) to initiate at least one fire hazard response action.
Citation Information
Cited By
Real-time fire preventing and extinguishing system for belt conveying system
CN122035538A