A system and method for assisting a deployed operator

Abstract

The present disclosure generally relates to a system for assisting a deployed operator in operational field conditions, where the system is specifically adapted to make use of a sensor module configured to combine infrared and LiDAR sensor data. The present disclosure also relates to a corresponding method for operating such a system.

Description

[0001] A SYSTEM AND METHOD FOR ASSISTING A DEPLOYED OPERATOR

[0002] TECHNICAL FIELD

[0003] The present disclosure generally relates to a system for assisting a deployed operator in operational field conditions, where the system is specifically adapted to make use of a sensor module configured to combine infrared and LiDAR sensor data. The present disclosure also relates to a corresponding method for operating such a system.

[0004] BACKGROUND

[0005] Emergency response operations, including for example firefighting, disaster relief, and tactical deployments, often involve rapidly evolving environments where visibility, access, and structural safety may be compromised. In such conditions, responders are required to make critical decisions while navigating areas with limited visual information, structural instability, or environmental hazards such as smoke, dust, or gas.

[0006] To support responders in these environments, various sensor technologies are applied to capture information about the surroundings. Such sensors are often carried by the responder or integrated into equipment worn or operated by the responder. For exam-pie, infrared (IR) sensors may be used to detect temperature variations indicative of heat sources or human presence. In some scenarios, visual cameras are also used to provide additional situational context. Data acquired from such sensors may be processed locally, in close proximity to the responder, in order to enhance the quality and relevance of the information presented. The objective is typically to assist the responder in making timely and well-informed decisions during ongoing operations.

[0007] With recent advances in artificial intelligence and computational methods, there is growing interest in applying more advanced forms of data analysis to sensor information acquired in the field. In some cases, the use of such techniques may involve computing resources with greater processing capacity than what is typically suitable for portable or body-worn sys-terns, resulting in the exploration of different system architectures where sensor-equipped field units operate in coordination with more capable processing platforms.

[0008] In view of the above, there appears to be room for further exploring how environmental sensing and data processing can be implemented in demanding field conditions, especially where rapid decision-making is critical and may directly impact responder safety and mission outcome.SUMMARY

[0009] In view of the above, it is an object of the present disclosure to provide improvements in relation to the implementation of environmental sensing and information delivery systems in operational field conditions, particularly for supporting a deployed operator.

[0010] According to an aspect of the present disclosure, it is therefore provided a system for assisting a deployed operator in operational field conditions, wherein the system comprises a first unit, comprising a first communication module, a sensor module comprising at least an infrared sensor and a LiDAR sensor, each of the sensors configured to generate sensor data, a first processing module configured to generate a first output based on a combination of the sensor data provided by each of the sensors of the sensor module, and a first output interface configured to provide situational information to the deployed operator, and a second unit operable remotely from the first unit, the second unit comprising a second communication module arranged in wireless communication with the first communication module and configured to receive at least a portion of the sensor data generated at the sensor module, a second processing module configured to generate a second output based on the received sensor data, and wherein the first unit is configured to select at least one of the first or second output as the situational information provided to the deployed operator.

[0011] The present disclosure is based upon the realization that advanced sensor data, such as temperature variations and spatial information, may be interpreted more effectively when a system architecture allows both local and remote processing, without relying on one processing point alone. By generating situational information both close to the deployed operator and at a remotely located second unit, the system enables a flexible and robust delivery of operational insights, for example including in situations where communication latency, environmental conditions, or task complexity varies.

[0012] Furthermore, the expressions “first output” and “second output” should within the context of the present disclosure be interpreted broadly to refer to any processed data generated respectively by the first processing module and the second processing module based on sensor data originating from the sensor module. These outputs represent intermediate or final data products derived from, for example, infrared and LiDAR sensor inputs, and may be the result of various processing techniques, including sensor fusion, environmental classification, spatial mapping, or feature extraction, as is further elaborated below.

[0013] As such, the outputs may take a range of technical forms, such as numerical models, annotated datasets, thermal overlays, 3D spatial reconstructions, symbolic representations, orparameterized alerts. The first output is typically generated locally at or near the deployed operator using the first unit, potentially with real-time constraints or limited computational resources. The second output, on the other hand, may be generated at a remote location with the second unit and may leverage more advanced or compute-intensive processing methods, such as Al-based interpretation, external data integration, or broader contextual modelling, as is elaborated further below.

[0014] Both the first and second outputs are intended to serve as candidate sources for the generation of “situational information” as provided to the deployed operator. Within the context of the present disclosure, the expression “situational information” should therefore be un-derstood to mean information derived from sensor data and processed into a form that supports the deployed operator in assessing the surrounding environment, identifying potential hazards, or understanding spatial layout and structural conditions.

[0015] Situational information may include, for example, visual overlays on a 3D map, auditory warnings, thermal contour maps, navigational indicators, or symbolic cues representing risk zones, obstacles, or safe routes. The form and content of the situational information may vary depending on operational context, available processing capacity, and environmental conditions. It may be based entirely on the first output, entirely on the second output, or on a combination of both, depending on which is available or prioritized at a given moment during operation.

[0016] Advantageously, the system allows the deployed operator to receive processed information in real-time without requiring guaranteed connectivity to the second unit. The ability to select between locally and remotely generated outputs enables a level of operational autonomy while still supporting richer interpretation through more powerful processing capabilities when available. The selection may also or instead be based on availability, data quality, or operational rules and does not require a specific fallback or redundancy trigger. Additionally, combining infrared and LiDAR sensor data in the first processing module supports the generation of more informative outputs, which may improve environmental awareness in visually obstructed or structurally complex settings.

[0017] In some embodiments, selection between the first and second outputs may be managed by a rule set that is applied dynamically during operation of the system according to the present disclosure. The selection logic may further evaluate factors such as signal latency, output completeness, confidence metrics, or operator-defined priority modes. For example, the system may default to locally generated output if a delay threshold is exceeded (for example above 1 - 5 seconds, while other thresholds of course are possible and within the scope of the present disclosure) or prefer second output when it includesremote classification results not available locally. Such dynamic prioritization enhances operational flexibility while ensuring the most relevant and actionable situational information is selected in real time.

[0018] The deployed operator typically includes a human responder, such as a firefighter, police officer, or soldier. However, the human responder may be assisted by a mobile field-deployed platform operating in proximity to the human responder, such as a drone or robotic vehicle equipped with the first unit. In such scenarios, a wireless communication link between the first and second units allows for greater mobility and system adaptability, which may be particularly relevant in search-and-rescue, training, or high-risk tactical deployments.

[0019] Use of the system according to the present disclosure is particularly beneficial in field operations where rapid decision-making is critical and may directly impact on the safety of the deployed operator or the effectiveness of the mission. In such conditions, delays or gaps in information can lead to suboptimal actions or increased risk. The ability to maintain a continuous flow of situational information, whether generated locally or remotely, ensures resilience to communication errors and supports operator awareness under challenging conditions.

[0020] In an optional embodiment of the present disclosure, it may be possible to implement a data enhancement scheme configured to filter or clean sensor data acquired by the sensor module prior to fusion or downstream processing. Such a scheme may be applied individually to each sensor stream, or in parallel with sensor fusion operations, and may be tailored based on contextual settings representative of the operational environment in which the deployed operator is located. For example, in field conditions affected by reduced visibility (such as smoke-filled or fog-obscured areas) each sensor modality may experience degradation in data quality, though to varying degrees. In such conditions, a visible-light camera may yield reduced contrast or fail to resolve features entirely, while LiDAR performance may degrade due to scattering and multipath effects. Furthermore, infrared sensors may remain partially effective but still exhibit signal attenuation or reduced contrast in heavy smoke or flame conditions.

[0021] In such a scenario, the enhancement scheme may apply filtering, denoising, or compensatory corrections to the raw sensor data, for example using learned models trained to identify and mitigate artifacts caused by smoke, heat haze, or particulate interference. The scheme may further include a contextual input representing the environmental condition (e.g. smoke level, ambient temperature, light availability), which may be inferred from auxiliary sensor readings or system metadata. In view of such contextual input, theenhancement scheme may selectively suppress low-confidence data, interpolate missing features, or re-weight data streams to favor more reliable sensor modalities, possibly resulting in a cleaner and more coherent multi-modal data representation that can be used to improve the performance of subsequent processing stages, such as object segmentation, environmental classification, and spatial mapping.

[0022] Possibly, the enhancement scheme may be implemented using a machine learning (ML) model configured to reconstruct or correct sensor features degraded under known environmental conditions. As an example, such a model may for example be trained to infer object contours from partially obscured LiDAR returns when smoke is present, or to enhance thermal boundaries detected by an infrared sensor based on patterns learned from annotated datasets. Such a model may also or alternatively operate in real time and may be deployed in the first processing module, allowing locally available data to be improved prior to transmission to the second unit. Additionally, combining the enhancement scheme with sensor fusion may provide a more stable and interpretable data product un-der degraded atmospheric conditions, thereby reducing user cognitive load and improving overall system performance.

[0023] In some embodiments, the first unit is configured to select the first output as the situational information in response to a communication delay or failure between the first and second communication modules. Such a configuration is particularly useful in situations where the wireless communication link between the first and second units becomes unstable or experiences excessive latency. Environmental conditions such as structural interference, smoke, or remote terrain may impact signal strength or availability. If the first unit were to wait for output from the second unit in these cases, delivery of situational information to the deployed operator could be delayed or interrupted.

[0024] To address such a scenario, the first unit may be implemented with a monitoring function that evaluates the communication link quality, for example performing periodic signal strength checks, latency measurements, or failure to receive acknowledgements within a set time window. Upon detection of a delay or disruption, the first unit can default to using the output generated locally at the first processing module. By enabling autonomous se-lection of the local output in response to degraded communication, the system ensures that the deployed operator continues to receive relevant situational information without relying on constant connectivity to the second unit, supporting uninterrupted decision-making during critical field operations. Providing such functionality improves system robustness and responsiveness in unpredictable or communication-constrained environments. It also al-lows the system according to the present disclosure to remain operational in the absenceof external infrastructure and contributes to overall reliability in mission-critical deployments.

[0025] Possibly, in some embodiments the sensor module is physically separated from the first processing module and connected via a wired or wireless data link. Such a configuration enables more flexible deployment of the sensor hardware, allowing the sensing functions to be positioned independently of the main processing and output components. Physical separation may be desirable in scenarios where the sensors must be exposed to hazardous or confined spaces, while the processing and output interface remain in a protected or more ergonomically accessible location on the operator (e.g. when the human responder is assisted by a drone in line with the discussion above).

[0026] The sensor module may for example be mounted on a helmet, handheld tool, or external structure, while the first processing module is carried by the body of the operator, for example in a backpack, a harness or belt-mounted unit. The connection between the modules may be implemented using a short-range wireless protocol (e.g., Bluetooth LE, ultra-wideband) or via a ruggedized wired connector, depending on the application environment and required data bandwidth.

[0027] Separating the sensor module from the processing module may increase mechanical design flexibility, support modular upgrades or replacements, and allow the sensing components to be placed closer to the relevant scene or hazard without relocating the full sys-tern. Such an implementation may be beneficial in tight spaces, high-temperature zones, or when dealing with potential contaminants. The use of a data link between the modules also permits a more distributed system architecture, which may be adapted for specific operational needs without affecting the processing logic or output generation pipeline. Additionally, providing the sensor module on a mobile robotic platform, such as a ground robot or aerial drone, enables the collection of environmental data from areas that may be inaccessible, unsafe, or suboptimal for direct human entry. Such a configuration may also allow the system to extend its sensing range and situational coverage while still delivering information back to the deployed operator. The remotely operated vehicle or drone may carry the sensor module and a communication interface that transmits the generated sensor data to the first unit.

[0028] Preferably, the sensor module further comprises a visible-light camera configured to generate additional sensor data. Including a visible-light camera in the sensor module allows the system to capture optical image data in addition to the thermal and spatial information provided by the infrared and LiDAR sensors. Such an implementation may enhance the ability of the system to interpret and visualize the environment in a form that may be moreimmediately recognizable and intuitive to the deployed operator. The visible-light camera may in line with the present disclosure be combined with the sensor data from the infrared and LiDAR sensors. The visible-light camera may for example be implemented using a standard image sensor, such as a CMOS or CCD device, configured for low-light or wide dynamic range performance. The camera may be calibrated alongside the infrared and LiDAR sensors to enable pixel- or frame-level correlation or may operate as an independent channel providing complementary visual context. The addition of visible-spectrum imaging facilitates overlay generation, hybrid visualizations, and real-time video feeds, which may be rendered at the first output interface or transmitted for remote monitoring. It also allows the use of conventional object detection, scene classification, or operator-guided interpretation, particularly in scenarios with good lighting or where color-based recognition is valuable.

[0029] In some embodiments, the sensor module may additionally comprise one or more supplemental sensors for capturing complementary environmental data. For example, the sensor module may include a gas sensor configured to detect hazardous gases, such as carbon monoxide or volatile organic compounds, or an acoustic sensor configured to detect environmental sounds, including structural stress noises or human calls. These additional sensor inputs may support enhanced interpretation of operational conditions, particularly in scenarios involving toxic exposure risks, partial collapses, or search-and-rescue missions. The first or second processing module may be configured to incorporate such inputs into the generation of situational information using suitable fusion or prioritization strategies.

[0030] In an embodiment of the present disclosure, the first output interface may be configured to provide at least one of image and audio-based situational information to the deployed operator. Providing situational information through image and / or audio output channels enables the system to adapt the presentation of information to the operational context and the immediate needs or constraints of the deployed operator. In environments with limited visibility, high noise levels, or physical barriers, relying solely on one modality may not be sufficient or practical.

[0031] The output interface may for example comprise a visual display integrated into or connected to the equipment of the operator, such as a helmet-mounted display, chestmounted screen, or wrist-worn device. The interface may also include an audio component, such as bone-conduction speakers, in-ear audio devices, or externally mounted loudspeakers, capable of delivering spoken cues, alerts, or tonal signals. The image-based output may include real-time sensor overlays, thermal maps, annotated 3D recon-structions, or live camera feeds. Audio output may include warnings, hazard notifications, or navigation cues generated either by rule-based logic or Al-driven interpretation of the sensor data. Accordingly, it may be possible to improve the likelihood that critical information will be perceived by the operator under varying environmental and cognitive load conditions. Such multi-modal delivery also supports redundancy in information presentation, which can be especially important when one sensory channel is impaired, such as for example due to smoke, darkness, or high ambient noise.

[0032] In line with the present disclosure, the first processing module may be configured to apply a sensor fusion algorithm to combine the sensor data generated by the infrared sensor and the LiDAR sensor. Applying a sensor fusion algorithm at the first processing module allows the system to generate situational information that integrates both thermal and spatial characteristics of the operational environment. Such a fusion of modalities provides a richer and more coherent representation of the surroundings than either sensor could deliver independently. In some embodiments the fusion algorithm may operate at different levels. For example, the fusion algorithm may operate at the raw data level (e.g. aligning depth points with thermal readings), at the feature level (e.g. extracting edges or heat signatures from each modality and merging them), or at the decision level (e.g. interpreting both inputs to identify hazards or safe zones). The algorithm may furthermore be implemented using conventional sensor alignment techniques, statistical models, or Al-based processing methods, depending on the complexity of the system and available computing resources.

[0033] Preferably, the first processing module is configured to weight the sensor data based on one or more environmental conditions detected during operation. By weighting sensor inputs dynamically, the system can adjust its data interpretation to changing environmental contexts. For example, in dense smoke conditions, infrared data may be more reliable than visual or LiDAR input. Conversely, in open and well-lit spaces, spatial and visual information may carry more significance. The processing module may apply predefined weighting rules or dynamically adjust weights based on detected environmental features, for example including light level, ambient temperature, motion patterns, or signal noise, The weights may influence how sensor data streams are fused, filtered, or prioritized for output generation. Accordingly, such an approach may also enhance the adaptability of the system and help ensure that the most relevant and reliable data sources are emphasized under specific conditions. It also contributes to reducing false positives or omissions in the generated situational information.

[0034] In some embodiments, the first or second processing module may further be configuredto trigger predefined response behaviors when specific environmental thresholds are detected. For example, the system may initiate a directional retreat recommendation when temperatures exceed a critical limit or suppress visual overlays in favor of auditory alerts under low-visibility conditions. Such rule-based adjustments may enhance operator fo-cus and responsiveness in high-stress scenarios and support prioritized delivery of critical information in line with real-time environmental changes.

[0035] Furthermore, in some embodiments the first processing module may be configured to generate a hybrid spatial representation of the operational environment based on the combined sensor data. A hybrid spatial representation refers to a processed output that inte-grates multiple sensor modalities into a unified spatial model, by for example combining LiDAR-generated 3D geometry with thermal overlays derived from the infrared sensor. In such an embodiment, the first processing module may map temperature data onto Li-DAR point clouds or mesh surfaces to create an annotated 3D view of the environment. Such a hybrid view may include temperature gradients, identify hotspots, or inferred object classifications embedded within the spatial layout.

[0036] In embodiments of the present disclosure, the second processing module comprises an Al model configured to classify environmental risks based on the received sensor data. Implementing an Al-based classification allows the second unit to interpret sensor data with higher abstraction, identifying patterns or anomalies that may indicate threats, hazards, or operational relevance. Here, the Al model may be trained to detect signatures of specific risks, such as structural collapse indicators, heat propagation patterns suggestive of fire spread, or visual cues associated with blocked egress routes. Classification results may then be used to generate interpretive situational information that goes beyond raw data or basic overlays. Furthermore, by offloading complex classification tasks to the second unit (which in line with the present disclosure may have access to greater computational resources) the system can perform deeper analysis without imposing additional processing burden on the first unit worn or carried by the operator.

[0037] In embodiments of the present disclosure, the Al model may be configured to adapt its analysis based on prior operational scenarios or training data. The Al model may as such improve its accuracy and relevance over time, for example through supervised retraining, online learning, or scenario-specific optimization. For example, the model may adjust how it interprets temperature distributions in different types of buildings or environments based on past data. Additionally, adapting the Al analysis in such a way may also increase the robustness of the model and performance in realistic field scenarios, particularly when deployment conditions deviate from the original training set. Furthermore, such an imple-mentation also opens the possibility for operator-specific or mission-specific tuning of the classification logic.

[0038] In a further embodiment, the second processing module may be configured to perform predictive modelling of environmental risks based on received sensor data and known reference conditions. For example, the second processing module may apply trained Al models configured to forecast the likely spread of a fire, assess risk of structural collapse, or infer potential hazard zones based on heat accumulation or smoke propagation trends. Such predictive outputs may also supplement or replace direct hazard identification and may be included in the generated situational information or delivered to external systems for supervisory analysis and response planning.

[0039] In embodiments of the present disclosure, the first unit may be configured to prioritize the second output over the first output when the received sensor data comprises data not available to the first processing module, possibly allowing the system to dynamically select the most complete or informative data source for generating situational information. For example, the first unit may monitor the data coverage of its local processing versus the received output from the second unit and apply a prioritization rule when the second output includes information beyond what is accessible locally.

[0040] In embodiments of the present disclosure, the second processing module may be configured to process a filtered subset of the received sensor data based on bandwidth con-straints or relevance scoring. By managing limited communication bandwidth between the first and second units, the system according to the present disclosure may transmit only a selected portion of the sensor data, prioritizing segments that are most likely to contribute to meaningful situational analysis. Filtering may, for example, be based on predefined rules (e.g., proximity to operator, temperature thresholds, rate of change) or on relevance scores computed locally before transmission. The second unit may then process only the subset it receives, rather than expecting complete raw datasets. As a result, such an implementation may efficiently use communication resources and ensure that available processing capacity is directed toward the most critical information. It also supports scalable deployment in bandwidth-constrained environments.

[0041] In a possible embodiment of the present disclosure, the first processing module may be configured to generate a real-time three-dimensional map of the operational environment based on the combined sensor data. Real-time 3D mapping allows the operator to visualize structural layouts, obstacles, and spatial context as they move through an environment. Such a capability is particularly useful in unfamiliar, obstructed, or rapidly changing scenarios, such as interior firefighting or collapsed structures. Possibly, the map may begenerated by integrating LiDAR data with position information and then further enhanced with thermal overlays or annotations derived from infrared data, where the resulting 3D model can be rendered live on the first output interface.

[0042] Additionally, it may also be possible, in line with the present disclosure, to configure the second processing module to also incorporate external data sources for contextual interpretation of the received sensor data. Incorporating external data allows the second unit to interpret sensor inputs in light of broader contextual information. Examples of external data include building plans, satellite imagery (if outdoor), weather data, or previously acquired operational logs. Such data may assist in interpreting ambiguous sensor readings, confirming hazard types, or generating predictions (e.g., in relation to fire spread modelling or structural risk analysis).

[0043] In a preferred embodiment, the first output interface may be configured to allow the deployed operator to select a data view based on one or more sensor data types. Providing the operator with control over which data type is displayed, such as for example, ther-mal view, structural layout, combined overlay, or raw camera feed, supports flexibility and responsiveness in the field. Additionally, the interface may in some embodiments allow manual toggling or automated switching based on context or gestures, assisting in tailoring the displayed information to the specific task or visibility conditions, while avoiding overwhelming the operator with unnecessary or redundant data.

[0044] In a further embodiment, the system may support fusion and output tailoring based on operational objectives, whereby the processing strategy adapts to the mission type or operator role. For example, during evacuation scenarios, thermal anomalies may be prioritized, while in structural assessment missions, 3D geometry may be emphasized. The first unit may receive configuration parameters defining such priorities before or during deployment, ensuring alignment with mission goals.

[0045] In embodiments of the present disclosure, the first processing module may be configured to compare current sensor data to previously acquired sensor data to detect environmental changes over time. Incorporating a temporal comparison function allows the system to identify evolving patterns or changes in the operational environment, such as increasing heat signatures, structural shifts, or movement of objects. Possibly, the first processing module may also be configured to store recent snapshots of sensor data in short-term memory and apply frame-to-frame or scan-to-scan comparisons to identify deviations, for example involving detecting temperature rise, obstacle displacement, or expanding zones of obscuration.In embodiments of the present disclosure, the second unit may also be configured to log received sensor data and associated outputs for post-operation review. By logging data and decision-support outputs, it may be possible to enable post-mission analysis, training, auditing, and reconstruction of events, which is particularly useful in scenarios involving incident investigation, responder debriefing, or improvement of operational protocols. Furthermore, the second unit may store timestamped sensor data streams, output selections, operator inputs, and Al classifications in a structured format suitable for later access. Storage may be local or cloud-based, depending on the infrastructure.

[0046] In some embodiments, the logging functionality of the second unit may include secure and tamper-resistant data handling features. For example, sensor data streams, processing outputs, and operator decisions may be cryptographically signed, version-tracked, or stored within an immutable log structure to support traceability and post-event verification. Such implementations may be relevant in contexts requiring regulatory compliance, incident auditing, or forensic review.

[0047] It may also, in some embodiments of the present disclosure, be possible to configure the first processing module to operate in different processing modes selected based on a type of operational scenario. Support for selectable processing modes allows the system to optimize performance based on known or expected conditions, for example, operating in “firefighting”, “hazardous gas”, or “search-and-rescue” mode. Each mode may apply different fusion logic, sensor prioritization, alert thresholds, or output formats. The processing mode may as such be pre-configured before deployment or selected dynamically by the operator or remote command system.

[0048] Preferably, the second unit may additionally be configured to transmit situational information to an external command system for coordination of multiple deployed operators. Transmitting processed situational information to a central command platform enables coordination across multiple operators or teams in the field. The command system may receive real-time environmental updates, operator positions, identified risks, and other mission-relevant data. Such integration supports broader situational awareness, command decision-making, and resource management during complex deployments.

[0049] In some implementations, the second unit may be configured to receive sensor data from multiple distinct first units operating concurrently within a shared operational area. Such a configuration enables a distributed sensing architecture in which environmental information is aggregated across units and processed centrally to create a broader situational model. Alternatively, separate sensor modules mounted on independent mobile platforms may transmit sensor data to a common second unit for coordinated interpretation. Suchmulti-agent architectures support scalable deployments and enable shared awareness across distributed personnel or autonomous units, particularly useful in wide-area search or incident containment scenarios.

[0050] According to another aspect of the present disclosure, there is provided a method for oper-ating an assistance system in operational field conditions, wherein the system comprises a first unit arranged in a vicinity of deployed operator, the first unit comprising a first processing module and a sensor module, wherein the sensor module comprises at least an infrared sensor and a LiDAR sensor, and wherein each of the sensors is configured to generate sensor data, a second unit operable remotely from the first unit, the second unit comprising a second processing module configured to generate an output based on the sensor data from the sensor module, wherein the method comprises the steps of initiating, by the first unit, transmission of at least a portion of the sensor data from the sensor module to the second unit via a wireless communication link, generating, by the first processing module, a first output based on the sensor data generated at the sensor module, evaluating, at the first unit, whether the output from the second unit satisfies a predefined quality metric, selecting, at the first unit, at least one of the outputs from the first and the second units as situational information based on the evaluation, and providing, by the first unit, the situational information to the deployed operator via an output interface. This aspect of the present disclosure provides similar advantages as discussed above in relation to the previous aspect of the present disclosure.

[0051] In some embodiments, the system may additionally be configured to interface with external suppression or safety infrastructure. For example, the first or second unit may transmit control signals to activate localized fire suppression systems, trigger ventilation actuators, or initiate containment measures in response to detected hazards. Such integration may be particularly applicable in semi-structured environments, including industrial sites or transport hubs, where automated mitigation systems can be coordinated alongside operator guidance.

[0052] It should be understood that although the system described herein comprises both the first and second units operating in cooperation, either unit may be implemented, deployed, or provided separately depending on the use case, infrastructure constraints, or system architecture. For example, the first unit may be provided as a standalone operational tool, worn or carried by the deployed operator, and capable of processing sensor data locally and delivering situational information independently of any external server or infrastructure. In such implementations, the first unit may be configured to operate autonomously and may omit the second unit altogether.Similarly, the second unit may be provided as an independent processing platform configured to receive sensor data from one or more remote sources. For instance, the second unit may be implemented as a remote server, edge computing node, or a cloud-based backend system provided via a software-as-a-service (SaaS) platform. In such config-urations, the second unit may process incoming sensor data to generate output without requiring the presence of a specific local output interface or deployed operator. Outputs generated by the second unit may be delivered to third-party systems, command infrastructure, or other interfacing components.

[0053] Accordingly, the present disclosure also encompasses implementations wherein only the first unit or only the second unit is provided, and such implementations may be adapted for use in different operational scenarios. The independent configuration and processing capabilities of each unit allow them to serve as distinct technical components, either within an integrated system or as standalone functional devices.

[0054] Further features of, and advantages with, the present disclosure will become apparent when studying the appended claims and the following description. The skilled addressee realizes that different features of the present disclosure may be combined to create embodiments other than those described in the following, without departing from the scope of the present disclosure.

[0055] BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The various aspects of the present disclosure, including its particular features and advantages, will be more readily understood from the following detailed description and the accompanying drawings, in which:

[0057] Fig. 1 conceptually illustrates a system according to a currently preferred embodiment of the present disclosure,

[0058] Figs. 2A and 2B illustrate example implementations of the first unit used in relation to the system of Fig. 1, where Fig. 2A shows the first unit provided with a deployed operator and Fig. 2B shows a configuration involving both a deployed operator and a remotely operated vehicle,

[0059] Fig. 3 illustrates a conceptual implementation of the second unit used in relation to the system of Fig. 1, and an optional external entity receiving situational information, and Fig. 4 is a flow chart conceptually illustrating operational steps performed by the system according to the present disclosure.DETAILED DESCRIPTION

[0060] Turning now to the drawings and to Fig. 1 in particular, there is conceptually illustrated an assistance system 100 configured for supporting a deployed operator in operational field conditions. The system 100 comprises a first unit 102, which is generally arranged in proximity to the deployed operator and may be worn or carried during field deployment, and a second unit 120 operable remotely from the first unit 102.

[0061] The first unit 102 comprises a first processing module 104 and a sensor module 106. The sensor module 106 includes at least an infrared sensor 108 and a LiDAR sensor 110, each configured to generate sensor data representative of the operational surroundings. The infrared sensor may comprise or be implemented as a thermal imaging sensor configured to detect temperature distributions or heat sources in the environment. The sensor module 106 may for example additionally include one or more supplemental sensors, such as acoustic sensors, gas sensors, or visual sensors including a visible-light camera, configured to detect environmental anomalies or provide complementary context not visible in thermal or spatial data. As discussed above, a visual sensor such as a camera may be provided and combined with the sensor data from the sensor module 106. The sensor data is processed locally by the first processing module 104, which is configured to generate a first output based on a combination of the sensor data from the infrared and LiDAR sensors 108, 110.

[0062] The first unit 102 further comprises a first output interface 112, which is configured to present situational information to the deployed operator. In some embodiments, the output interface may include a display unit configured to present a graphical user interface (GUI). The GUI may be arranged to provide image-based and / or symbolic information to the operator and, in certain implementations, may also support user interaction, such as selection of data views, confirmation of system prompts, or manual override functions. The display may be integrated into the gear of the operator, such as a helmet-mounted screen or a chest-worn interface module. The output interface may also comprise an audio component, such as a speaker or headset link, for delivery of auditory alerts or navigation cues.

[0063] Communication between the first unit 102 and the remotely located second unit 120 is facilitated by a first communication module 114, configured for wireless transmission and reception of data. The wireless communication may utilize standard protocols and physical layers, such as Wi-Fi, LTE, 5G, Bluetooth, or dedicated short-range radio links, depending on operational field conditions and system configuration.The second unit 120 is operable remotely from the first unit and comprises a second communication module 122, arranged in wireless communication with the first communication module 114 and configured to receive at least a portion of the sensor data generated by the sensor module 106. The second unit 120 further comprises a second processing module 124, configured to generate a second output based on the received sensor data. The second unit 120 may be implemented in various forms depending on the operational architecture. For example, the second unit may be provided as a field server, a fixed base station, or as a cloud-based backend system deployed within an enterprise or public infrastructure.

[0064] In some implementations, the second processing module 124 may be located on a remote computational server, accessible via secure data links over cellular or satellite networks. In such configurations, the second unit is capable of applying more advanced processing techniques, such as artificial intelligence (Al)-based risk classification, contextual modelling using external datasets, or fusion with legacy or geographic information system (GIS) data sources. The second processing module 124 may apply predictive models to infer risk trajectories, such as estimating fire spread or structural instability, based on temporal patterns in received sensor data. The results of the second processing module 124 may be transmitted back to the first unit 102 for selection and display, or to third-party systems for monitoring, coordination, or further interpretation, as will be further discussed below in relation to Fig. 3.

[0065] The first and second processing modules 104, 124, may be realized using a variety of computational components. Each processing module 104, 124 may be manifested as a general-purpose processor, a graphics processing unit (GPU), an application-specific processor, a circuit containing processing components, a group of distributed processing components, or a field-programmable gate array (FPGA). In some embodiments, system-on-chip (SoC) solutions may be used to consolidate processing and communication functionality in compact embedded form factors. The processor may include any number of hardware components suitable for conducting signal processing, data evaluation, spatial modelling, or for executing software instructions.

[0066] Each processing module 104, 124 may be communicably connected to one or more memory devices, which may be locally integrated or remotely accessible. The memory may comprise one or more local or distributed devices for storing sensor data, intermediate processing results, or executable software code for implementing processing tasks. Suitable memory types include volatile memory, non-volatile memory, embedded database systems, or network storage resources. In some embodiments, memory contents may in-elude code supporting environmental interpretation, sensor calibration, decision logic for output selection, or Al model parameters.

[0067] Turning now to Fig. 2A, there is illustrated a first detailed embodiment of the first unit 102 in use by a deployed operator 202 in an operational field environment 210. In the present embodiment of the present disclosure, the first unit 102 is arranged to be worn or carried by the operator 202, who is equipped with a harness 204 designed to support and stabilize the various components of the first unit 102. The harness 204 includes a mounting structure for the sensor module 106, such that the infrared sensor 108 and LiDAR sensor 110 are oriented in a forward-facing direction relative to the operator 202, thereby allowing the sensors to acquire data representative of the environment directly in front of the operator 202 as they move through the environment 210.

[0068] The sensor module 106 may be physically integrated into the harness 204 or mounted on an articulated frame to allow limited directional adjustment. In some embodiments, the harness 204 may include passive vibration isolation features or protective housing elements to safeguard the sensor module 106 during movement through physically challenging environments, such as rubble, smoke-filled corridors, or narrow passageways. Also illustrated in Fig. 2A is a face mask 212 worn by the operator 202, which is provided with or connected to a user interface 214. The user interface 214 may be configured to receive and display situational information from the first output interface 112 and may com-prise a compact display positioned within the field of view of the operator 202. The user interface 214 may possibly be implemented as an optical see-through visor, monocular micro-display, or heads-up display (HUD) integrated into the structure of the face mask 212. The user interface 214 may additionally support user interaction, for example through tactile input, voice control, or gesture detection.

[0069] The user interface 214 may additionally support operator feedback input, for example to confirm perceived hazards, mark points of interest, or override system-suggested interpretations. Such operator annotations may be stored locally or transmitted to the second unit, allowing real-time refinement of Al classification models or informing broader team coordination.

[0070] In the illustrated example, the operator 202 is positioned within the operational environment 210, and a fire 216 is present ahead. In such a scenario, the combination of thermal sensing from the infrared sensor 108 and spatial data from the LiDAR sensor 110 may support enhanced detection of heat sources, obstructions, or structural openings otherwise obscured by smoke or visual clutter. The processed situational information may becontinuously or periodically updated and delivered to the user interface 214 in order to guide movements and decisions made by the operator 202.

[0071] Turning now to Fig. 2B, an alternative embodiment is illustrated in which the sensor module 106 is provided separately from the remainder of the first unit 102. In the illustrated embodiment, the sensor module 106 is mounted to a mobile unmanned platform 220 arranged to operate within the same operational environment 210 as the operator 202. The platform 220 may be implemented in various physical forms, including but not limited to ground-based legged robots, tracked or wheeled unmanned vehicles, or aerial drones. The degree of autonomy may vary depending on system configuration and use case, including manually operated platforms controlled by the operator 202, semi-autonomous platforms with waypoint navigation or obstacle avoidance, or fully autonomous systems operating based on onboard or remote intelligence.

[0072] The platform 220 carries the sensor module 106 in a forward-facing orientation to enable environmental scanning ahead of its path. In some implementations, the sensor module 106 may be mounted on a gimbal or articulated mast to support active directional control or panoramic scanning. Sensor data acquired by the sensor module 106 on the platform 220 may be wirelessly transmitted back to the first processing module 104, which remains physically associated with the operator 202. Alternatively, the data may be transmitted to the second unit 120 for further processing, depending on bandwidth availability, process-ing load, or operational preferences.

[0073] Such a configuration allows environmental data to be gathered from locations that may be hazardous, structurally unstable, or otherwise inaccessible to the operator 202. For example, the platform 220 may be deployed into a collapsed structure, smoke-filled tunnel, or toxic environment, while the operator 202 remains in a safer area and receives processed situational information remotely. As also discussed above, the physical separation of the sensing component and the processing / interface unit expands the operational range and flexibility of the system 100 while maintaining the ability to support the operator 202 in real time. It is to be noted that in both the embodiment illustrated in Fig. 2A and the alternative embodiment in Fig. 2B, the system 100 leverages its modular architecture to adapt to different deployment scenarios while maintaining robust environmental sensing, local or remote processing, and continuous delivery of relevant situational information to the operator 202.

[0074] Turning now to Fig. 3, there is provided a further detailed view of the second unit 120 according to the present disclosure, illustrating how the system architecture may supportdistributed or centralized processing outside the immediate vicinity of the deployed operator 202. The second unit 120 may receive data from multiple sensor modules deployed across different platforms, enabling aggregated environmental analysis or operator coordination through a distributed sensing architecture. As previously introduced, the second unit 120 is operable remotely from the first unit 102 and is configured to receive sensor data originating from the sensor module 106, either directly or via the first communication module 114.

[0075] In the embodiment illustrated in Fig. 3, the second unit 120 comprises one or more processing entities, shown conceptually as a processing cluster 300. The processing cluster 300 may represent a group of computational nodes arranged as local field servers, edge computing devices, or remote cloud-based infrastructure. The distribution of processing tasks across multiple nodes allows the second unit 120 to scale according to operational complexity, data load, or mission requirements.

[0076] Each node in the processing cluster 300 may include an instance of the second process-ing module 124, and may be configured to handle specific processing tasks, for example, image enhancement, thermal pattern recognition, 3D mapping, or risk classification. In some embodiments, the second processing module 124 may access external data repositories, such as building plans, historical environment data, satellite imagery, or previously logged mission data, thereby enabling contextual interpretation of the incoming sensor data.

[0077] Furthermore, when greater computational power is available, the second processing module 124 may execute advanced analysis routines, including deep neural networks or other Al-based models trained to interpret multi-modal sensor inputs. For instance, temporal-spatial heat pattern recognition may be used to infer structural fire progression, or LiDAR-based geometry analysis may assist in collapse risk assessment. Classification results may be fused into the second output and transmitted back to the first unit 102 or routed onward depending on system configuration.

[0078] To support coordination and oversight, the second unit 120 may also be configured to interface with an external command system 310, for example operated by a remote supervisor 320 or mission coordinator. In the embodiment shown in Fig. 3, the command system 310 includes a display terminal and user interface 322 configured to receive, review, and interpret the second output generated by the processing cluster 300. The command system 310 may present live or near-real-time processed environmental information, situational cues, or operator status data.In some embodiments, the command system 310 may also be configured to transmit situational information or feedback back to the first unit 102 for presentation to the operator 202, either in support of mission-level guidance or in response to detected hazards not yet observed locally by the operator. The command system 310 may provide the feedback via the same wireless link used for regular system 100 operation and may in some embodiments also be configured to influence output selection or processing parameters at the first unit 102. For example, the command system may instruct the first processing module 104 to switch modes, request a change in sensor orientation, or relay urgent navigational directives.

[0079] That said, even though the command system 310 may play a central role in managing a team of multiple operators 202, the architecture also accommodates use in single-operator scenarios. In such cases, the command system 310 may serve as a safety monitor, remote support terminal, or real-time analysis companion, offering an additional layer of situational evaluation and feedback. Such an implementation may also support mission flexibility and extend the utility of the second unit 120 beyond basic data offloading. In all such embodiments, the second unit 120 enhances the capability of the system 100 by enabling distributed intelligence, multi-level feedback, and external coordination, thus improving mission effectiveness and operator safety without compromising the autonomy of local decision-making at the first unit 102.

[0080] With reference now to Fig. 4, there is presented an exemplary operational flow of the assistance system 100 in further detail. Specifically, Fig. 4 illustrates the sequence of actions performed by the system 100 during typical use in the field, including interaction between the first unit 102, the second unit 120, and the deployed operator 202. In line with the present disclosure, the first unit 102 initiates, SI, transmission of at least a portion of the sensor data to the second unit 120 via the wireless communication link established between the first communication module 114 and the second communication module 122. Depending on bandwidth conditions, mission constraints, or local preprocessing logic, the transmitted data may be filtered, prioritized, or compressed. Such an implementation may also support real-time operation even under constrained or variable connectivity.

[0081] In parallel with the data transmission, or independently thereof, the first processing module 104 generates, S2, a first output based on the locally available sensor data. As described earlier, the output may be derived through sensor fusion, spatial mapping, or environmental feature extraction. Where applicable, the locally generated first output is also logged for temporal comparison or post-mission analysis.

[0082] Upon receipt of the sensor data, the second processing module 124 of the second unit 120processes the data to generate a second output. As illustrated in Fig. 3, the output may be produced through more computationally intensive methods, including Al-based risk classification, multi-modal interpretation, or contextual integration with external datasets. The second output is then transmitted back to the first unit 102. At this point, the first unit 102 performs, S3, an evaluation of the second output. Such an evaluation may compare quality, latency, completeness, or confidence scores of the received second output against a predefined quality metric. The evaluation logic may be static, rule-based, or dynamically adjusted based on environmental or mission parameters. In case of non-availability, e.g., the quality will be considered as low or null and subsequently handled accordingly.

[0083] Based on the evaluation, the first unit 102 then selects, S4, an output, either the first output or the second output, or a combination of them both, as the situational information to be provided to the operator. The selection may also prioritize the first output in scenarios where the second output is unavailable or delayed or may select the second output where more comprehensive analysis or external context has been applied.

[0084] Finally, the selected situational information is presented, S5, to the deployed operator 202 via the first output interface 112, which may include a display and / or audio interface as described in Fig. 2A. The output may comprise thermal overlays, spatial maps, hazard cues, or navigational guidance, tailored to improve situational awareness and support timely and safe decision-making. In some configurations, additional input from the external command system 310 may also be incorporated, providing strategic-level assistance or supplemental insights from a remote supervisor or coordination center.

[0085] Furthermore, the control functionality of the present disclosure may be implemented using existing computer processors, or by a special purpose computer processor for an appropriate system, incorporated for this or another purpose, or by a hardwire system. Embod-iments within the scope of the present disclosure include program products comprising machine-readable medium for carrying or having machine-executable instructions or data structures stored thereon. Such machine-readable media can be any available media that can be accessed by a general purpose or special purpose computer or other machine with a processor. Byway of example, such machine-readable media can comprise RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, solid state drives or other non-volatile flash based storage devices, or any other medium which can be used to carry or store desired program code in the form of machine-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer or other machine with a processor. When information is transferred or provided over a network or anothercommunications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a machine, the machine properly views the connection as a machine-readable medium. Thus, any such connection is properly termed a machine-readable medium. Combinations of the above are also included within the scope of machine-readable me-dia. Machine-executable instructions include, for example, instructions and data which cause a general-purpose computer, special purpose computer, or special purpose processing machines to perform a certain function or group of functions.

[0086] In some embodiments, the infrared sensor and the LiDAR sensor are selected as complementary sensing modalities for operation in degraded operational environments. In-frared sensing may remain useful where visible-spectrum imagery is obstructed, while LiDAR sensing may provide spatial geometry and ranging information even when thermal interpretation alone is insufficient. The respective sensing modalities may also exhibit different degradation modes under field conditions. For example, infrared sensing may be adversely affected by thermal wash-out, sensor saturation, or reduced thermal con-trast in high-heat environments, whereas LiDAR sensing may be adversely affected by smoke scattering, particulate attenuation, backscatter, multipath effects, or reduced return quality in particulate-dense environments. By combining infrared and LiDAR sensor data, the system may maintain situational awareness with greater robustness than would be achievable using either modality alone.

[0087] In some embodiments, fusion of infrared sensor data and LiDAR sensor data may be performed at one or more of a raw-data level, a feature level, and a decision level. At the raw-data level, thermal values or thermal image data may be geometrically registered to LiDAR-derived spatial coordinates, for example by projecting thermal values onto a LiDAR point cloud, mesh, voxel space, or occupancy representation. At the feature level, features extracted from the infrared data, such as heat-signature edges, hotspots, thermal gradients, or thermal contours, may be merged with features extracted from the LiDAR data, such as contours, surfaces, openings, obstacles, or structural boundaries. At the decision level, classifications, hazard assessments, target identifications, route suggestions, or operator alerts independently derived from each sensing modality may be combined to generate a fused situational assessment.

[0088] In some embodiments, the first processing module and / or the second processing module is configured to generate a hybrid spatial representation of the operational environment in the form of a thermal-annotated three-dimensional data structure. The thermal-annotated three-dimensional data structure may comprise, for example, a thermal-annotated point cloud, a thermal-annotated mesh, a thermal-annotated occupancy grid,a thermal-annotated voxel model, or a thermal-annotated digital twin of at least a portion of the operational environment. Such a representation may associate thermal attributes with spatial geometry such that an object, surface, boundary, route, opening, or hazard may be understood in both thermal and geometric terms.

[0089] In some embodiments, the first output and the second output each comprise a respective candidate situational output derived from common or overlapping sensor observations. The first output may be generated locally and the second output may be generated remotely, and each may be independently suitable for presentation to the deployed operator as situational information. In such embodiments, the second output is not limited to optimization data, calibration data, update parameters, or metadata for modifying the first output, but may instead comprise a complete or partial situational output capable of being selected in place of the first output.

[0090] In some embodiments, the first unit comprises a selection module, output arbiter, output manager, or switching controller configured to receive the first output and the second output as separate candidate inputs and to select at least one of the first output or the second output for presentation to the deployed operator. The selection module may be logically separate from the first processing module, may be implemented as hardware, software, firmware, or a combination thereof, and may operate continuously, periodically, or event-driven during operation of the system.

[0091] In some embodiments, the predefined quality metric used to evaluate the second output comprises one or more of: latency, completeness, confidence, spatial coverage, temporal freshness, classification stability, communication quality, modality availability, output validity, environmental suitability, and task relevance. Latency may correspond to a time from transmission of sensor data or a request to receipt of the second output. Completeness may correspond to whether the second output includes a full result, a partial result, or an error state. Confidence may correspond to a score, probability, certainty measure, ranking, or confidence interval associated with a classification, detection, segmentation, or interpretation result. Environmental suitability may correspond to an estimate of whether the second output is likely to be more reliable than the first output under a presently de-tected environmental condition.

[0092] In some embodiments, evaluation of the second output may comprise applying a watchdog timer, timeout window, threshold comparison, null-response check, error-state check, or validity check. If the second output is not received within an evaluation window, is received in an error state, is incomplete, or fails to satisfy one or more quality conditions, a quality value associated with the second output may be set to a low value, a null value, or a zerovalue for purposes of output selection.

[0093] In some embodiments, the first unit is configured to present the first output by default and to switch to the second output when the second output satisfies the predefined quality metric. In some embodiments, the first unit is configured to present the second output when the second output satisfies the predefined quality metric and to revert to the first output when a communication delay, communication interruption, output degradation, output invalidity, or loss of confidence is detected. In some embodiments, switching between the first output and the second output may be stateful, hysteretic, threshold-based, or mode-based, such that unnecessary oscillation between outputs is reduced.

[0094] In some embodiments, the first output is not overwritten by the second output. Rather, the first output and the second output may be maintained as distinct candidate outputs, and the selected situational information may comprise the first output, the second output, or a composition generated after selection. In some embodiments, selection comprises replacement of one candidate output with another candidate output for operator presen-tation, rather than passive updating of one output using optimization data returned from a remote processor.

[0095] In some embodiments, weighting of the infrared sensor data and the LiDAR sensor data is dynamically adjusted based on one or more environmental conditions detected during operation. The environmental conditions may comprise one or more of flame presence, high ambient temperature, hot gas convection, smoke density, particulate loading, fog, steam, reflective surfaces, enclosed spaces, outdoor spaces, low-light conditions, and structural confinement. In some embodiments, the weighting may cause the system to rely more strongly on thermal interpretation under one condition and more strongly on spatial geometry under another condition.

[0096] In some embodiments, the first unit is configured for use by an emergency responder, such as a firefighter, rescue worker, hazardous-material responder, police officer, military operator, or other deployed operator acting in a hazardous field environment. The first output interface may be integrated into or operably coupled to a helmet, visor, face mask, breathing apparatus mask, goggles, headset, body-worn display, chest-mounted display, wrist-mounted display, or other responder equipment. Audio situational information may be delivered through speakers, in-ear devices, bone-conduction transducers, or other acoustic interfaces.

[0097] In some embodiments, the first unit is configured to operate under intermittent or degraded communications connectivity. Such degradation may arise from structural attenuation, ter-rain, smoke, heat, building materials, range limitations, infrastructure unavailability, network congestion, or mission movement. In such embodiments, generation of the first output at the first unit provides a local fallback path by which situational information remains available even if the second output is delayed, absent, incomplete, or below threshold quality.

[0098] In some embodiments, the sensor module is mounted on a body-worn assembly, a helmetmounted assembly, a handheld assembly, a mast, a wall-deployable unit, a pole, a robotic vehicle, an unmanned ground vehicle, an aerial drone, or another mobile or stationary platform. The first processing module may be co-located with the sensor module or physi-cal ly separated from the sensor module and connected thereto by a wired or wireless data link. The first unit may therefore comprise a distributed local architecture in which sensing, processing, and operator interfacing are not necessarily located in a single enclosure. In some embodiments, the second unit is implemented as one or more of: an edge server, a vehicle-mounted compute node, a mobile command post, a portable compute unit, a base-station-adjacent processor, a cloud server, a cluster of servers, or a distributed compute environment. The second unit may be configured to receive sensor data from one first unit or from a plurality of first units, and to generate respective second outputs for one or more deployed operators operating in the same or different operational areas.

[0099] In some embodiments, the second processing module is configured to process a filtered subset of the sensor data based on bandwidth availability, relevance scoring, hazard likelihood, region-of-interest selection, operator task mode, or communication constraints. The filtered subset may correspond to one or more zones, one or more object candidates, one or more thermal anomalies, one or more LiDAR clusters, or one or more segments of a three-dimensional representation.

[0100] In some embodiments, the system is configured such that the second output comprises information not locally available to the first processing module, including one or more of: context derived from external datasets, model-based hazard prediction, data aggregated from other units, historical scene information, broader-area mapping, command-level information, or higher-complexity Al classification. In such embodiments, the first unit may prioritize the second output over the first output when the second output contains such additional information and satisfies the predefined quality metric.

[0101] In some embodiments, the situational information presented to the deployed operator may comprise one or more of: a thermal-annotated three-dimensional map, a route indicator, a hotspot marker, a structural hazard marker, a victim indicator, an obstruction indicator, agas-warning indicator, a classification label, a confidence annotation, an audio warning, a spoken instruction, a mode-specific overlay, or a fused environmental visualization. The deployed operator may select between different data views, and each view may emphasize one or more sensor modalities or fused outputs.

[0102] In some embodiments, the first unit, the second unit, or both, are configured to log sensor data, intermediate fusion results, first outputs, second outputs, quality metric values, selection events, communication events, and operator-interface events. Such logged information may be used for post-operation review, training, audit, system refinement, Al model retraining, or reconstruction of an operational event.

[0103] In some embodiments, the selection module of the first unit is configured to evaluate the second output against the predefined quality metric and to select between the first output and the second output based on the evaluation. The selection module may therefore implement both an evaluation function and an output-selection function, such that the first unit is able to determine whether the second output is suitable for presentation to the deployed operator and, on that basis, select the first output, the second output, or a composition thereof as the situational information.

[0104] In some embodiments, the first unit is configured to select the first output as the situational information when the second output is unavailable, delayed, incomplete, invalid, or below a required quality threshold, and to select the second output as the situational information when the second output satisfies the predefined quality metric. In some embodiments, the first unit is configured to maintain availability of situational information to the deployed operator by locally generating the first output irrespective of whether the second output is received.

[0105] In some embodiments, the predefined quality metric comprises a composite metric, a multi-part metric, or a set of metric conditions. The predefined quality metric may comprise at least one of: a latency condition, a completeness condition, a confidence condition, a validity condition, a coverage condition, and an environmental suitability condition. In some embodiments, the second output is treated as failing the predefined quality metric when at least one required metric condition is not satisfied.

[0106] In some embodiments, the evaluation of the second output comprises assigning a quality value, quality score, validity value, or selection value to the second output, wherein the assigned value is reduced, nulled, or set to zero when the second output is not received within an evaluation window, is received in an error state, is incomplete, or otherwise fails to satisfy the predefined quality metric.In some embodiments, the first output comprises a locally generated thermal-annotated three-dimensional representation of at least a portion of the operational environment, and the second output comprises a remotely generated thermal-annotated three-dimensional representation of at least a portion of the operational environment. In some embodiments, the first output and the second output are generated from common or overlapping sensor observations but differ in at least one of processing complexity, contextual enrichment, classification content, prediction content, or spatial scope.

[0107] In some embodiments, the second unit is configured to transmit the second output to the first unit for evaluation and selection at the first unit. In some embodiments, the second unit is configured to return the second output as a candidate situational output for operator presentation, rather than merely transmitting update data, optimization data, or auxiliary metadata for modifying the first output.

[0108] In a further aspect, the present disclosure relates to a first unit for assisting a deployed operator in operational field conditions, the first unit comprising: a first communication module; a sensor module comprising at least an infrared sensor and a LiDAR sensor, each configured to generate sensor data; a first processing module configured to generate a first output based on a combination of sensor data provided by the infrared sensor and the LiDAR sensor; a first output interface configured to provide situational information to the deployed operator; and a selection module configured to select at least one of the first output or a second output received from a remote second unit as the situational information provided to the deployed operator.

[0109] In a further aspect, the present disclosure relates to a second unit operable remotely from a first unit used by or in proximity to a deployed operator, the second unit comprising: a second communication module configured to receive at least a portion of sensor data generated by a sensor module comprising at least an infrared sensor and a LiDAR sensor; and a second processing module configured to generate a second output based on the received sensor data, wherein the second output is configured to serve as a candidate situational output for selection at the first unit.

[0110] In some embodiments, the thermal-annotated three-dimensional data structure comprises a point cloud in which LiDAR-derived spatial points are associated with infrared-derived thermal attributes. In some embodiments, the thermal attributes comprise one or more of temperature values, temperature ranges, hotspot indicators, thermal gradients, thermal classes, or thermal confidence values associated with corresponding spatial points or spatial regions.In some embodiments, the system is configured for operation in environments in which communications quality may vary over time due to building attenuation, structural obstruction, smoke, terrain, distance, mobility, or infrastructure limitations. In such embodiments, the local generation of the first output and the selective use of the second output provide a communication-tolerant architecture in which situational information remains available to the deployed operator despite intermittent connectivity to the second unit.

[0111] The present disclosure further encompasses, separately or in combination, the following implementations and claimable forms.

[0112] In a further aspect, the present disclosure relates to a sensor module for use in an assis-tance system for a deployed operator, the sensor module comprising: at least an infrared sensor; at least a LiDAR sensor; and optionally a visible-light camera, wherein the sensor module is configured to generate multi-modal sensor data for generation of a thermal-annotated three-dimensional representation of an operational environment.

[0113] In a further aspect, the present disclosure relates to a method for operating an assistance system in operational field conditions, the method comprising: generating, at a first unit, a first output based on fused infrared sensor data and LiDAR sensor data; transmitting at least a portion of the sensor data to a second unit; generating, at the second unit, a second output based on the transmitted sensor data; evaluating, at the first unit, the second output against a predefined quality metric; selecting, using a selection module at the first unit, at least one of the first output or the second output as situational information; and presenting the situational information to a deployed operator.

[0114] In a further aspect, the present disclosure relates to a non-transitory computer-readable storage medium storing instructions which, when executed by one or more processors of a first unit, cause the first unit to: receive sensor data from at least an infrared sensor and a LiDAR sensor; generate a first output based on a combination of the sensor data; receive a second output generated remotely from at least a portion of the sensor data; evaluate the second output using a predefined quality metric; select at least one of the first output or the second output as situational information; and provide the situational information to a deployed operator.

[0115] In a further aspect, the present disclosure relates to a non-transitory computer-readable storage medium storing instructions which, when executed by one or more processors of a second unit, cause the second unit to: receive at least a portion of sensor data originating from at least an infrared sensor and a LiDAR sensor; generate a second output based on the received sensor data; and transmit the second output for selection at a first unit assituational information for a deployed operator.

[0116] In a further aspect, the present disclosure relates to a computer program product comprising instructions which, when executed by one or more processors, cause performance of any method, step, operation, evaluation, selection, fusion, or output-generation process disclosed herein.

[0117] In a further aspect, the present disclosure relates to a distributed assistance system comprising: a first unit as described herein; a second unit as described herein; and a wireless communication link between the first unit and the second unit, wherein the first unit is configured to maintain local situational-information generation independently of availabil-ity of the second unit and wherein the second unit is configured to generate a remotely processed candidate situational output for selection at the first unit.

[0118] Although the figures may show a sequence the order of the steps may differ from what is depicted. Also two or more steps may be performed concurrently or with partial concurrence. Such variation will depend on the software and hardware systems chosen and on designer choice. All such variations are within the scope of the disclosure. Likewise, software implementations could be accomplished with standard programming techniques with rule-based logic and other logic to accomplish the various connection steps, processing steps, comparison steps and decision steps. Additionally, even though the present disclosure has been described with reference to specific exemplifying embodiments thereof, many different alterations, modifications and the like will become apparent for those skilled in the art.

[0119] In addition, variations to the disclosed embodiments can be understood and effected by the skilled addressee in practicing the claimed present disclosure, from a study of the drawings, the disclosure, and the appended claims. Furthermore, in the claims, the word "comprising” does not exclude other elements or steps, and the indefinite article ”a” or ”an” does not exclude a plurality.

Claims

1. Asystem for assisting a deployed operator in operational field conditions, wherein the system comprises:a first unit, comprising:a first communication module,a sensor module comprising at least an infrared sensor and a LiDAR sensor, each of the sensors connected to the first unit and configured to generate sensor data,a first processing module configured to generate a first output based on a combination of the sensor data provided by each of the infrared and the LiDAR sensors of the sensor module, anda first output interface configured to provide situational information to the deployed operator, anda second unit operable remotely from the first unit, the second unit comprising:a second communication module arranged in wireless communication with the first communication module and configured to receive at least a portion of the sensor data generated at the sensor module,a second processing module configured to generate a second output based on the received sensor data, andwherein the first unit is configured to select at least one of the first output generated locally in the first unit or the second output generated remotely in the second unit as the situational information provided to the deployed operator.

2. The system according to claim 1 wherein the second output is selectable in place of, rather than for modifying, the first output.

3. The system according to claims 1 or 2, wherein the first unit is configured to select the first output as the situational information in response to a communication delay or failure between the first and second communication modules.

4. The system according to any one of claims 1 to 3, wherein the sensor module is physically separated from the first processing module and connected via a wired or wireless data link.

5. The system according to any one of the preceding claims, wherein the sensor module is mounted on a remotely operated vehicle or drone operating in proximity to the deployed operator.

6. The system according to any one of the preceding claims, wherein the sensor module further comprises a visible-light camera configured to generate additional sensor data.

7. The system according to any one of the preceding claims, wherein the first output interface is configured to provide at least one of image and audio-based situational information to the deployed operator.

8. The system according to any one of the preceding claims, wherein the first processing module is configured to apply a sensor fusion algorithm to combine the sensor data generated by the infrared sensor and the LiDAR sensor.

9. The system according to any one of the preceding claims, wherein the first processing module is configured to weight the sensor data based on one or more environmental conditions detected during operation.

10. The system according to any one of the preceding claims, wherein the first processing module is configured to generate a hybrid spatial representation of the operational environment based on the combined sensor data.

11. The system according to any one of the preceding claims, wherein the second processing module comprises an Al model configured to classify environmental risks based on the received sensor data.

12. The system according to any one of the preceding claims, wherein the Al model is configured to adapt its analysis based on prior operational scenarios or training data.

13. The system according to any one of the preceding claims, wherein the first unit is configured to prioritize the second output over the first output when the second output comprises data not available to the first processing module.

14. The system according to any one of the preceding claims, wherein the second processing module is configured to process a filtered subset of the received sensor data based on bandwidth constraints or relevance scoring.

15. The system according to any one of the preceding claims, wherein the first processing module is configured to generate a real-time three-dimensional map of the operational environment based on the combined sensor data.

16. The system according to any one of the preceding claims, wherein the second processing module is further configured to incorporate external data sources for contextual interpretation of the received sensor data.

17. The system according to any one of the preceding claims, wherein the first output interface is configured to allow the deployed operator to select a data view based on one or more sensor data types.

18. The system according to any one of the preceding claims, wherein the first processing module is configured to compare current sensor data to previously acquired sensor data to detect environmental changes over time.

19. The system according to any one of the preceding claims, wherein the second unit is configured to log received sensor data and associated outputs for post-operation review.

20. The system according to any one of the preceding claims, wherein the first processing module is configured to operate in different processing modes selected based on a type of operational scenario.

21. The system according to any one of the preceding claims, wherein the second unit is configured to transmit situational information to an external command system for coordination of multiple deployed operators.

22. The system according to any one of claims 1 to 21wherein the first unit comprises a selection module configured to receive the first out- put and the second output as distinct candidate situational outputs and to select one of the candidate situational outputs for presentation to the deployed operator based on a quality metric indicative of at least one of latency, completeness, or confidence.

23. A method for operating an assistance system in operational field conditions, wherein the system comprises:a first unit arranged in a vicinity of deployed operator, the first unit comprising a first processing module and a sensor module, wherein the sensor module comprises at least an infrared sensor and a LiDAR sensor, and wherein each of the sensors is connected to the first unit and configured to generate sensor data, a second unit operable remotely from the first unit, the second unit comprising a second processing module configured to generate an output based on the sensor data from the sensor module,wherein the method comprises the steps of:initiating, by the first unit, transmission of at least a portion of the sensor data from the sensor module to the second unit via a wireless communication link, generating, by the first processing module, a first output based on the infrared and the LiDAR sensor data generated at the sensor module,evaluating, at the first unit, whether the output from the second unit satisfies a predefined quality metric,selecting, at the first unit, at least one of the first output generated locally inthe first unit and the second output generated remotely in the second unit as situational information based on the evaluation, andproviding, by the first unit, the situational information to the deployed operator via an output interface.

Images