Prediction of evolutionary scenarios for an accumulation phenomenon

Abstract

Software for the analysis of accumulation phenomena (1A) in an environment to be monitored (10); the software for the analysis of accumulation phenomena (1A) being storable in and executable by electronic processing resources (2), and designed to cause, when executed, these electronic processing resources (2) to become configured to receive a time sequence of representations of the environment to be monitored (10); where each representation of this time sequence represents the environment to be monitored (10) at a different time instant. In addition, the software for the analysis of accumulation phenomena (1A) is designed to cause, when executed, these electronic processing resources (2) to become configured to implement a segmentation model (6) configured to locate, or segment, one or more target areas occupied by one or more substances deposited on, or covering, a surface locatable in an input representation. The software for the analysis of accumulation phenomena (1A) is designed to cause, when executed, these electronic processing resources (2) to become configured to compute (block 8) an evolutionary scenario of a deposition, accumulation, or coverage phenomenon, based on the time sequence of representations of the environment to be monitored (10) and based on the segmentation model (6) implemented. The evolutionary scenario of the phenomenon is a time sequence of representations, of different future time instants compared to the current time instant, representing one or more substances deposited on, or covering, a surface present in an environment to be monitored (10). In addition, the accumulation phenomena analysis software (1A) is designed so that, when executed, these electronic processing resources (2) become configured to provide an output (9) based on this evolutionary scenario of the phenomenon.

Description

[0001] PREDICTION OF EVOLUTIONARY SCENARIOS FOR AN ACCUMULATION PHENOMENON

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This Patent Application claims priority from Italian Patent Applications No. 102024000021077 and No. 102024000021079 filed on September 20, 2024, the entire disclosure of which is incorporated herein by reference.

[0004] TECHNICAL FIELD OF THE INVENTION

[0005] This invention relates in general to a computer system for analysing and extracting information from an environment to be monitored.

[0006] In particular, this invention relates to a computer system for determining information indicating the presence of a deposition, accumulation, or coverage phenomenon, of one or more substances in the environment to be monitored.

[0007] PRIOR ART

[0008] AS is known, the monitoring of an area or an environment is essential for assessing and understanding the presence and evolution of phenomena of various kinds.

[0009] Monitoring allows to identify and assess potential risks in the environment to be monitored. Knowing the presence of potentially harmful phenomena allows for the timely adoption of preventive and mitigation measures.

[0010] Through monitoring, targeted interventions can be implemented to reduce or eliminate the risks identified.

[0011] It is also known that it is essential to employ a continuous monitoring system in order to determine or detect any deposits, or accumulations, of a substance (for example, water) in real time.

[0012] The risks of a substance being deposited or accumulated may include the obstruction of a passageway, air or water pollution, or landslides.

[0013] It is known that a continuous monitoring system can include level sensors, early warning systems and surveillance cameras. Specifically, these level sensors can be installed at various strategic points to detect changes in the level of the accumulated or deposited substance. The level sensors can be based on different principles, such as buoyancy, hydrostatic pressure or the use of sonar.

[0014] In addition, warning systems have been known to be used to promptly alert competent authorities, and responsible personnel, based on what is measured by the sensors. Specifically, the warning systems are configured to transmit notifications via SMS, email or audible alarms.

[0015] It is well known that a scientific article (WINDHEUSER L. ET AL: "An End-To- End Flood Stage Prediction System Using Deep Neural Networks", EARTH AND SPACE SCIENCE, vol. 10, no. 1, 1 January 2023) describes an automated system for river flood level prediction, based on deep neural network processing of images acquired from river webcams and time data. The system described employs a U-Net for image segmentation and LSTM models for multi-hourly water level prediction.

[0016] It is also known that the scientific article by MOY DE VITRY et al., entitled "Scalable flood level trend monitoring with surveillance cameras using a deep convolutional neural network" (Hydrology and Earth System Sciences, vol. 23, n. 11, 15 November 2019) proposes a scalable approach for qualitative monitoring of urban flood level changes using automatic video analysis from surveillance cameras. Using a deep convolutional neural network to segment flooded areas and compute an index called SOFI (Static Observer Flooding Index), the described system succeeds in estimating visible water level fluctuations with good correlation to real data, without the need for specific camera calibration.

[0017] SUBJE T AND SUMMARY OF THE INVENTION

[0018] The Applicant has been able to observe that the known solutions could be improved.

[0019] The purpose of this invention is, therefore, to provide software for the analysis of accumulation phenomena in an environment to be monitored that allows the solutions of the prior art to be at least partially improved. In particular, this invention makes it possible to locate the position, not just the presence, of one or more substances deposited on, or covering, a surface in the environment to be monitored.

[0020] According to this invention, software for the analysis of accumulation phenomena is provided as claimed in the appended claims.

[0021] BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 shows a block diagram of a system for analysing accumulation phenomena according to a preferred embodiment of the present invention.

[0023] Figure 2 shows a block diagram of a system for analysing accumulation phenomena according to an additional embodiment of the present invention.

[0024] Figures 3, 4, 5, 6, 7 show examples of representations of environments to be monitored and possible areas of interest to be monitored according to the present invention.

[0025] DESCRIPTION OF PREFERRED EMBODIMENTS OF THE INVENTION

[0026] The present invention will now be described in detail with reference to the attached figures in order to allow a skilled person to implement it and use it. Various modifications to the described embodiments will be readily apparent to those skilled in the art and the general principles described may be applied to other embodiments and applications without however departing from the protective scope of the present invention as defined in the attached claims. Therefore, the present invention should not be regarded as limited to the embodiments described and illustrated herein but should be allowed the broadest protection scope consistent with the features described and claimed herein.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning commonly understood by one of ordinary skill in the art to which the invention belongs. In case of conflict, the present specification, including the definitions provided, will control. In addition, the examples are provided purely for illustrative purposes and, as such, must not be considered as limiting.

[0028] In particular, the block diagrams included in the attached figures and described below are not to be understood as a representation of the structural features, i.e. construction restrictions, but must be understood as a representation of functional features, i.e. intrinsic properties of the devices defined by the effects obtained, that is to say functional restrictions, which can be implemented in different ways, so as to protect the functionalities thereof (operational capability).

[0029] In order to facilitate the understanding of the embodiments described herein, reference will be made to some specific embodiments and a specific language will be used to describe the same. The terminology used herein is used for the purpose of describing particular embodiments only and is not intended to limit the scope of this invention.

[0030] It should be noted that, from hereafter, the term deposition conveniently refers to the process in which substances or materials settle, or are placed, in an environment, whether natural or man-made, to be monitored 10.

[0031] It should be noted that, from hereafter, the term accumulation conveniently refers to the gradual concentration of substances or materials in an environment to be monitored 10. Conveniently, accumulation refers to the process in which a substance accumulates or is deposited on a surface without necessarily coating it completely.

[0032] It should be noted that, from hereafter, the term covering conveniently refers to the fact that a substance has occupied, or spread over, at least part of the surface; optionally, all the available space on the surface. By way of example, if the substance were a liquid, this covering could be a flooding event.

[0033] Specifically, the deposition, accumulation and covering can cover a wide range of substances, including liquids, foams and granular materials; for example, water, oils, paints, snow, sand, mud, leaves, debris and more.

[0034] Figure 1 shows a block diagram of a system for analysing accumulation phenomena 1 in an environment to be monitored 10, or that one wishes to monitor, according to a preferred embodiment of the present invention.

[0035] The system for analysing accumulation phenomena 1 in an environment to be monitored 10 comprises electronic processing resources 2 storing, and configured to execute, a software for the analysis of accumulation phenomena 1A to perform an analysis of the environment to be monitored 10 in order to determine the presence of deposition, accumulation, or coverage phenomenon (that is, whether such is occurring).

[0036] In addition, the system for analysing accumulation phenomena 1 comprises one or more sensors 3 designed to capture, and transmit, one or more representations, or images, of the environment to be monitored 10 at different time instants. By way of non-limiting example, these sensors 3 comprise one or more of the following: a camera, a video camera, a Charge-Coupled Device (CCD) sensor, or a Complementary Metal-Oxide- Semiconductor (CMOS) sensor.

[0037] The software, or computer product, for the analysis of accumulation phenomena is storable in and executable by the electronic processing resources 2 and is designed to cause, when executed, said electronic processing resources 2 to become configured to perform an analysis of the environment to be monitored 10 so as to determine the presence of, and possibly further information about, a deposition, accumulation, or coverage phenomenon.

[0038] With regard to the analysis of accumulation phenomena in the environment to be monitored 10, and the functionalities implemented by the software for the analysis of accumulation phenomena 1A, it should be emphasised that what matters are the operations that must be implemented to realise this functionality and not the hardware and software architectures with which these operations are implemented. In fact, these operations may be implemented by means of a concentrated architecture, that is, by a single electronic device (by way of example, by a single computer), or by means of a distributed cooperative architecture, that is, distributed among several electronic devices in communication and cooperating with each other according to a proprietary logical architecture that the software for the analysis of accumulation phenomena 1A decides to adopt.

[0039] The electronic processing resources 2 are configured to receive, specifically from the sensors 3 of the system for analysing accumulation phenomena 1, a time sequence of representations of the environment to be monitored 10; wherein each representation of this time sequence represents the environment to be monitored 10 at a different time instant. Specifically, the time sequence of representations of the environment to be monitored 10 represents, and is indicative of, a change over time of the environment to be monitored 10.

[0040] Conveniently, each representation of this time sequence is an image representing that environment to be monitored 10. In particular, the time sequence of representations of the environment to be monitored 10 comprises several images captured, or taken, at regular intervals. Optionally, these representations of the time sequence of representations of the environment to be monitored 10 are not necessarily generated or captured exclusively in the visible spectrum domain; for example, by one or more of the following technologies: infrared (IR), thermal, ultraviolet (UV), radar or other similar means.

[0041] Figures 3, 4, 5, 6, 7 show examples of representations of environments to be monitored according to this invention. By way of example, the environment to be monitored 10 could be a road bordering landslide-prone terrain (as depicted in Figure 3), a harbour quay (as in the images in Figure 4), an industrial work area (as in the images in Figure 5), a desert area (as in the images in Figure 6), or a depressed road, for example a subway (as in Figure 7). Specifically, one might want to monitor a road bordering landslide-prone terrain to predict a possible landslide; in another example, one might want to monitor a waterfront, or a subway, or an industrial work area to detect their potential flooding. Specifically, the representations of the environment to be monitored 10, of said sequence, have at least one shared or overlapping region. In other words, despite the fact that the representations are associated with different time instants, a part of the scene, or of the environment 10, represented remains unchanged. The time sequence of representations of the environment to be monitored 10 preferably has a fixed or static framing (position, angle and composition of the image remain constant for each time instant represented in the sequence) in order to ensure visual continuity between the different images.

[0042] According to one aspect of this invention, the electronic processing resources 2 are also configured to determine whether the framing is fixed or movable based on the time sequence of representations of the environment to be monitored 10 as input. Specifically, the electronic 2 processing resources are configured to perform a comparison between different representations of the sequence, conveniently after processing them according to a predefined data processing technique, in order to determine whether the framing is fixed or moving. By way of example, the electronic processing resources 2 are configured to determine whether the framing is fixed or moving by performing background subtraction techniques on the input time sequence. The electronic processing resources 2 are conveniently configured to output information indicating whether the framing is fixed or moving. Specifically, the electronic processing resources 2 are configured to process representations based on information indicating whether the framing is fixed or moving. More specifically, the electronic processing resources 2 are configured to process the representations, to determine information indicating the presence of the phenomenon, only in the event that the framing is determined to be fixed.

[0043] The electronic processing resources 2 are conveniently configured to process the representations, or images, of the time sequence of representations of the environment to be monitored 10 according to an edge analysis technique. In particular, the electronic processing resources 2 are configured to compute, based on the time sequence of representations of the environment to be monitored 10, representations or images of edges of one or several elements represented in that sequence.

[0044] Specifically, the electronic processing resources 2 are designed to process the time sequence representations of the environment to be monitored 10 by modifying these representations so that the identified edges of the elements present in the latter become evident. By way of non-limiting example, the electronic processing resources 2 are configured to modify the time sequence representations of the environment to be monitored 10, overlapping the representations of the edges of the elements represented in that sequence over them.

[0045] The electronic processing resources 2 are preferably configured to compute, based on the time sequence of representations of the environment to be monitored 10, one or more depth maps indicating information relating to the geometry and / or topography of surfaces and elements in the environment to be monitored 10; wherein, in detail, the depth maps are three-dimensional representations of surfaces and elements located in said environment to be monitored 10, indicating the distance between the sensors 3 and the elements, and surfaces, in the environment to be monitored 10. In particular, the electronic processing resources 2 are also configured to process these representations of the environment to be monitored 10 using a depth map generation technique. In particular, depth maps can be generated using different technologies, for example artificial vision technologies. Optionally, the electronic processing resources 2 are configured to modify, based on the computed depth maps, the representations of the environment to be monitored 10 so that they become representative of information relating to the geometry and topography of surfaces and elements of the environment to be monitored 10.

[0046] Specifically, the electronic processing resources 2 are configured to detect or recognise a number of patterns occurring in the time sequence representations of the environment to be monitored 10 to determine the presence of a deposition, accumulation, or coverage phenomenon. Specifically, the electronic processing resources 2 are configured to recognise such patterns by means of a technique for analysing the dynamics of alterations detectable among successive images of the sequence received. More specifically, the electronic processing resources 2 are configured to determine the presence of the deposition, accumulation, or coverage phenomenon based on one or more patterns identified in the conveniently processed representations of the environment to be monitored 10.

[0047] More specifically, the electronic processing resources 2 are configured to recognise one or more, conveniently all, the following patterns (and possibly other different patterns): sub-horizontal edges that alter their position over time, areas bordered by closed edges whose extent varies over time, localised alterations (reductions) in depth maps, areas of stasis with increasing size. More specifically, the electronic processing resources 2 are configured to identify a number of sub-horizontal edges that alter their position, or shape, in the different representations of the time sequence of representations of the environment to be monitored 10; more specifically, in the representations of the edges of the different elements represented in that sequence.

[0048] In addition, the electronic processing resources 2 are conveniently configured to identify one or more areas, delimited by closed edges, the extent of which varies in the time sequence representations of the environment to be monitored 10 specifically by working out a difference between the different areas, delimited by closed edges, of various time sequence representations. In particular, wherein, said areas delimited by closed edges are recurrent, or remain, in the same position for different representations, or for edge representations (if computed), of this time sequence.

[0049] Specifically, the electronic processing resources 2 are configured to additionally detect localised alterations, or reductions, in the depth maps computed based on the time sequence of representations of the environment to be monitored 10.

[0050] The electronic processing resources 2 are also conveniently configured to locate portions of an image, or a representation, of the time sequence characterised by concurrent optical flows associated with areas of stasis of increasing size. By way of example, the stasis areas of increasing size can be identified by observing a number of regions in which optical flows have little or no variation over time, indicating no movement or very little movement; these areas could correspond to static objects or regions in which movement is blocked. In particular, the electronic processing resources 2 are also configured to locate these portions characterised by concurrent optical flows by means of optical flow analysis algorithms that can be applied to estimate the direction and speed of pixel movement within images, for example, the Lucas-Kanade algorithm or dense optics.

[0051] The electronic processing resources 2 are configured to implement a segmentation model 6 configured to locate, or segment, one or more target areas occupied by one or more substances deposited on, or covering, a surface locatable in an input representation. Specifically, this segmentation model 6 is configured to identify the area, conveniently also the location, of several substances present in the environment to be monitored 10 and of one or several surfaces present in that environment to be monitored 10. More specifically, this segmentation model 6 is configured to identify the areas of one or more surfaces present in a representation of the environment to be monitored 10, and to identify the areas of one or several substances present on these surfaces.

[0052] According to one aspect of this invention, this segmentation model 6 is a neural model comprising at least one segmentation neural network. By way of non-limiting example, a neural network comprising an encoder layer and a decoder layer (for example, a U-net neural network).

[0053] According to the preferred embodiment of the present invention, the segmentation model 6 is configured to perform the location or segmentation of substances present on, or deposited on or covering, a surface regardless of the type of such substances. In addition, the segmentation model 6 is preferably configured to perform the location or segmentation of substances present on, or deposited on or covering, a surface regardless of the shape of this surface.

[0054] In particular, the segmentation model 6 is a neural model trained on a heterogeneous dataset in terms of the type of substances to be detected, specifically to be agnostic to the type of substances to be detected. More specifically, this heterogeneous dataset comprises a plurality of training representations or images, and their labels, representing different liquids, foams and granular materials. Each training representation represents one or more of the following substances: water, oils, paints, snow, sand, mud, leaves and debris.

[0055] In particular, the segmentation model 6 is also trained on a heterogeneous dataset in terms of the shape of surfaces to be detected, specifically to be agnostic to the shape of the surfaces to be detected and on which one or more substances to be detected are deposited or accumulate.

[0056] In particular, the electronic processing resources 2 are configured to train, or to receive as input, the segmentation model 6.

[0057] According to an aspect of this invention, the electronic processing resources 2 are configured to compute, or generate, a time sequence of representations of target areas occupied by one or more substances deposited on, or covering, a surface of the environment to be monitored 10 based on the time sequence of representations of the environment to be monitored 10 and based on the implemented segmentation model 6. Specifically, the electronic processing resources 2 are designed to segment (block 6) or identify one or several target areas, occupied by at least one substance deposited on or covering a surface, by making an inference about the time sequence of representations of the environment to be monitored 10 by means of the segmentation model 6 implemented. Specifically, the target areas are areas present in the time sequence representations of the environment to be monitored 10.

[0058] According to one aspect of this invention, the segmentation model 6 is also configured to detect information relating to the geometry and topography of the substances; and it is also configured to determine the distance between such substances and at least one of the sensors 3; for example, by means of one or more of the depth maps. Specifically, the representations of the target areas are also representative and indicative of the distance between them and the sensors 3.

[0059] In detail, the electronic processing resources 2 are configured to compute the time sequence of representations of target areas occupied by one or more substances deposited on, or covering, a surface of the environment to be monitored 10 based on the computed depth maps and based on the implemented segmentation model 6. Specifically, the electronic processing resources 2 are designed to segment (block 6) or locate several target areas, occupied by at least one substance deposited on or covering a surface, by performing an inference about the depth maps computed by means of the segmentation model 6 implemented. More in detail, the electronic processing resources 2 are configured to determine a time sequence of depth maps based on the different depth maps computed and to perform an inference (in order to identify target areas) about this sequence of depth maps by means of the segmentation model 6 implemented.

[0060] According to one aspect of this invention, the electronic processing resources 2 are designed to segment (block 6) or locate different target areas, occupied by at least one substance deposited on or covering a surface, based on the computed depth maps, the representations of the environment to be monitored 10 and based on the segmentation model 6 implemented; in particular, by performing an inference on the computed depth maps in combination with the representations of the environment to be monitored 10 by means of the implemented segmentation model 6.

[0061] In particular, the electronic processing resources 2 are configured to compute an index of the presence of the phenomenon, that is, indicative of the presence of the phenomenon, based on the identified target areas. In detail, the electronic processing resources 2 are configured to compute the index of the occurrence of the phenomenon based on a comparison of different identified target areas of different time instants; more specifically, based on a rate of occurrence of the phenomenon. By way of example, the electronic processing resources 2 might be able to locate the phenomenon when a target area expands over time.

[0062] Figure 2 shows a block diagram of a system for analysing accumulation phenomena 1 according to another embodiment of the present invention, optionally independent of other embodiments of this invention (including the preferred embodiment of the present invention).

[0063] According to said further embodiment of this invention, optionally independent of other embodiments of this invention, the electronic processing resources 2 are further configured to identify (block 5), in the time sequence of representations of the environment to be monitored 10, one or several areas, or regions, of interest 11 in which the phenomenon of deposition, accumulation or coverage might occur. In particular, the electronic processing resources 2 are configured to delimit or segment (block 5) within an image, or representation, of the environment to be monitored 10 one or more areas of interest 11 in which to focus the detection of the phenomenon.

[0064] Figures 3, 4, 5, 6, 7 also show possible areas of interest 11 for the environments to be monitored according to this invention.

[0065] By way of example, these areas of interest 11 may be one or more of the following: a road surface, a pavement, a riverbank, a depression in the ground, a harbour quay, a railway embankment, a floor of an industrial plant, or another type of surface.

[0066] In particular, the electronic processing resources 2 are configured to implement a context identification model 4 configured to determine, or identify, one or more context features of an environment to be monitored 10; wherein, such context features identify the context of that environment 10. Specifically, this context identification model 4 is configured to perform a characterisation of the context, identifiable in one or more representations of the environment to be monitored 10, in order to identify (block 5) one or several areas of interest 11 in that environment 10. In particular, the context identification model 4 can recognise whether the input representation represents an urban / natural environment, whether it is a road, a subway or a square, whether there is a river, a coastline, a harbour or another type of environment. More specifically, this context identification model 4 is trained to segment the input representation, for example using an object detection technique, by identifying the location and presence of one or several elements indicative of the context of the environment to be monitored 10. The information obtained from segmentation can be used to better understand the context of the image, for example by identifying the objects present and the spatial relationships between them.

[0067] More specifically, this context identification model 4 is a neural model, comprising at least one neural network, trained to perform a characterisation of the context identifiable in one or several input representations. By way of non-limiting example, this context identification model 4 comprises one or more Capsule Networks and / or Convolutional Neural Networks (CNN). In particular, the electronic processing resources 2 are configured to train, or to receive as input, the context identification model 4.

[0068] Alternatively, the context identification model 4 is a different artificial vision model, specifically one that does not comprise neural networks and is configured to employ one or more artificial vision techniques. Optionally, this artificial vision model is configured to collaborate, or combine, with a trained neural network to perform contextual characterisation.

[0069] The context identification model 4 is preferably configured to determine the context of several areas represented, even if they are not in the visible spectrum.

[0070] As an example, the context identification model 4 is configured to employ one or more techniques for analysing the relative sizes and distances between identified objects. Conveniently, such size and distance analysis techniques could make use of reconstructions of image depth maps through perspective analysis of the scene in which a number of recognised objects are sized and positioned in depth due to knowledge of multiple elements. These elements include the characteristic size of an identified object (for example, the average height of a street lamp post), the focal length of the camera, its installation height and its inclination with respect to the horizon.

[0071] According to an optional aspect of this invention, the context identification model 4 is further configured to recognise and classify the objects present in the input representation so as to determine, or compute, the context features of an environment to be monitored 10 also based on the recognised classes.

[0072] The context identification model 4 is also conveniently configured to determine or compute a number of data points indicating spatial relationships between objects in the input representation, and the context comprises these data points. Optionally, the context identification model 4 is also trained to classify the context type of an environment 10 represented in one or several input representations, for example, in the time sequence of representations of the environment to be monitored 10. This context identification model 4 is preferably trained on several images of different time instants to analyse the images over time to strengthen the model's ability to determine the context of an environment to be monitored 10 represented in a time sequence of representations.

[0073] In addition, the electronic processing resources 2 are configured to locate (block 5), in the time sequence of representations of the environment to be monitored 10, the areas of interest 11 where the phenomenon might occur based on the context identification model 4, and possibly based on an input indicative of a type of area of interest 11 to be monitored.

[0074] In particular, the electronic processing resources 2 are configured to locate (block 5), in the time sequence of representations of the environment to be monitored 10, the areas of interest 11 in which the phenomenon might occur based on, or in relation to, one or more features, identified through the context identification model 4, identifying the context of the environment to be monitored 10.

[0075] According to one aspect of this invention, the electronic processing resources 2 are configured to locate one or more areas by segmenting one or more time sequence representations of the environment to be monitored 10. In addition, said electronic processing resources 2 are configured to select (block 5), as areas of interest 11, one or more of said areas identified based on the identifying features of the context of the environment to be monitored 10.

[0076] In particular, the electronic processing resources 2 are configured to classify, or extract textual information from, the identified areas and to select (block 5) the areas of interest 11, from among such identified areas, based on a comparison or mapping between such classes, or textual information, of the identified areas and the identifying features of the context of the environment to be monitored 10. By way of example, such textual information is descriptive of the context of the environment to be monitored 10.

[0077] By way of non-limiting example, if the context of the environment to be monitored 10 is a port, an area of interest 11 where the phenomenon might occur could be a port quay. According to one aspect of this invention, the electronic processing resources 2 are configured to locate (block 5), in the time sequence of representations of the environment to be monitored 10, the areas of interest 11 where the phenomenon might occur based on an input indicative of the type of one or more areas of interest 11 to be monitored. In particular, such input is either textual content or received speech content; conveniently, if it is speech, the electronic processing resources 2 are also configured to compute a corresponding textual content from the received speech content.

[0078] Specifically, this input, received from a user's electronic device (to determine the type of areas of interest 11 to be monitored), describes one or more types of areas of interest 11 to be defined or describes such areas of interest 11. By way of non -limiting example, such an input could be one of the following textual contents: “all streets”, or “the pavements on the right of the image”. By way of non-limiting example, the input corresponds with the type of area of interest 11 that you want to monitor.

[0079] Specifically, the electronic processing resources 2 are configured to perform a textual analysis, possibly by means of a language model (for example, a neural model such as an LLM, a Large Language Model), of the input received in order to identify the type (that is, an identifying textual content of that type) of one or more areas of interest 11 that the user wishes to monitor. Conveniently, such a linguistic model is a multilingual model configured, or trained, to perform textual analyses and understand textual content written in different languages in order to process information from different linguistic content. The electronic processing resources 2 are also configured to locate (block 5), in the time sequence of representations of the environment to be monitored 10, the areas of interest 11 where the phenomenon might occur based on the textual analysis carried out, or based on the types of areas of interest 11 determined. Conveniently, the language model is trained to determine, or recognise, the presence of multiple technical terms identifying areas (for example roads, pavement, plant, platform, field) and position information and spatial relationships between objects (for example, one object is “to the right” of another, “next to” another, “above” or “below” another, and so on).

[0080] More specifically, the electronic processing resources 2 are also configured to classify, using the (conveniently neural) language model, an input by type of area of interest 11. More specifically, the electronic processing resources 2 are also configured to identify (block 5) areas of interest 11 based on one or more classes of type of area of interest 11 determined to be associated with the input received. More specifically, the electronic processing resources 2 are also configured to determine the position and shape of an area, of the input representation, classified to be of a specific type of area of interest 11 (and one that can be determined based on the input received).

[0081] In addition, the electronic processing resources 2 are conveniently configured to locate, by means of the segmentation model 6, one or more target areas occupied by substances deposited on, or covering, a surface present in a located area of interest 11 where such phenomenon might occur and determine the presence of the phenomenon based on the located target areas.

[0082] More specifically, for each area of interest 11 located, the electronic processing resources 2 are configured to locate the target areas occupied by substances in that area of interest 11

[0083] According to one preferred embodiment of this invention, the electronic processing resources 2 are also configured to compute (block 8) an evolutionary scenario of a deposition, accumulation, or coverage phenomenon, based on the time sequence of representations of the environment to be monitored 10 and based on the segmentation model 6 implemented. In particular, the evolutionary scenario of the phenomenon is a time sequence of representations, of different future time instants compared to the current time instant, representing one or more substances deposited on, or covering, a surface present in an environment to be monitored 10. Specifically, the evolutionary scenario of the phenomenon is indicative and representative of how the phenomenon evolves, that is, what will happen in one or more future time instants, starting from the time instant associated with the last representation of the time sequence of representations of the environment to be monitored 10. More specifically, the evolutionary scenario of the phenomenon is a time sequence of representations of a predefined number of future time instants.

[0084] According to one aspect of this invention, the electronic processing resources 2 are configured to implement a prediction model 8 configured to compute (block 8) a future evolutionary scenario of a deposition, accumulation or coverage phenomenon, based on the time sequence of representations, of past time instants, as input. Specifically, such a prediction model 8 is a neural model, that is, a neural prediction network trained with series or sequences of evolutions of different types, specifically of accumulation or coverage phenomena, to infer the future evolutionary scenario. Specifically, the prediction model 8 is a generative or evolutionary model trained to be able to infer (block 8) future evolutionary scenarios on the basis of time sequences of representations of past time instants. More specifically, the electronic processing resources 2 are configured to compute (block 8) using the prediction model 8, the future evolutionary scenario of a deposition, accumulation or coverage phenomenon so that this future evolutionary scenario comprises one or more representations (for example, images) of a predicted future scenario.

[0085] In particular, the prediction model 8 is a neural model trained on a heterogeneous dataset in terms of the type of substances to be detected, specifically to be agnostic to the type of substances to be detected. More specifically, this heterogeneous dataset comprises multiple sequences of training representations or images representing different liquids, foams and granular materials. Each training representation represents one or several of the following substances: water, oils, paints, snow, sand, mud, leaves and debris.

[0086] By way of illustration, the prediction model 8 is a GANN LSTM (Generative Adversarial Neural Network Long Short-Term Memory) trained with time sequences of images of the phenomena of interest 11. A first time period of each sequence (that is, a number of first images of the sequence) may correspond to an example received as input and a second period, that is, the final period (a number of last images of the sequence), may correspond to an output required in the training. By way of non-limiting example, the prediction model 8 is trained on stable diffusion at a time series of regions of interest 11 (ROI).

[0087] In particular, the electronic processing resources 2 are configured to compute (block 8) the evolutionary scenario of the phenomenon based on the time sequence of the representations of the target areas. In particular, the electronic processing resources 2 are configured to compute (block 8) the evolutionary scenario of the phenomenon based on the time sequence of the representations of the target areas and based on the prediction model 8; more specifically, by performing an inference with the prediction model 8 about the time sequence of the representations of the target areas.

[0088] Specifically, the electronic processing resources 2 are also configured to compute (block 8) the evolutionary scenario of the phenomenon based on a comparison (block 7), or difference, made between target areas of different representations of the time sequence of the representations of the target areas. More specifically, the electronic processing resources 2 are configured to compute (block 7) a time sequence of differences comprising several representations of the differences between these target areas over time, that is, among the several representations of the target areas. In particular, the representations of the differences comprise, or represent, areas or regions of interest 11 computed by performing a difference between representations of the time sequence representations of the target areas. In addition, the electronic processing resources 2 are configured to compute, or predict, (block 8) the evolutionary scenario of the phenomenon based on the computed time sequence of differences as the latter is indicative of the change over time of the phenomenon analysed.

[0089] According to one aspect of this invention, the electronic processing resources 2 are configured to compute a rate of occurrence of the phenomenon based on the time sequence of representations of the environment to be monitored 10; and to compute (block 8) the evolutionary scenario of the phenomenon also based on the rate of occurrence of the phenomenon. In detail, the electronic processing resources 2 are configured to compute the rate of occurrence of the phenomenon based on the time sequence of representations of the target areas and, more in detail, based on the computed time sequence of differences.

[0090] In particular, the electronic processing resources 2 are configured to compute the rate of occurrence of the phenomenon by evaluating or computing how much the size of the target areas varies over time; wherein, the time can be computed by considering the time instants associated with the different representations of this time sequence. In detail, the electronic processing resources 2 are configured to compute the rate of occurrence of the phenomenon based on the computed time sequence of differences and based on the time instants associated with each representation of the time sequence. More specifically, the electronic processing resources 2 are configured to compute the rate of occurrence of the phenomenon by employing an algorithm to compute the area of difference between a first representation of a target area and a second representation associated with a first time instant and a second time instant, respectively.

[0091] According to one aspect of this invention, the electronic processing resources 2 are configured to provide an output 9, that is, a message or notification, according to this evolutionary scenario of the phenomenon. In particular, the electronic processing resources 2 are configured to transmit to one or several electronic monitoring devices either this evolutionary scenario of the phenomenon or an index of the severity of the phenomenon computed based on the evolutionary scenario of the phenomenon.

[0092] According to an aspect of this invention, the electronic processing resources 2 are configured to compute a severity index of the phenomenon based on the evolutionary scenario of the phenomenon and based on one or several, conveniently all, context characteristics of the environment to be monitored 10 identified by means of the context identification model 4 and to provide an output 9 based on the severity index of the phenomenon. By way of example, the electronic processing resources 2 are configured to compute a phenomenon severity index by performing a weighted average of several computed factors; wherein, one of these factors is computed based on the evolutionary scenario of the phenomenon. Conveniently, the electronic processing resources 2 are also designed to compute the phenomenon severity index based on the rate of occurrence of the phenomenon, specifically the rate of occurrence of the phenomenon is one of the factors for computing the phenomenon severity index.

[0093] Specifically, the electronic processing resources 2 are conveniently configured to compute the severity index using a classifier model (for example, an artificial neural network) wherein such a classifier model is configured to receive as input features of the context and / or the rate of occurrence of the phenomenon and possibly the type of medium identified. It also computes and outputs the severity index (for example, continuous from 0 to 1 or in terms of a number of classes) having been trained with analogous examples associated with predefined severity indices (that is, the labels of such examples).

[0094] According to a different aspect of this invention, the electronic processing resources 2 are configured to provide an output 9 based on the index of the presence of the phenomenon computed according to the target areas identified. In particular, the electronic processing resources 2 are configured to compute a severity index of the phenomenon based on the index of the presence of the phenomenon and based on one or more context features of the environment to be monitored 10 identified by means of the context identification model 4.

[0095] Optionally, the electronic processing resources 2 are also configured to estimate the extension of the phenomenon and / or the depth of the accumulation based on the evolutionary scenario of the phenomenon and to compute the severity index of the phenomenon also based on these estimates of the extension and depth of the accumulation.

[0096] Said electronic processing resources 2 are also conveniently configured to identify the type of one or several substances deposited on, or covering, the surface of the environment to be monitored 10 and, optionally, are configured to compute the severity index of the phenomenon including based on the type of these substances. Specifically, the electronic processing resources 2 are designed to identify the type of such substances by means of an object detection technique or a classification technique.

[0097] Specifically, the electronic processing resources 2 are configured to implement a substance classification model configured, or trained, to classify substances deposited on, or covering, the surface of the environment to be monitored 10.

[0098] More specifically, this substance classification model is a classification neural network, for example a Convolutional Neural Network (CNN), trained on several images of such substances and on related textual labels indicative of the class associated with the substances in these images. Specifically, the electronic processing resources 2 are configured to train this substance classification model. This substance classification model is conveniently configured to classify the generic type of substance (for example, liquid, foam or granular material) and / or to classify the specific type of substance (for example, petrol, water, snow, sand, salt, etc.).

[0099] In particular, the electronic processing resources 2 are configured to transmit a message or notification 9, for example the evolutionary scenario of the phenomenon and / or the phenomenon severity index, based on the fulfilment of a notification condition, to these electronic monitoring devices, specifically whether the notification condition is determined to be satisfied. Specifically, the notification condition is satisfied when the phenomenon severity index is indicative of a hazard or emergency.

[0100] More specifically, the electronic processing resources 2 are configured to determine whether the phenomenon severity index is indicative of a risk, or hazard, or an emergency based on a comparison between the phenomenon severity index, or a number of factors by which it is computed, and one or more pre-defined safety thresholds. By way of nonlimiting example, the phenomenon severity index is indicative of a hazard or emergency when it is greater than one or more predefined safety thresholds. On the basis of what has been described, the benefits made possible by this invention are evident.

[0101] In detail, this invention makes it possible to locate a phenomenon of accumulation, or deposition or covering, of a substance in an environment to be monitored 10 and to analyse the evolution of this phenomenon.

[0102] In particular, this invention makes it possible to predict (block 8) the evolution of a phenomenon of accumulation on, deposition on, or covering of a surface of one or more substances so as to anticipate and signal an emergency or dangerous situation based on this future evolutionary scenario. Specifically, this invention makes it possible to predict, and notify a user of, a dangerous or emergency situation based on that evolution.

[0103] Furthermore, the Applicant noted that this invention makes it possible to locate any substance deposited on, or covering, a surface regardless of the type of substance.

[0104] Furthermore, this invention makes it possible to characterise a context of an environment to be monitored 10 in such a way as to locate the type of environment being monitored 10 and various information thereof. Specifically, this invention makes it possible to locate (block 5) one or more areas of interest 11 of the monitored environment 10 in which to focus the analysis of the phenomenon.

Claims

CLAIMS1. Software for the analysis of accumulation phenomena (1 A) in an environment to be monitored (10); the software for the analysis of accumulation phenomena (1A) being storable in and executable by electronic processing resources (2), and designed to cause, when executed, said electronic processing resources (2) to become configured to: receive a time sequence of representations of the environment to be monitored (10); where each representation of said time sequence represents the environment to be monitored (10) at a different time instant; implement a segmentation model (6) configured to locate, or segment, one or more target areas occupied by one or more substances deposited on, or covering, a surface locatable in an input representation; compute (block 8) an evolutionary scenario of a deposition, accumulation, or coverage phenomenon, based on the time sequence of representations of the environment to be monitored (10) and based on the implemented segmentation model (6); where the evolutionary scenario of the phenomenon is a time sequence of representations, of different future time instants compared to the current time instant, representing one or more substances deposited on, or covering, a surface present in an environment to be monitored (10); and provide an output (9) based on said evolutionary scenario of the phenomenon.

2. The software for the analysis of accumulation phenomena (1A) according to claim 1, wherein the segmentation model (6) is a neural model trained on a heterogeneous dataset in terms of the types of substances to be located.

3. The software for the analysis of accumulation phenomena (1A) according to claim 1 or 2, and designed to cause, when executed, said electronic processing resources (2) to become configured to: compute a time sequence of representations of target areas occupied by one or more substances deposited on, or covering, a surface of the environment to be monitored (10) based on the time sequence of representations of theenvironment to be monitored (10) and based on the implemented segmentation model (6); and compute (block 8) the evolutionary scenario of the phenomenon based on a comparison (block 7), or difference, made between target areas of different representations of the time sequence of the representations of the target areas.

4. The software for the analysis of accumulation phenomena (1A) according to claim 3, and designed to cause, when executed, said electronic processing resources (2) to become configured to: compute, based on the time sequence of representations of the environment to be monitored (10), one or more depth maps indicative of information relating to the geometry and / or topography of surfaces and elements of the environment to be monitored (10); and compute the time sequence of representations of target areas occupied by one or more substances deposited on, or covering, a surface of the environment to be monitored (10) based on the computed depth maps and based on the implemented segmentation model (6).

5. The software for the analysis of accumulation phenomena (1 A) according to any of the preceding claims, and designed to cause, when executed, said electronic processing resources (2) to become configured to: compute a rate of occurrence of the phenomenon based on the time sequence of representations of the environment to be monitored (10); and compute (block 8) the evolutionary scenario of the phenomenon also based on the rate of occurrence of the phenomenon.

6. The software for the analysis of accumulation phenomena (1 A) according to any of the preceding claims, and designed to cause, when executed, said electronic processing resources (2) to become configured to: locate (block 5), in the time sequence of representations of the environment to be monitored (10), one or more areas of interest (11) where the phenomenon of deposition, accumulation, or coverage might occur;locate, by means of the segmentation model (6), one or more target areas occupied by substances deposited on, or covering, a surface present in a located area of interest where such phenomenon might occur; and determine the presence of the phenomenon based on the located target areas.

7. The software for the analysis of accumulation phenomena (1A) according to claim6, and designed to cause, when executed, said electronic processing resources (2) to become configured to: implement a context identification model (4) configured to determine, or identify, one or more features of the context of an environment to be monitored (10); and locate (block 5), in the time sequence of representations of the environment to be monitored (10), the areas of interest (11) where the phenomenon might occur based on the context identification model (4), and possibly based on an input indicative of a type of area of interest to be monitored.

8. The software for the analysis of accumulation phenomena (1A) according to claim7, and designed to cause, when executed, said electronic processing resources (2) to become configured to: compute a severity index of the phenomenon based on the evolutionary scenario of the phenomenon and based on one or more context features of the environment to be monitored (10) identified by means of the context identification model (4); and provide an output (9) based on the severity index of the phenomenon.

9. The software for the analysis of accumulation phenomena (1A) according to claim8, and designed to cause, when executed, said electronic processing resources (2) to become configured to: identify the type of one or more substances deposited on, or covering, the surface of the environment to be monitored (10); and compute the severity index of the phenomenon also based on the type of such substances.

10. The software for the analysis of accumulation phenomena (1 A) according to any of claims 6 to 9, and designed to cause, when executed, such electronic processing resources (2) to become configured to: locate (block 5), in the time sequence of representations of the environment to be monitored (10), the areas of interest (11) where the phenomenon might occur based on an input indicative of the type of one or more areas of interest (11) to be monitored.

11. The software for the analysis of accumulation phenomena (1A) in an environment to be monitored (10); the software for the analysis of accumulation phenomena (1A) being storable in and executable by electronic processing resources (2), and designed to cause, when executed, said electronic processing resources (2) to become configured to: receive a time sequence of representations of the environment to be monitored (10); where each representation of said time sequence represents the environment to be monitored (10) at a different time instant; locate (block 5), in the time sequence of representations of the environment to be monitored (10), one or more areas of interest (11) where the phenomenon of deposition, accumulation, or coverage might occur; implement a segmentation model (6) configured to locate, or segment, one or more target areas occupied by one or more substances deposited on, or covering, a surface locatable in an input representation; locate, by means of the segmentation model (6), one or more target areas occupied by substances deposited on, or covering, a surface present in a located area of interest (11) where such phenomenon might occur; and determine, and provide an output (9) based on, an index of the presence of the phenomenon according to the target areas located.

12. The software for the analysis of accumulation phenomena (1A) according to claim 11, and designed to cause, when executed, said electronic processing resources (2) to become configured to:implement a context identification model (4) configured to determine, or identify, one or more features of the context of an environment to be monitored (10); and locate (block 5), in the time sequence of representations of the environment to be monitored (10), the areas of interest (11) where the phenomenon might occur based on the context identification model (4), and possibly based on an input indicative of a type of area of interest (11) to be monitored.

13. The software for the analysis of accumulation phenomena (1A) according to claim12, and designed to cause, when executed, said electronic processing resources (2) to become configured to: compute a severity index of the phenomenon based on the index of the presence of the phenomenon and based on one or more context features of the environment to be monitored (10) identified by means of the context identification model (4); and provide an output (9) based on the severity index of the phenomenon.

14. The software for the analysis of accumulation phenomena (1A) according to claim13, and designed to cause, when executed, said electronic processing resources (2) to become configured to: identify the type of one or more substances deposited on, or covering, the surface of the environment to be monitored (10); and compute the severity index of the phenomenon also based on the type of such substances.

15. The software for the analysis of accumulation phenomena (1 A) according to any of claims 11 to 14, and designed to cause, when executed, such electronic processing resources (2) to become configured to: locate (block 5), in the time sequence of representations of the environment to be monitored (10), the areas of interest (11) where the phenomenon might occur based on an input indicative of the type of one or more areas of interest (11) to be monitored.

16. The software for the analysis of accumulation phenomena (1 A) according to any of claims 11 to 15, and designed to cause, when executed, such electronic processing resources (2) to become configured to: compute (block 8) an evolutionary scenario of a deposition, accumulation, or coverage phenomenon, based on a time sequence of representations of the areas of interest (11) located, wherein the deposition, accumulation, or coverage phenomenon might occur, and based on the implemented segmentation model (6); where the evolutionary scenario of the phenomenon is a time sequence of representations, of different future time instants compared to the current time instant, representing one or more substances deposited on, or covering, a surface present in an environment to be monitored (10); and provide an output (9) based on said evolutionary scenario of the phenomenon.

17. The software for the analysis of accumulation phenomena (1A) according to claim16, and designed to cause, when executed, said electronic processing resources (2) to become configured to: compute a time sequence of representations of target areas occupied by one or more substances deposited on, or covering, a surface of the environment to be monitored (10) based on the time sequence of representations of the environment to be monitored (11) and based on the implemented segmentation model (6); and compute (block 8) the evolutionary scenario of the phenomenon based on a comparison (block 7), or difference, made between target areas of different representations of the time sequence of the representations of the target areas.

18. The software for the analysis of accumulation phenomena (1A) according to claim17, and designed to cause, when executed, said electronic processing resources (2) to become configured to: compute, based on the time sequence of representations of the environment to be monitored (10), one or more depth maps indicative of informationrelating to the geometry and / or topography of surfaces and elements of the environment to be monitored (10); and compute the time sequence of representations of target areas occupied by one or more substances deposited on, or covering, a surface of the environment to be monitored (10) based on the computed depth maps and based on the implemented segmentation model (6).

19. The software for the analysis of accumulation phenomena (1 A) according to any of claims 16 to 18, and designed to cause, when executed, such electronic processing resources (2) to become configured to: compute a rate of occurrence of the phenomenon based on the time sequence of representations of the areas of interest (11) located; and compute (block 8) the evolutionary scenario of the phenomenon also based on the rate of occurrence of the phenomenon.

20. The software for the analysis of accumulation phenomena (1 A) according to any of claims 11 to 19, wherein the segmentation model (6) is a neural model trained on a dataset that is heterogeneous in terms of the type of substances to be located.

21. An accumulation phenomena analysis system (1) in an environment to be monitored (10); the accumulation phenomena analysis system comprising: one or more sensors (3) designed to capture, and transmit, one or more representations, that is, images, of the environment to be monitored (10) at different time instants; and electronic processing resources (2) storing, and configured to execute, the software for the analysis of accumulation phenomena (1A) according to any of the preceding claims.

Images