Method and system for detecting environmental anomalies

Abstract

The invention relates to a system (1) and method for detecting the presence of anomalies in an environment (10). Such a method comprises a step of constructing a reference model (13) comprising metadata describing an environmental anomaly potentially present in the environment and metadata describing a normal state of the environment. It comprises a step of acquiring descriptive content (6) describing an area of the environment, followed by a step of detecting the environmental anomaly in the descriptive content (6) by exploiting the knowledge contained in the reference model (13). Such a method may furthermore comprise a step of localizing the detected environmental anomaly in the environment using map data when the reference model contains map data. A list (15) of detected and localized environmental anomalies may be updated.

Description

[0001] Method and system for detecting environmental anomalies

[0002] The present invention relates to methods and systems for detecting anomalies in an environment to be monitored.

[0003] The term environment refers to any space, region, area, site or geographical location, public or private, such as the territory of a territorial entity (a city, a municipality, a commune, a village or a district for example), a street, a square, a beach, a natural park, a site / place intended to host an event (a festival, a concert, or a sporting event for example), a shopping center, an airport, a train station, a tourist area (a resort or an amusement park for example), a warehouse, a landfill or waste disposal site, a recycling center, a garbage dump, a construction site, communication routes (motorways, railways, banks of a canal for example) or, more generally, a delimited physical space, indoor and / or outdoor.

[0004] An anomaly in such an environment is understood to mean the presence of an object or characteristics that should not exist there under normal conditions. This refers to any object or characteristics or conditions that can be observed by a human being, such as abandoned waste, household waste, bulky items, dirt, pollution, graffiti or disorder (broken or faulty equipment, a deficiency in a crop, a poorly sorted object or non-compliant deposit, etc.) in a given environment.

[0005] It is known from the state of the art to use cameras to provide images of the environment under surveillance to a monitoring center responsible for detecting anomalies and triggering, accordingly, appropriate actions. These cameras can be fixed or embedded in land or air vehicles or in user equipment. The acquired images are analyzed manually (human interpretation) or automatically to detect anomalies.

[0006] In the case of automatic processing of acquired images, two techniques are most often used: deep learning type neural networks to detect and recognize objects or object detection by background subtraction by comparing images to identify the prolonged appearance of new objects.

[0007] However, as proposed in document US2023 / 0140079 A1, techniques based on machine learning require, for their design, a very large number of manually annotated images (typically several million per object category). They also work for a limited number of categories with photos taken in a controlled environment and presenting similarities with the reference image bank.

[0008] As for anomaly detection methods based on background subtraction, they require fixed cameras and relatively stable environmental conditions (viewing angle, lighting, background). In addition, these methods are generally limited to environments of dimensions comparable to the range of the image sensors used. Therefore, they are not suitable for a large public environment or one exposed to diverse and varied contexts (dynamic variations of the background or backdrop, camera movements, lighting conditions, weather conditions, user passage, periodic and justified supply of objects or materials, for example). For these reasons, methods based on background subtraction can only deal with a local and not a global area.They also consider any addition of objects or materials as the appearance of anomalies without making any distinction between what is a normal development of the monitored area and what is an abnormal or degraded development of said monitored area.

[0009] An object of the present invention is to remedy the aforementioned drawbacks.

[0010] To this end, there is proposed, firstly, a method for detecting the presence of environmental anomalies in an environment, this method comprising the following steps: - construction in a first database of a reference model of said environment so that said reference model comprises metadata describing a type of environmental anomaly likely to be present in said environment;

[0011] - acquisition by means of acquisition of descriptive content of an area of ​​the environment;

[0012] - detection by a detection server of the presence of an environmental anomaly in the descriptive content by exploiting the metadata included in the reference model and describing a type of environmental anomaly likely to be present in said environment.

[0013] To be implemented in a large environment and / or exposed to diverse and varied contexts, such a process is arranged so that:

[0014] - the step of constructing said reference model is adapted so that said reference model also includes metadata describing a normal state of the environment, i.e. without said environment including any anomaly;

[0015] - the step of detecting the presence of an environmental anomaly in the descriptive content is adapted to exploit the metadata included in the reference model and describing respectively a type of environmental anomaly likely to be present in said environment and a normal state of the environment.

[0016] According to an advantageous embodiment, a detected environmental anomaly can further be located. In this case, the step of constructing said reference model can be adapted so that said reference model also includes cartographic data of the environment. The method therefore comprises a step of localization by a localization server of said anomaly whose presence has been detected in the descriptive content using the cartographic data of the reference model.

[0017] To provide relevant processing of a detected, or even localized, environmental anomaly, a method according to the invention may advantageously include a step of adding the detected and / or localized environmental anomaly to a list of environmental anomalies, said list being stored in a second database.

[0018] Various additional features may be provided, alone or in combination, including:

[0019] - the method includes a step of filtering by an application server, duplicates in the list of anomalies when it exists;

[0020] - the method comprises a step of sorting by a pre-processing server, the descriptive content according to the context of acquisition of this descriptive content;

[0021] - the cartographic data are initialized by a pre-existing three-dimensional map enriched as descriptive content of areas of the environment is acquired;

[0022] - the method comprises a step of updating the metadata describing a normal state of the environment by means of automatic learning from the acquired descriptive content;

[0023] - the method comprises a step of updating the metadata describing a type of environmental anomaly likely to be present in said environment by means of automatic learning from the list of environmental anomalies.

[0024] Secondly, a system for detecting the presence of environmental anomalies in an environment is proposed, comprising:

[0025] - a first database comprising a reference model of said environment comprising metadata describing a type of environmental anomaly likely to be present in said environment; - means for acquiring descriptive content of an area of ​​the environment;

[0026] - a detection server to which the acquisition means are connected, this detection server comprising a detection module configured to detect the presence of an environmental anomaly in the descriptive content by exploiting the metadata included in the reference model and respectively describing a type of environmental anomaly likely to be present in said environment.

[0027] For the reasons previously mentioned in connection with a method according to the invention, such a system is arranged so that:

[0028] - the reference model also includes metadata describing a normal state of the environment, i.e. without the said environment containing any anomalies;

[0029] - the detection module is configured to detect the presence of an environmental anomaly in the descriptive content by exploiting the metadata included in the reference model and describing respectively a type of environmental anomaly likely to be present in said environment and a normal state of the environment.

[0030] Various additional features may be provided, alone or in combination, including:

[0031] - the reference model of said environment also includes cartographic data of the environment in the form of a digital map;

[0032] - the system comprises a location server arranged to locate the environmental anomaly whose presence has been detected in the descriptive content by the detection server using the cartographic data of the environment of the reference model; - the system comprises a second database maintaining a list of environmental anomalies detected by the detection module and located by the location server;

[0033] - the system includes a pre-processing server integrating a pre-processing module configured to sort the descriptive content according to the context of acquisition of this descriptive content;

[0034] - such a pre-processing module includes a conversational agent capable of requesting additional information concerning descriptive content;

[0035] - the acquisition means are carried on board land, sea or air vehicles.

[0036] Other characteristics and advantages of the invention will appear more clearly and concretely on reading the following description of embodiments, which is given with reference to the appended drawings in which:

[0037] - Figure 1 schematically illustrates an environmental anomaly detection system according to various embodiments;

[0038] - figure 2 schematically illustrates steps of a method for detecting environmental anomalies according to various embodiments;

[0039] - Figure 3 schematically illustrates an implementation architecture of the system and method for detecting environmental anomalies according to various embodiments.

[0040] Referring to Figure 1, a system 1 is shown for detecting the presence of an environmental anomaly in an environment 10. No limitation is attached to the extent or nature of this environment 10 which may be a natural space, a city, a municipality, a farm, a real estate complex or, more generally, a space under surveillance. The system 1 comprises at least one acquisition means 2-5 allowing the acquisition of descriptive content 6 of an area of ​​the environment 10. By way of non-limiting examples, the acquisition means 2-5 is a smart mobile phone (or "smartphone" according to English terminology), a fixed or on-board camera, a lidar, an infrared sensor, a computer, connected glasses, a messaging service or, more generally, any means allowing the conveyance of information or descriptive content 6 of an area of ​​the environment 10 under surveillance.Here, the term “area of ​​the environment 10” means any space or part that can be located relatively or absolutely in this environment 10.

[0041] In one embodiment, one or more acquisition means 2-5 are embedded in mobile supports, in particular land, sea or air vehicles (such as a drone). Preferably, the system 1 uses a variety of acquisition means 2-5, in particular means qualified as “opportunity” capable of spontaneously capturing an area of ​​the environment (for example, in the form of images or videos) such as smartphones equipping users present in the environment 10, cameras or lidars embedded on board vehicles of opportunity traveling at least part of the environment 10 (such as a waste collection vehicle, a bus, a tram, a metro, a postal vehicle or a taxi in the case of an urban environment 10), discussion groups on social networks (by connecting to these groups, or by using keywords or “Hashtags”), or mobile applications making it possible to report an environmental anomaly.A system 1 may further comprise acquisition means 2-5 in the form of surveillance cameras already installed in the environment 10 (for example, connected to a supervision center or nomadic).

[0042] Depending on the acquisition means 2-5 used, the descriptive content 6 may comprise an image, audio content, textual content, a point cloud, or a combination thereof. The acquisition (step 21 in FIG. 2) of descriptive content 6 uses, in one embodiment, lighting, also described as “opportunity” such as natural lighting or lighting that can be activated or activated at opportune times, for example, in the morning, in the evening or in cloudy weather to accentuate contrasts or avoid shadows cast in sunny weather. An acquisition means 2-5 comprises, in one embodiment, active sensors (illumination lights, active thermal camera) to illuminate and reveal details more easily.

[0043] The acquisition of descriptive content s of an area of ​​the environment 10 may be systematic, planned (for example, by means of a drone following a predefined flight plan), adapted on the fly or opportunistically (for example to zoom in on areas likely to contain anomalies), in regular sampling (for example, by fixed cameras), in response to the occurrence of a triggering or random event (typically, by Smartphones), in real time or in delayed time (for example, descriptive content 6 extracted from a discussion thread on a social network), or on the fly and opportunistically using acquisition means 2-5 equipping users present in the environment 10.A plurality of descriptive contents 6 can also be acquired synchronously or asynchronously, with regular or irregular spatial and temporal sampling, such as an acquisition of descriptive contents 6 by drone following swaths with a certain overlap rate.

[0044] The acquisition of a descriptive content 6 is, in one embodiment, triggered externally by at least context data such as position data (in particular, GPS coordinates) or movement data (speed or direction for example) of the acquisition means 2-5, or by a sensor such as a presence sensor. For example, when a vehicle incorporating an acquisition means 2-5 approaches an area of ​​interest in the environment 10 (such as a waste collection point), the GPS position of the vehicle can be used to start the acquisition and stop it when the vehicle moves away from the area of ​​interest. For example, the detection by an external sensor of the presence of a vehicle or people near the monitored area can trigger the acquisition by a fixed surveillance camera, whether in video mode, in photography mode, or in sequence mode of photographs according to a predefined frequency (called, in English, “time lapse”).

[0045] The heterogeneous descriptive contents 6 (RGB / IR image, video, text or voice messages, point clouds, social network posts for example) coming from the various acquisition means 2-5 are provided as input to a preprocessing module 11 (preprocessing step 22 in Figure 2) configured to sort these descriptive contents 6 according to the context of their acquisition and to encode them according to predefined formats.

[0046] A context for acquiring descriptive content s can be determined from metadata associated with the descriptive content 6 received such as the acquisition means 2-5 source of this descriptive content 6 (for example, data from a drone, a fixed camera, an on-board camera or a post on a social network), explicit position data or data that can be deduced from metadata when these are available (IP address, radio cell to which the acquisition means is attached for example). An interpretation of the contextual data advantageously makes it possible to homogenize the descriptive content 6 received and make their subsequent assembly easier.

[0047] In one embodiment, the preprocessing module 11 comprises a conversational agent (or “Chatbot” according to English terminology) capable of requesting additional information concerning a received descriptive content 6. This request can be addressed to the acquisition means 2-5 source of the received descriptive content 6 or to another acquisition means 2-5 present in the area in which the received descriptive content 6 was acquired. This functionality is of great interest for descriptive content 6 originating from an acquisition triggered by a human in connected mode who does not have the possibility of taking a photo easily (for example, garbage collectors on their rounds, bus drivers, motorists or city dwellers on bicycles). A conversation can, thus, be initiated to verify or supplement information concerning an environmental anomaly which is the subject of a received descriptive content 6.To analyze the received descriptive contents 6 and deduce a context therefrom, the pre-reprocessing module 11 may be provided with natural language processing algorithms to convert or translate a voice message into text with automatic speech recognition algorithms (e.g. Whisper, OWSM or CoVoST), to encode the text by transformation (e.g. Word2Vec or BERT) and / or machine learning algorithms and models to generate a description from images, with or without prompting (e.g. Google Vision API, Image GPT, Florence2, CLIP, Meta SAM) or characterize 3D objects from a point cloud (e.g. K-FPN) or, more generally, artificial intelligence algorithms with generative or multimodal capacity (such as, for example, MUM or GPT-4).Determining a context for acquiring descriptive content 6 advantageously makes it possible to attenuate the differences between descriptive content 6 relating to the same environmental anomaly (variabilities in the shooting angle, brightness, zoom level, clarity or terms used to describe an environmental anomaly).

[0048] The descriptive contents 6 instantiated with a unique identifier and encoded according to predefined formats are communicated to an environmental anomaly detection module 12. This detection module 12 is configured to detect (detection step 23 in FIG. 2) an environmental anomaly in the descriptive content 6 using information from a reference model 13 of the environment 10.

[0049] The reference model 13 comprises metadata describing a normal state of the environment 10 (i.e., an environment 10 without anomalies). This metadata comprises expected characteristics of the environment 10 without environmental anomalies, such as characteristics of shape, color, texture, and dimension indications (expressed in units of length or in pixels in a relative manner with respect to surrounding visual cues). This is, for example, a set of color and / or texture bands representing the color tones and / or roughness of the normal surface state of the environment at a given location, a set of photographs in the vicinity of a voluntary waste drop-off point in the absence of waste. Thus, it is possible to describe a beach or range of colors and / or textures of sand on a beach, or of the surface of a walking path adjoining the latter, in the absence of waste.In one embodiment, the metadata describing a normal state of the environment 10 can be obtained from cartographic data of the environment 10, for example to assign a function or a use to an area of ​​the monitored environment. Thus, an area corresponding to a dumpster can be assigned to the reception of plants and not of detritus or rubble, thus reflecting a normal state of the area associated with such a dumpster. Metadata describing a normal state of the environment 10 are, in one embodiment, obtained from descriptive contents 6 in which anomalies are erased using, for example, functions based on generative artificial intelligence such as the "magic eraser" of the "Google Photos" application.More generally, retouching or data extraction functions based on artificial intelligence, such as Magic Eraser or automatic clipping, can be used to generate metadata describing a normal state of the environment 10 from descriptive content 6 (in particular, images or videos) of this environment 10.

[0050] Furthermore, the reference model 13 comprises metadata describing at least one environmental anomaly likely to be present in the environment 10. This metadata describes the type of anomalies expected in the environment 10 (for example, debris, household waste, bulky items or cardboard boxes). Metadata describing an environmental anomaly commonly found in the environment 10 may specify its color or colors, its shape and / or its dimensions (in units of length or in pixels relative to surrounding visual cues). In combination or alternatively, metadata describing an environmental anomaly may comprise a textual, vocal and / or graphical description (a plurality of photographs taken, preferably, under different weather and / or lighting conditions).In one embodiment, metadata describing at least one environmental anomaly likely to be present in the environment 10 are generated by a generative artificial intelligence algorithm (for example, Dalle-E or Midjourney) from descriptive content (in the form of text, an image, a sound signal and / or a point cloud) of said anomaly. Alternatively or additionally, the metadata describing an environmental anomaly likely to be present in the environment 10 may be obtained from cartographic data of said environment 10.

[0051] Advantageously, the reference model 13 made available to the detection module 12 makes it possible to detect and classify objects of varied appearance (type of object, shape, color, actual dimensions) present in very different contexts (surrounding environment, lighting conditions, weather conditions) from images with variable characteristics due to variations in the shooting (distance to the object, viewing angle) and in the quality of the images (type of sensor, resolution, dimensions of the object in the image, dimensions of the image, camera optics).The reference model 13 makes, in fact, the system 1 more robust to hazards (day / night, grazing light, rain, wind) and to variations in environmental conditions and acquisition conditions of the descriptive content 6 by conferring tolerance to noise in the descriptive content 6 (poor quality of the shot), to camera jitter (unstable shot), to an unsuitable image sensor or lidar setting, to a variation in brightness or weather, to dynamic movement in the background, to a change in the background, to camouflage of objects perceived as part of the background, or to foreground objects that become static. Thanks to the reference model 13, the system 1 is suitable for more cases of sensors, types of environments and / or types of anomaly.Furthermore, the use of artificial intelligence to interpret the descriptive contents 6 and / or the knowledge included in the reference model 13 makes it possible to overcome the limitations of the usual machine learning methods based on the manual annotation of a large volume of images. In one embodiment, the reference model 13 is built as descriptive contents 6 of areas of the environment 10 are acquired.

[0052] By exploiting the knowledge included in the reference model 13, the detection module 12 is designed to detect (step 23 in FIG. 2) an anomaly included in one or more descriptive contents 6. For this, the detection module 12 implements, in one embodiment, artificial intelligence programs, in particular algorithms for automatic processing of natural language or semantic analysis, image processing algorithms for automatic recognition of objects, supervised or unsupervised classification algorithms, or fuzzy logic algorithms.

[0053] In an illustrative embodiment, algorithms based on neural networks, in particular of the deep learning type, make it possible, by using the knowledge contained in the reference model 13, to detect and locate (see below) within an image objects considered, according to the reference model 13, as being environmental anomalies. As non-limiting examples, the “deep learning Yolo v5” algorithm (or, for example, “VGG-16”, “EfficientNet-B3” and “ResNet-50”) previously trained on a training sequence containing images of environmental anomalies to be detected is capable, from acquired images, of detecting the presence of these environmental anomalies, of determining their position in the image (segmentation) and of recognizing the type of these environmental anomalies (classification).To ignore the presence of humans or animals in the acquired images, the detection module 12 can use pre-trained algorithms.

[0054] In another embodiment, the detection module 12 uses algorithms based on deterministic processing exploiting the specific knowledge of the reference model 13, without prior learning phase of neural networks. For example, when a descriptive content 6 comprises an image, the detection module 12 can perform a color (or texture or shape or dimension) analysis to identify the elements of the image that do not correspond to the normal state of the environment: these elements are directly considered as anomalies. Thus, the anomaly can be detected by identifying a distortion of the normal state of the environment. Alternatively, the detection module 12 can perform a color (or texture, shape or dimension) analysis to identify the elements of the image that correspond to an environmental anomaly: these elements are directly considered as anomalies.Thus, the anomaly can be detected by matching the characteristics of an anomaly. Finally, in one embodiment, the two approaches can be combined, or even sequenced: an anomaly can be detected by considering that it does not correspond to the normal state of the environment and that it corresponds to the characteristics of the anomalies expected in the environment 10.

[0055] Alternatively, step 23 of detecting an environmental anomaly uses a hybrid approach combining both a semantic (deterministic) approach and a statistical approach (learning artificial intelligence algorithms using neural networks). The advantage of such an approach is to significantly reduce the complexity of the program and improve its performance, by extracting only useful information and simplifying the approach for deep learning algorithms. For example, when descriptive content 6 comprises an image, initial processing steps may be used to isolate an area of ​​the image likely to contain an anomaly via deterministic analysis of color, shape, or texture, and subsequent processing steps may be used to confirm and identify the type of anomaly by object recognition via a trained neural network.

[0056] In one embodiment, when a descriptive content 6 comprises an image, the detection module 12 searches for one or more elements of this image in the reference model 13. When the image comes from a drone taking horizontal photos, a 2D correlation of the image with metadata describing a normal state of the environment 10 including photos in a top view makes it possible to precisely find the anomaly or anomalies in the environment 10. When the image comes from a fixed camera, a 2D correlation of the image with metadata describing a normal state of the environment 10 (360-degree photos of the area taken during a calibration in the absence of environmental anomalies) makes it possible to detect and locate the presence of an anomaly.In the same way, when the image comes from a truck systematically taking photos during its rounds, a 2D correlation of the image with metadata describing a normal state of the environment 10 (including views of the street or reference photos, for example, around voluntary drop-off points), makes it possible to detect and precisely locate the environmental anomaly. When a descriptive content 6 includes an object having strong similarities with an anomaly likely, according to the reference model 13, to be present in the environment 10, this object is considered by the system 1 to be an environmental anomaly. Thus, the invention makes it possible to detect only what is an anomaly and to ignore what is a normal state of the environment, including when the environment 10 varies dynamically in time and space.Indeed, depending on the environment in which the acquisition means 2-5 is located, at the precise location and at a given time, the detection module 12 can optimize the parameterization of the detection algorithm by exploiting the knowledge of the normal state of the environment 10 from the reference model 13, in order to best detect the anomalies likely, according to the reference model 13, to be present in the environment 10. Similarly, the detection module 12 can select, from a plurality of algorithms, the algorithm most suited to detecting the anomalies likely to be present in the environment 10 whose normal state is known.The parameterization may, for example, consist of ignoring certain colors or textures or shapes corresponding to the normal state of the environment at that location and at that time (for example, the color of sand or asphalt), highlighting certain colors or textures or shapes corresponding to anomalies likely to be present at that location and at that time (for example, the color or shape of food packaging), or performing a combination of the two techniques.The parameterization may consist of adjusting the algorithm to take into account the uniformity of the background or the background of the image while considering the dimensions of the object to be detected (for example, to search for a lost glove on a large expanse of virgin snow, the contrast of the colors compared to the background of the image may be sufficient), or on the contrary, taking into account the heterogeneity of the background or the background of the image while considering the physical characteristics of the object to be detected (for example, to search for a cigarette butt on vegetated sandy soil, the form factors may be discriminating).Algorithm selection may involve choosing the most suitable algorithm from a set of distinct algorithms, respectively optimized for different conditions (e.g. algorithms trained respectively to detect a can on a yellow background corresponding to sand, on a black background corresponding to asphalt and on a textured background corresponding to a sparse lawn). Different algorithms or different versions of the same algorithm may thus be executed in parallel, so as to select the output that gives the highest likelihood score.Also, optimization consisting of using different versions of algorithms or different algorithms can be carried out for the same descriptive content 6: for example, when the descriptive content 6 is an image, the image can be divided into several distinct areas and distinct optimal algorithms (or distinct versions of algorithms) can be used respectively for different areas of the image: thus, when a camera captures an image comprising different waste collection bins, a different algorithm can be used to detect anomalies present in each bin, depending on their assignment corresponding to their normal and expected state (for example plants, rubble, wood) and possibly with anomalies likely to be present for each waste stream (for example respectively plant bag, rubble bag, and Placoplâtre sheets).By using a reference model 13 of an environment 10 during the analysis of a descriptive content 6, more precisely by taking into account the normal state of said environment (which may be plural) and the type(s) of anomaly sought, such a system 1 according to the invention can thus be used in a very wide variety of environments, in order to detect a wide variety of anomalies.

[0057] The detection of an environmental anomaly in a descriptive content 6 in the form of an image can also be obtained by identifying areas of the image that may possibly correspond to anomalies, performing a deterministic analysis based on the color, texture or shape of objects included in these areas, extracting a window around the potential anomaly taking into account the metadata describing the environmental anomalies likely to be present in the environment 10, and querying a deep learning algorithm (of the “Google Lens” or “ImageGPT” type) to identify and confirm the anomaly. Thus, the invention makes it possible to significantly improve the detection performance by simplifying the tasks to be performed.On the one hand, the results are improved since the last classification step (the most complex) is requested with prepared and better framed data, ensuring a higher success rate, in a context where objects of varied appearance can be present in very different contexts from descriptive contents 6 of variable quality. On the other hand, the processing capacity of many large images is greatly improved since the last processing steps (the most expensive in terms of computation) are performed only if necessary and not systematically on all the pixels of all the images.

[0058] It should be noted that the detection of an environmental anomaly in a received descriptive content 6 can be direct (recognition of an anomaly thanks to the metadata describing this anomaly of the reference model 13) or indirect from observations indicating the potential presence of an anomaly. For example, the detection of a deviation of a flow of pedestrians or vehicles from an expected route to bypass an obstacle can raise doubt about the presence of an anomaly. Thus, an anomaly can be detected by identifying a distortion of the normal state of the environment by relying on indirect observations (in the example the observation of pedestrian flows makes it possible to raise doubt about the potential presence of an anomaly).As before, the processing can be carried out in stages: the doubt raised by a separation of pedestrian or vehicle flows can be removed by more detailed analyses, for example by carrying out more detailed image processing on a specific window of the image, by zooming with the source acquisition means 2-5 (for example a camera) on a central area (for example if it is a fixed PTZ type camera, an acronym for "Pan, Tilt and Zoom") or by sending a drone to fly over the suspect area. Thus, the detection of anomalies can integrate several nested processes and interact with the process of acquiring the descriptive content 6 of the environment.

[0059] In one embodiment, the detection module 12 determines a probability of presence of an environmental anomaly. An anomaly detected with a probability lower than a predefined threshold value can be submitted to a human operator for confirmation or denial. The responses provided by the human operator are used to progressively train the artificial intelligence algorithm of the detection module 12 to improve its performance (the human confirmation / denial serving as training data). An advantage of such a semi-automatic mode of the detection module 12 is that it is operational without waiting for an automatic mode to achieve sufficient reliability. A semi-automatic mode of the detection module 12 also makes it possible to update the reference model 13 progressively and reliably, and in particular to complete the training of the artificial intelligence algorithms.

[0060] When an environmental anomaly is detected, the mere detection of the latter may be sufficient to trigger an alert or initiate relevant processing of said anomaly. Alternatively or additionally, a system 1 according to the invention may further comprise a location module 14, configured to locate (step 24 in FIG. 2) in the environment 10, a detected anomaly. For this, the reference model 13 may advantageously integrate cartographic data of the environment 10. These cartographic data are, in one embodiment, a georeferenced plan of a determined area of ​​the environment describing a layout of the premises with the presence of permanent or temporary structures (for example, a layout of containers, marquees or temporary stands, etc.).Such cartographic data may, alternatively or additionally, consist of a list of georeferenced markers, a digital map, preferably a 3D map, of the environment 10. A 3D map of the environment 10 is useful for distinguishing the desired flight altitude of an aerial vehicle carrying an image sensor and the actual height between this aerial vehicle and the ground at the location of the image. The cartographic data are, in one embodiment, initialized by a pre-existing three-dimensional map enriched as and when descriptive contents 6 of areas of the environment 10 are acquired.

[0061] The location determined by the location module 14 consists of determining the position, preferably in 3D, in a reference frame linked to the environment 10 or relative to a reference point included in this environment 10. The altitude information can be useful in urban environments in the presence of constructions and buildings. Otherwise, the 3D coordinate system can be reduced to a 2D system (ignoring the altitude, for example). When the dimensions of the anomaly are significant, the coordinates of the barycenter of said anomaly are retained.

[0062] Metadata of a descriptive content 6 may be used to assist in locating the detected anomaly in the environment 10. This metadata is, for example, position data of the acquisition means 2-5 (GPS data of a drone or a mobile phone), the IP address of the user terminal that posted the message or the access point to which this user terminal is connected or position data associated with or included in this message. Position data of an environmental anomaly may also be determined from a descriptive content 6 in the form of a voice message or a text message explicitly indicating an address of the anomaly.

[0063] To determine the position of a detected anomaly, the location module 14 uses, in one embodiment, orthophotography or ortho-mapping to correct imperfections of the image sensors 2-5 and of the image capture (geometric rectifications). Also, photogrammetry makes it possible to obtain precise 3D images from several photographic views of the objects. In addition, the georeferencing of the images associated with high-precision reference shots (for example, tacheometer, 3D scanner, GNSS with satellite correction) makes it possible to associate precise positions with images.

[0064] When a descriptive content 6 is an image with contextual geographic positioning information, more precise positioning can be achieved by positioning relative to the location base (using the GPS position of the acquisition means 2-5, for example a camera, and positioning the object in the image relative to said acquisition means 2-5). When the latter and the GPS receiver are not confused or sufficiently close, the position of the acquisition means 2-5 can be deduced. For example, when acquisition means 2-5 consist of four cameras on board a vehicle and installed respectively at the four ends of the roof, and the GPS receiver is in the center of the vehicle, the position of each camera can be calculated according to the dimensions of the roof and the orientation of the vehicle (compass information).A re-alignment of the image with respect to the vehicle position data and, possibly, with respect to the flyover altitude and / or the flight angles of an aerial vehicle makes it possible to access the location of the anomaly in the environment 10. In one embodiment, the position of the pixels of the object in the image 6 is converted into an absolute position in the environment 10 using the GPS position of the camera (acquisition means 2-5) and the information from the shot (camera angles, height, zoom level, geometric deformation of the lens, focal length, depth of field for example). In one embodiment, the method for detecting environmental anomalies further comprises a step of characterizing or describing each of the detected environmental anomalies.In fact, at least one class from a list of predefined classes (for example, bulky, food packaging, cigarette butt or graffiti) is attributed to the detected environmental anomaly and possibly additional characteristics, in particular physical (such as material, shape, dimensions, mass, volume, number of constituents, color, gravity, recyclability, volatility and / or state for example), are attributed to it.

[0065] Each of the detected and located environmental anomalies is listed or added (step 25 in Figure 2) to a list of environmental anomalies 15. A timestamping step makes it possible to date the environmental anomalies in this list. It is thus possible to precisely count all the environmental anomalies present at a given date in a given area of ​​the environment 10 (presence in time and space of the anomalies). This is a recording database associating environmental anomalies with geographical positions in the environment 10. The recording of an environmental anomaly includes, for example, a description of the anomaly, its location in 3D, the timestamp and confidence intervals for each component (such as “presence of plastic bottle @ 88%; location xyz ± 3m; 25 / 9 / 2022 6:32 p.m. ± 1 s”).

[0066] Different operations on the list of environmental anomalies can be envisaged by means of a processing module 16 such as generating alerts and / or notifications upon detection of an anomaly of predefined types or present in predefined locations, carrying out spatial and temporal analyses of the presence of anomalies, carrying out dated diagnoses (assessment on a given date, cleanliness, inventory), defining objective quality indicators, monitoring the evolution over time of the situation in an area of ​​the environment 10, generating suggestions for optimizing environmental anomaly management services (routes of tours, frequency of tours, location of collection points, suitable collection means, planning of tours for example).The environmental anomalies listed in list 15 can be statistically compiled with indicators (e.g., objective cleanliness indicator) to study problem areas and possibly adjust services. In addition, a routing or route calculation algorithm can generate a route passing through the positions of the detected anomalies, following predefined constraints, plan interventions or adjust waste collection rounds, area maintenance, maintenance operations or other interventions.

[0067] In one embodiment, the processing module 16 is configured to perform a step of filtering duplicates (an anomaly deduplication operation) in the list 15 of environmental anomalies because the same environmental anomaly may be present on several descriptive contents 6 originating from the same acquisition means 2-5 or reported by several acquisition means 2-5. Deduplication consists of instantiating the environmental anomalies in a unique manner in order to avoid counting the same anomaly twice (or more). The removal of duplicates makes it possible to refine the location of the environmental anomalies, in particular when the observation angles are different (stereoscopy). Indeed, the observation of the same anomaly from different locations makes it possible to refine its position by triangulation.For this, each incoming environmental anomaly is, in one embodiment, compared to the anomalies present in the list 15. In order to decide whether it is the same anomaly, a multi-criteria metric taking into account the description of the anomaly and its characteristics (dimensions, colors, weight, location, date for example) can be adopted. In one embodiment, the processing module 16 can use the conversational agent to query an acquisition means 2-5 present in the area in which an environmental anomaly is detected and, thus, remove a possible duplicate.

[0068] The processing module 16 is, in one embodiment, configured to filter the list of anomalies 15 by removing therefrom the anomalies whose detection probability is lower than a predefined threshold value, whose position and / or date could not be determined sufficiently precisely or whose characteristics are inconsistent. This filtering function can be implemented in fuzzy logic, with one or more decision thresholds.

[0069] An optimization of the management of the environment 10 using the processing module 16 can be envisaged such as a maximization of a ratio between a cleanliness, attractiveness or maintenance objective (absence of anomalies) and a cost of the means deployed to treat these anomalies. For this, an analysis of the list of detected anomalies makes it possible to generate suggestions concerning the methods and means of action to treat (remove, or reduce, for example) these anomalies (routes and frequencies of the rounds for treating anomalies, sizing of the means of collection, cleaning or maintenance, addition or redistribution of collection points for example).

[0070] More generally, the recorded environmental anomalies can be analyzed in the spatial and / or temporal domain in order to identify trends and make informed decisions in the medium and long term concerning the management of environmental anomalies in the environment 10. Statistics can be calculated on the environment 10 or by zone thereof (number of anomalies, number of anomalies by type, average number of anomalies, average number of anomalies by type, number of anomalies per day of the week and per zone, maximum number of weekly anomalies over a year and per zone for example). Before / after reports and comparisons can thus be drawn up by the processing module 16. A human-machine interface makes it possible to interact with the processing module 16 in order, in particular, to visualize a superposition of the environmental anomalies on the cartographic data of the environment 10.This visualization makes it possible, in one embodiment, to filter the list of environmental anomalies (by type of anomaly, by geographical location, by date) or to group them by cluster. In order to improve the relevance of the reference model 13, an anomaly detection method according to the invention comprises, in one embodiment, a step of updating the metadata describing the environmental anomalies likely to be present in the environment 10 by means of automatic learning from the list 15 of environmental anomalies. This updating step is preferably periodic. In one embodiment, the updating of the metadata describing the environmental anomalies can be done semi-automatically or manually, i.e. with the intervention of a human operator.

[0071] Advantageously, the method for detecting environmental anomalies further comprises a step of updating the metadata describing a normal state of the environment 10 by means of automatic learning from the acquired descriptive contents 6. Such an update makes it possible to refine the knowledge on the normal state of the environment 10 (for example more precise learning of the background, installation of a structure, construction of a building) and to refine the knowledge on the anomalies present in the environment 10. The reference model 13 is, consequently, scalable so that the system 1 can be operational quickly, without waiting for the end of a disproportionate learning effort. In an advantageous embodiment, the updating of the metadata describing the normal state of the environment can be done semi-automatically or manually, that is to say with the intervention of a human operator.

[0072] In one embodiment, the reference model 13 can be constructed as descriptive contents 6 of areas of the environment 10 are acquired. The acquisition of contents 6 describing the real and updated state of the environment 10 thus makes it possible to discover the environment 10 more precisely and to update the reference model 13 by modifying the metadata describing a normal state of the environment, the metadata describing at least one environmental anomaly and / or the cartographic data of the environment when said reference model 13 includes them.

[0073] In an illustrative implementation, the system 1 and the method for detecting environmental anomalies presented above can be used for the detection of waste on a beach intended to host an event. The cartographic data of the reference model 13 can, in this case, be initialized by a pre-existing map containing aerial views of the beach (for example, from “GoogleMaps” or “Géoportail”). This cartographic data can be refined with a flyover of the site before the event to obtain a better image resolution. A matching of the images on the pre-existing map can be carried out by georeferencing. This flight can be carried out before the installation of the structures of the event is carried out and / or when the structures are installed (in this case, the structures are part of the reference map).Preferably, visual landmarks whose exact dimensions are known are identified. These may be objects specifically deployed during reconnaissance, or known objects remaining during the event. Metadata describing a normal state of the beach relates to a beach without waste (for example, a color, a shape and / or a texture of sand) and structures installed on this beach. A waste likely to be present in this environment is, for example, a cigarette butt. Following an acquisition of images before, during and after said event, the detection module 12 can detect a cigarette butt in an image acquired from analyses relating to the color, the texture and / or the shape of objects included in these images.To do this, the detection module 12, by exploiting the reference model 13, can identify the areas of the image that do not correspond to the expected colors or textures of the sand, then locate in these areas the pixels approaching the typical color of a cigarette butt (simple distance), or a group of neighboring pixels approaching the typical texture of a cigarette butt (correlation). A contour detection is then performed. The length of the object is checked and compared to the typical length of a cigarette butt to validate or not the detection. In another embodiment, the detection module 12 can cut out pixel windows corresponding to the dimensions of a cigarette butt. For each window, a color and / or texture analysis can be performed; if the threshold is exceeded (for example 90% of the pixels correspond to a cigarette butt), a cigarette butt is detected. The precise location of the cigarette butt is obtained for the window maximizing the exceeding of the threshold (local maximum).This results in detection, localization and recognition of the “cigarette butt” anomaly.

[0074] In addition, using the position of the descriptive content 6, the reference model 13 can be locally updated to optimize the anomaly detection performance. This makes it possible, for example, to parameterize the detection algorithm in an optimized manner for this local environment or to select the best detection algorithm for this local environment. In the example of the illustrative implementation, the position of the drone can be used to extract the expected characteristics of the normal state of the environment at this specific location (e.g., the color of the sand depending on whether it belongs to a particular sand strip, or vegetation corresponding to particular colors and textures). In the same way, the position of the drone can be used to extract the characteristics of the anomalies expected at this specific location (e.g., cigarette butts if there is a bar nearby, or food packaging if there are takeaways).This information makes it possible to optimize the detection performance of the process, including by modifying the method of acquiring the descriptive content 6 (for example, the detection of cigarette butts could require flying at a lower altitude than that required for the detection of larger food packaging).

[0075] In another illustrative implementation, the systems and methods described above may be used for waste detection in an urban environment by cameras mounted on a waste collection vehicle. The map data may include a pre-existing map enriched with photographs of the street associated with addresses or geographic positions (e.g., from "Google Street View"). It is also possible to describe the urban environment with texture, color, and shape characteristics (e.g., sidewalk, street furniture, collection bins) and to use pre-existing and trained algorithms to ignore the presence of humans or animals. The anomaly characteristics describe the type of anomalies expected during waste collection: debris, household waste, bulky items, or cardboard, for example.As above, the description may include the color, shape, and dimensions of these environmental anomalies. It may also include a set of photographs depicting commonly encountered anomalies in the area. Metadata describing a normal state of the urban environment may include photographs of the surroundings of voluntary drop-off points in the absence of waste. As described above, the deployment of System 1 improves waste management and the cleanliness of urban environments.

[0076] Referring to Figure 3, an implementation architecture of the system 1 comprises a plurality of acquisition means 2-5 such as a drone equipped with an image sensor, a smartphone, a lidar, a discussion group on a social network and / or a fixed or mobile camera. Each of these means 2-5 is connected via a first appropriate communication interface 7 (such as a mobile communication interface of the 4G, 5G or equivalent type, a satellite communication interface, a wired communication interface, or a WIMAX or Wi-Fi communication interface) to a preprocessing server 31 integrating the preprocessing module 11. The preprocessing server 31 is provided with centralized or distributed computing and memory resources (for example in a Cloud according to English terminology).The preprocessing module 11 is, in one embodiment, a computer program product installed on a memory medium of the preprocessing server 31 and capable of being implemented within a computer processing unit of the preprocessing server 31 to cause a preprocessing 22, as described above, of the descriptive contents 6 received via the first communication interface 7. The descriptive contents 6 preprocessed by the preprocessing server 31 are communicated to a detection server 32 hosting the detection module 12. This detection server 32 is provided with centralized or distributed computing and memory resources (for example in a Cloud according to English terminology).The detection module 12 is, in one embodiment, a computer program product installed on a memory medium of the detection server 32 and capable of being implemented within a computer processing unit of the detection server 32 to cause the implementation of a detection step 23, as described above, of an environmental anomaly in the descriptive contents 6 by exploiting knowledge included in the reference model 13. In one embodiment, the reference model 13 is stored in a first database 33 remote or local to the detection server 32.

[0077] A location server 34 integrating the location module 14 is programmed to locate in the environment 10 an anomaly detected by the detection server 32, using the cartographic data of the reference model 13. For this, the location module 14 is, in one embodiment, a computer program product installed on a memory medium of the location server 34 and capable of being implemented within a computer processing unit of the location server 34 to cause the implementation of a location step such as step 24 described above as well as a step 25 of adding any located environmental anomaly to a database 35.

[0078] An application server 36 comprising the processing module 16 makes it possible to carry out various post-processing operations on the list 15 of environmental anomalies stored in the second database 35. The processing module 16 is, in one embodiment, a computer program product installed on a memory medium of the application server 36 and capable of being implemented within a computer processing unit of the application server 36. In one embodiment, the servers 31, 32 and 34 and the databases 33, 35 are part of a computer system, distributed or not, accessible to the application server 36 via a second wired or wireless communication interface 8. In other words, said servers 31, 32 and 34 can constitute, alone or in combination, one or more physical and computer entities.The application server 36 can also be integrated into the same computer system as the servers 31, 32 and 34. In another embodiment, one or more of the functionalities of the aforementioned servers are integrated into the acquisition means 2-5 such as a Smartphone, a drone or a portable camera integrating a processor and a memory for the implementation of one or more steps of the method for detecting environmental anomalies according to any one of the embodiments presented above.

Claims

CLAIMS 1. Method for detecting the presence of environmental anomalies in an environment (10), this method comprising the following steps: - construction in a first database (33) of a reference model (13) of said environment (10) so that said reference model (13) comprises metadata describing a type of environmental anomaly likely to be present in said environment (10); - acquisition (21) by an acquisition means (2-5) of a descriptive content (6) of an area of the environment (10); - detection (23) by a detection server (32), of the presence of an environmental anomaly in the descriptive content (6) by exploiting the metadata included in the reference model (13) and describing a type of environmental anomaly likely to be present in said environment (10); said method being characterized in that: - the step of constructing said reference model (13) is adapted so that said reference model (13) further comprises metadata describing a normal state of the environment (10), that is to say without said environment (10) comprising any anomaly; - the step of detecting (23) the presence of an environmental anomaly in the descriptive content (6) is adapted to exploit the metadata included in the reference model (13) and describing respectively a type of environmental anomaly likely to be present in said environment (10) and a normal state of the environment (10).

2. Method according to the preceding claim, for which: - the step of constructing said reference model (13) is adapted so that said reference model (13) also comprises cartographic data of the environment (10); - said method comprises a step of locating (24) by a server (34) locating said anomaly whose presence has been detected (23) in the descriptive content (6) using the cartographic data of the reference model (13).

3. Method according to the preceding claim, comprising a step of adding (25) the detected and located environmental anomaly in a list (15) of environmental anomalies, said list (15) being stored in a second database (35).

4. Method according to claim 3, comprising a step of filtering by an application server (36), duplicates in the list of anomalies (15).

5. Method according to any one of claims 1 to 4, comprising a step of sorting by a pre-processing server (31), the descriptive content (6) according to the context of acquisition of this descriptive content (6).

6. Method according to claim 2, for which the cartographic data is initialized by a pre-existing three-dimensional map enriched as descriptive contents (6) of areas of the environment (10) are acquired.

7. Method according to any one of claims 1 to 6, comprising a step of updating the metadata describing a normal state of the environment (10) by means of machine learning from the acquired descriptive content (6).

8. Method according to claim 3, comprising a step of updating the metadata describing a type of environmental anomaly likely to be present in said environment (10) by means of automatic learning from the list of environmental anomalies (15).

9. System (1) for detecting the presence of environmental anomalies in an environment (10), this system (1) comprising: - a first database (33) comprising a reference model (13) of said environment (10) comprising metadata describing a type of environmental anomaly likely to be present in said environment (10); - means for acquiring (2-5) descriptive content (6) of an area of the environment (10); - a detection server (32) to which the acquisition means (2-5) are connected, this detection server (32) comprising a detection module (12) configured to detect the presence of an environmental anomaly in the descriptive content (6) by exploiting the metadata included in the reference model (13) and respectively describing a type of environmental anomaly likely to be present in said environment (10); said system (1) being characterized in that: - the reference model (13) further comprises metadata describing a normal state of the environment (10), i.e. without said environment (10) comprising any anomaly; - the detection module (12) is configured to detect the presence of an environmental anomaly in the descriptive content (6) by exploiting the metadata included in the reference model (13) and describing respectively a type of environmental anomaly likely to be present in said environment (10) and a normal state of the environment (10).

10. System (1) according to claim 9, for which: - the reference model (13) of said environment (10) further comprises cartographic data of the environment (10) in the form of a digital map; - said system further comprises: o a location server (34) arranged to locate the environmental anomaly whose presence has been detected in the descriptive content (6) by the detection server (32), using the environmental cartographic data (10) of the reference model (13); o a second database (35) keeping up to date a list (15) of environmental anomalies detected by the detection module (12) and located by the location server (34).

11. System (1) according to claim 9 or 10, comprising a pre-processing server (31) integrating a pre-processing module (11) configured to sort the descriptive content (6) according to the context of acquisition of this descriptive content (6).

12. System (1) according to claim 11, for which the preprocessing module (11) comprises a conversational agent capable of requesting additional information concerning descriptive content (6).

13. System (1) according to any one of claims 9 to 12, for which the acquisition means (2-5) are on board land, sea or air vehicles.

Images