Method and device for detecting a moving body near a structure

Abstract

The invention relates to a method (100) for detecting a moving body close to a structure, which comprises: - a step (105) of capturing a video stream; - a step (110) of decomposing the video stream into a sequence of images; - a step (125) of subtracting a first image and a second image from the sequence; - a step (150) of detecting a movement which is representative of the presence of a moving body; - a step (155) of defining coordinates which are representative of a search window of the moving body; - a step (170) of constituting a sequence of truncated images by selecting, in the initial sequence of images, an area of the images which are restricted to the search window; and - a step (175) of processing, by means of a trained neural network, the sequence of truncated images.

Description

[0001] METHOD AND DEVICE FOR DETECTING A MOVING BODY IN THE PROXIMITY

[0002] OF A STRUCTURE

[0003] TECHNICAL FIELD OF THE INVENTION

[0004] The present invention relates to a method and device for detecting a moving body near a structure. It applies, in particular, to the field of intelligent management of wind energy production parks.

[0005] STATE OF THE ART

[0006] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been conceived or pursued previously. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section constitute prior art solely because of their inclusion in this section.

[0007] Animal interaction with certain infrastructure can pose a danger to these animals. For example, some species of birds and bats can be killed by the movement of wind turbine blades.

[0008] Conversely, some species can damage existing structures or their uses. For example, the performance of photovoltaic panels can be affected by bird droppings that use these structures as resting places.

[0009] Several patents describe different solutions for keeping these animals away from structures with which they are likely to interact, so as to limit the risks caused by these interactions. Among these patents, the use of sounds is often described. These solutions consist of distributing sound emitters around or on the equipment to be protected. For example, patent US9125394B2 and patent application US20170127664A1 describe specific ultrasound emission systems (whistles, emitters fixed on bars outside the structure).

[0010] However, these approaches share as their main drawback their low capacity to detect the presence of animals.

[0011] To overcome this drawback, some systems implement devices based on artificial intelligence, such as the systems described in patent application CN11 1127516.

[0012] Such systems implement image-to-image stacking of an activation map (from the English "feature map") produced by a neural network.

[0013] Activation maps are the result of various computer processes, making the results difficult for a human operator to interpret and validate. This difficulty often results in a poor design, resulting in a lower ability to detect the presence of a moving body.

[0014] We know the scientific publication of Yoshihashi R. et al., entitled "Bird detection and species classification with time-lapse images around a wind farm: Dataset construction and evaluation" which discloses the development of a database intended to be used in the development of an automatic bird detection system for wind farms. In this document, the step of defining coordinates around a bird is manual and takes place before the bird detection step.

[0015] We also know the commercial and technical brochure by Rioperez A. et al., entitled "DTBird: A self-working system to reduce Bird mortality in wind farms" which discloses a bird detection system for a wind farm whose image processing method is not described.

[0016] We also know of neural networks such as "R-CNN" (acronym for "Region Based Convolutional Neural Networks") and "YOLO" (acronym for "You Only Look Once"). These are algorithms commonly used in the field of region-based object detection. These systems have convolutional layers responsible for extracting the characteristics necessary for detection, such as the shape, color, and / or texture of an object.

[0017] However, these extracted features are not sufficient to detect a moving body in some cases. For example, when a body, at a distance, has a point shape while moving forward or away from the camera such as a flying bird, a drone or an airplane, the body is not detected.

[0018] SUMMARY OF THE INVENTION

[0019] The object of the present invention is to detect moving bodies by means of the sequential analysis of the same zone in a succession of images, called a "search box", initially determined as a function of the detection, on an image, of a shape representative of a moving body.

[0020] BRIEF DESCRIPTION OF THE FIGURES

[0021] Other advantages, aims and particular characteristics of the invention will emerge from the following non-limiting description of at least one particular embodiment of the method and device which are the subject of the present invention, with reference to the appended drawings, in which:

[0022] - figure 1 represents, schematically, and in the form of a flowchart, a particular succession of steps of the method which is the subject of the present invention and

[0023] - figure 2 represents, schematically, a particular embodiment of the device which is the subject of the present invention.

[0024] STATEMENT OF THE INVENTION

[0025] The present invention aims to remedy all or part of these drawbacks.

[0026] To this end, according to a first aspect, the present invention aims at a method for detecting a body in motion near a structure, which comprises: - a step of capturing a video stream representative of the environment near the structure,

[0027] - a step of decomposition, by a calculation device, of the video stream into an initial sequence of at least two successive images,

[0028] - a step of subtraction, by a calculation device, of a first image and a second image from the initial sequence,

[0029] - a step of detection, by a calculation device, of a movement representative of the presence of a body in motion as a function of the subtraction carried out,

[0030] - a step of defining, by a calculation device, coordinates representative of a search box of the moving body as a function of coordinates of the detected movement,

[0031] - a step of constituting, by a calculation device, a sequence of truncated images by selection, in the initial sequence of images, of a zone of images restricted to the search box,

[0032] - a step of processing, by a trained neural network, the sequence of truncated images to recognize a moving body in the sequence of images.

[0033] Thanks to these provisions, the volume of data transmitted to the neural network is limited to a useful part, of a plurality of images, in which a shape which could be that of a moving body has already been detected.

[0034] These provisions make it possible to reduce the number of requests to the neural network per unit of time. This results in several advantages:

[0035] - a reduction in the costs of computing platforms,

[0036] - a possibility of sharing the processing of several cameras on the same platform and

[0037] - the possibility of using particularly deep neural networks (six or more layers of neurons).

[0038] In addition, the sequence of truncated images, formed during the constitution step, is a search box. This search box is similar to a "virtual volume", whose base is the area defined on the initial image around a bird, during the automatic definition step, the volume extending from the base on the following images, as long as a movement is detected. In other words, the area defined on the initial image, upon detection of a movement, is repeated on the following images to form this search box, as long as a movement is detected. The definition frame has the same coordinates on the images following the initial image so as not to analyze this area several times.

[0039] In embodiments, the method which is the subject of the present invention further comprises, when the structure comprises a moving part, a step of transmitting a command to stop the moving part when a moving body is detected.

[0040] In embodiments, the moving body is an animal. In optional embodiments, the method of the present invention comprises:

[0041] - a time integration step, by a calculation device, of results from the processing step,

[0042] - a step of detecting, in the integrated results, a pattern representative of a risk of collision between the moving body and the moving part.

[0043] In embodiments, the step of issuing a command to stop the moving part is carried out based on the detected pattern.

[0044] These embodiments make it possible to determine whether a moving body is moving towards or away from the structure, for example.

[0045] In optional embodiments, the method which is the subject of the present invention comprises a step of emitting an audible and / or visual alert depending on the detected pattern.

[0046] These embodiments make it possible to graduate the level of response to the presence of an animal, initially by frightening it, by emitting a sound, then by stopping the moving part if this proves insufficient.

[0047] In optional embodiments, the method which is the subject of the present invention comprises, downstream of the subtraction step, a thresholding step applied to the result of the subtraction step.

[0048] In optional embodiments, the method which is the subject of the present invention comprises, downstream of the thresholding step, an erosion step applied to the result of the thresholding step.

[0049] In optional embodiments, the method which is the subject of the present invention comprises, downstream of the erosion step, a dilation step applied to the result of the erosion step.

[0050] These embodiments allow the elimination of small and insignificant movements considered as noise.

[0051] In optional embodiments, the method which is the subject of the present invention comprises, downstream of the expansion step, a segmentation step applied to the result of the expansion step.

[0052] In optional embodiments, the network is a convolutional neural network.

[0053] According to a second aspect, the present invention relates to a device for detecting a moving body near a structure, which comprises:

[0054] - a means of capturing a video stream representative of the environment near the structure,

[0055] - a means of decomposing the video stream into an initial sequence of at least two successive images, - a means of subtracting a first image and a second image from the initial sequence,

[0056] - a means of detecting a movement representative of the presence of a moving body as a function of the subtraction carried out,

[0057] - a means of defining representative coordinates of a search box of the moving body as a function of coordinates of the detected movement,

[0058] - a means of constituting a sequence of truncated images by selecting, in the initial sequence of images, an area of ​​the images restricted to the search box,

[0059] - a means of processing, by a trained neural network, the sequence of truncated images to recognize a moving body in the sequence of images.

[0060] In embodiments, the device which is the subject of the invention further comprises, when the structure comprises a moving part, a means for transmitting a command to stop the moving part when a moving body is detected.

[0061] The device which is the subject of the present invention has the same advantages as the method which is the subject of the present invention. Embodiments equivalent to those of the method which is the subject of the present invention can also be implemented.

[0062] DETAILED DESCRIPTION

[0063] This description is given without limitation, each characteristic of an embodiment being able to be combined with any other characteristic of any other embodiment in an advantageous manner.

[0064] Please note that the figures are not to scale.

[0065] As understood from the present description, various inventive concepts may be implemented by one or more methods or devices described below, several examples of which are provided herein. The actions or steps performed in carrying out the method or device may be ordered in any suitable manner. Accordingly, it is possible to construct embodiments in which the actions or steps are performed in a different order than illustrated, which may include performing certain acts simultaneously, even if they are presented as sequential acts in the illustrated embodiments.

[0066] The expression "and / or", as used herein and in the claims, is to be understood to mean 'either or both' of the elements so conjoined, i.e., elements which are present conjunctively in some cases and disjunctively in other cases. Multiple elements listed with "and / or" are to be interpreted in the same way, i.e., 'one or more' of the elements so conjoined. Other elements may optionally be present, other than the elements specifically identified by the "and / or" clause, whether or not they are related to these specifically identified elements.Thus, by way of non-limiting example, a reference to "A and / or B", when used in conjunction with open language such as "comprising" may refer, in one embodiment, to A only (optionally including elements other than B); in another embodiment, to B only (optionally including elements other than A); in yet another embodiment, to both A and B (optionally including other elements); etc.

[0067] As used in this specification and in the claims, the expression "at least one", with reference to a list of one or more elements, is to be understood to mean at least one element selected from one or more elements in the list of elements, but not necessarily including at least one of each element specifically listed in the list of elements and not excluding every combination of elements in the list of elements. This definition also allows for the optional presence of elements other than the specifically identified elements in the list of elements to which the expression "at least one" refers, whether or not related to those specifically identified elements.Thus, by way of non-limiting example, "at least one of A and B" (or, equivalently, "at least one of A or B", or, equivalently, "at least one of A and / or B") may refer, in one embodiment, to at least one, optionally including more than one, A, without B present (and optionally including elements other than B); in another embodiment, to at least one, optionally including more than one, B, without A present (and optionally including elements other than A); in yet another embodiment, to at least one, optionally including more than one, A, and at least one, optionally including more than one, B (and optionally including other elements); etc.

[0068] In the claims, as well as in the description below, all transitional expressions such as "comprising", "including", "carrying", "having", "containing", "involving", "holding", "consisting of", and the like, are to be understood as open, i.e., as meaning including but not limited to. Only the transitional expressions "consisting of" and "consisting essentially of" are to be understood as closed or semi-closed transitional expressions, respectively.

[0069] A structure is any physical element in the environment that can be a human construction, a natural element, or flora. Such a structure is, for example, a building, a field, a river, or a hill.

[0070] A "structure with a moving part" is any real estate structure with an operable moving part. Such a structure is, for example, a wind turbine.

[0071] Figure 1, which is not to scale, shows a schematic view of an embodiment of the method 100 which is the subject of the present invention. This method 100 for detecting a body in motion near a structure comprises:

[0072] - a step 105 of capturing a video stream representative of the environment near the structure,

[0073] - a step 110 of decomposition, by a calculation device, of the video stream into an initial sequence of at least two successive images, - a step 125 of subtraction, by a calculation device, of a first image and a second image from the initial sequence,

[0074] - a step 150 of detection, by a calculation device, of a movement representative of the presence of a body in motion as a function of the subtraction carried out,

[0075] - a step 155 of defining, by a calculation device, coordinates representative of a search box of the moving body as a function of coordinates of the detected movement,

[0076] - a step 170 of constituting, by a calculation device, a sequence of truncated images by selection, in the initial sequence of images, of a zone of images restricted to the search box,

[0077] - a step 175 of processing, by a trained neural network, the sequence of truncated images to recognize a moving body in the sequence of images.

[0078] The capturing step 105 is performed, for example, by an input device 240 as described with reference to FIG. 2. Such an input device 240 is, for example, a video camera operating in the visible or infrared spectrum.

[0079] The decomposition step 110 is carried out, for example, by implementing a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2.

[0080] During this decomposition step 110, the video stream captured during the capture step 105 is decomposed into a succession of independent images, all or part of which is retained. In preferred variants, each image is retained, improving the precision of the analysis carried out.

[0081] In certain variants, the method 100 which is the subject of the present invention, as represented in FIG. 1, comprises a step (not represented) of selective conservation of images resulting from the decomposition step 110, according to a determined frequency (for example, one image in two).

[0082] In certain variants, the method 100 which is the subject of the present invention, as represented in FIG. 1, comprises a step 115 of resizing the stored images.

[0083] The resizing step 115 is carried out, for example, by implementing a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2.

[0084] The dimensions of the images thus resized can be predetermined or determined by a user, via a configuration interface of the graphical interface type or an application interface of the API type.

[0085] The images obtained during the decomposition or resizing step 110 can be implemented during a storage step 195, described below. In certain variants, the method 100 which is the subject of the present invention, as represented in FIG. 1, comprises a step 160 of converting the stored or resized images into gray levels.

[0086] The conversion step 160 is carried out, for example, by implementing a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2.

[0087] During this conversion step 160, an image processing algorithm is for example implemented to convert color images into grayscale images.

[0088] In certain variants, the method 100 which is the subject of the present invention, as represented in FIG. 1, comprises a step 120 of copying the images stored, resized and / or converted into gray levels.

[0089] The copying step 120 is carried out, for example, by implementing a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2.

[0090] During this copying step 120, all or part of the images stored, resized and / or converted into grayscale are copied into a memory 225 of a computer system 205, as described with reference to FIG. 2.

[0091] The subtraction step 125 is carried out, for example, by implementing a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2.

[0092] During this subtraction step 125, a digital subtraction, pixel by pixel, defined by abscissa coordinates, “x”, and ordinates, “y”, is carried out, so as to produce an image corresponding to an intensity of difference between the two images.

[0093] The images selected for this subtraction step 125 are preferably two immediately successive images in the captured image stream.

[0094] In certain variants, the method 100 which is the subject of the present invention, as represented in FIG. 1, comprises a thresholding step 130 on the result of the subtraction step 125.

[0095] The thresholding step 130 is carried out, for example, by implementing a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2.

[0096] During this thresholding step 130, a difference validity limit value, also called a “thresholding value,” is applied to the result of the subtraction step 125. The thresholding value between the two subtracted images is implemented to nullify the differences considered insignificant. The pixel values ​​of the result of the subtraction step 125 that are less than the thresholding value are set to 0, and otherwise, to 1. Consequently, the result of the thresholding step 125 will be a binary image composed of the pixel values ​​of 0 and 1. The validity limit value may be predetermined or determined by a user, via a configuration interface of the graphical interface type or an application interface of the API type.

[0097] In certain variants, the method 100 which is the subject of the present invention, as represented in FIG. 1, comprises a step 135 of erosion on the result of the thresholding step 130.

[0098] The erosion step 135 is carried out, for example, by implementing a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2.

[0099] During this erosion step 135, all pixels for which the structuring element centered on this pixel touches the outside of the structure are searched. In other words, all particles less than or equal to the size of the structuring element, also called the "erosion kernel", are set to zero. The result is a trimmed structure.

[0100] In certain variants, the method 100 which is the subject of the present invention, as represented in FIG. 1, comprises a step 140 of expansion on the result of the erosion step 135.

[0101] The expansion step 140 is carried out, for example, by implementing a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2.

[0102] During this dilation step 140, a structuring element is moved over each pixel of the image to see if the structuring element intersects the structure of interest. The result is a structure that is larger than the original structure. Depending on the size of the structuring element, some particles may become connected, and some holes may disappear.

[0103] In certain variants, the method 100 which is the subject of the present invention, as represented in FIG. 1, comprises a segmentation step 145 on the result of the expansion step 140.

[0104] The segmentation step 145 is carried out, for example, by implementing a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2.

[0105] During this segmentation step 145, an algorithm for identifying structures of interest in the image, also called connected component analysis, connected component labeling, blob extraction, or region labeling, is implemented. This algorithm implements a connected component identification approach. Each individual connected component, which is a group of connected pixels, is grouped or labeled as a blob. A blob is then a region formed from a set of spatially connected pixels. Connected component analysis is then an algorithmic application of graph theory used to determine the connectivity of "blob"-like regions in a binary image. The result is a labeled image, with each label corresponding to a region.

[0106] The detection step 150 is carried out, for example, by implementing a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2. This detection step 150 corresponds, for example, to the implementation of an algorithm which, as a function of the result of the segmentation step 145, detects the presence of a shape likely to be a moving body. This detection can be carried out as a function of a confidence value in the segmentation carried out, for example. Shape similarity criteria can be predetermined or determined by a user, via a configuration interface such as a graphical user interface or an application interface such as an API.

[0107] The definition step 155 is carried out, for example, by the implementation of a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2.

[0108] During definition step 155, a centroid of abscissa coordinates, "xc", and ordinates, "yc", of a moving body is defined.

[0109] This definition step 155 corresponds, for example, to the implementation of an algorithm which, based on the dimensions of the shape detected during the detection step 150, determines coordinates to form a search box. A search box results, for example, from dividing the initial image into squares of 120 x 120 pixels (or other dimensions adapted to the use case). In other words, a search box can be a sub-image, corresponding to a truncated image, of 120 x 120 pixels in which the centroid of the moving body is located. If in one of these squares, at least one shape likely to be a moving body is detected, this square is selected to be analyzed by the neural network. This approach avoids sending the same area to be analyzed several times if several shapes likely to be a moving body are detected very close to each other.

[0110] In certain variants, the method 100 which is the subject of the present invention, as represented in FIG. 1, comprises, in parallel with the copying step 120, a step 165 of stacking images resulting from the decomposition step 110, the resizing step 115 or the conversion step 160.

[0111] The stacking step 165 is carried out, for example, by implementing a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2.

[0112] During the stacking step 165, a memory 225 of a computer system 205, as described with reference to FIG. 2, is for example implemented to store and index, in the capture order, the images thus stacked.

[0113] A stacked image is the result of preserving the darkest pixels in a pixel-by-pixel comparison of the two grayscale images being the old stacked image from the previous iteration and the current image resulting from conversion step 160.

[0114] The constitution step 170 is carried out, for example, by the implementation of a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2. During this constitution step 170, the search box defined during step 155 is applied to a succession of captured images, resulting from the decomposition step 110 or from an optional downstream step. This application corresponds to a truncation of the initial image to obtain an image of reduced dimensions around an area of ​​interest corresponding to the area in which a suspicious shape was detected during the detection step 150. The search boxes are clipped from the stacked image defined in step 155. The search box containing the given centroid of abscissa coordinates, "xc", and ordinates, "yc", is defined in step 155.The search box is then clipped from the corresponding stacked image.

[0115] At the end of the building step 170, a sequence of truncated images from the stacked image is obtained. In other words, the search box cropped from the stacked image is introduced as input into the convolutional neural network. This sequence of truncated images serves as input for processing by a model trained by machine learning.

[0116] As understood, in variants, the stream of truncated images thus implemented corresponds to a stream of color or grayscale images.

[0117] The processing step 175 is carried out, for example, by implementing a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2.

[0118] During this processing step 175, the sequence of truncated images is provided to a model trained by machine learning to provide a prediction (or a “class”) relating to the presence or absence of a moving body in the captured video stream. Such a prediction is preferably made per image. For example, such a prediction may relate to the presence of several moving bodies per image at different positions on the image.

[0119] In preferred variants, the trained model corresponds to the result of automatic learning carried out by a convolutional neural network.

[0120] In certain variants, the method 100 which is the subject of the present invention, as represented in FIG. 1, a step 180 of temporal integration of the processing carried out.

[0121] The integration step 180 is carried out, for example, by the implementation of a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2.

[0122] During this integration step 180, a succession of results, organized according to an execution timestamp, are accumulated. Such an integration corresponds, for example, to a succession of class predictions, corresponding for example to a succession of successive detections of shapes representative of a body in motion near the structure. In variants, the result of this integration step is provided to a storage step 195.

[0123] In certain variants, the method 100 which is the subject of the present invention, as represented in FIG. 1, a step 195 of storage on detection.

[0124] The storage step 195 is carried out, for example, by the implementation of a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2 associated with a memory 225.

[0125] This storage step 195 makes it possible to obtain qualitative and quantitative traceability of the detection and stops of moving parts, by visual inspection for example.

[0126] In embodiments, the method 100 which is the subject of the present invention, as represented in FIG. 1, further comprises, when the structure comprises a mobile part, a step 190 of transmitting a command to stop the mobile part when a moving body is detected.

[0127] In certain variants, the method 100 which is the subject of the present invention, as represented in FIG. 1, comprises a step 190 of transmitting a command to stop the mobile part.

[0128] Step 190 of issuing a stop command implements, for example, a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2, associated with an interface 260 for wired or wireless communication with the structure. The exact nature of this command depends on the signals expected by the moving part. This command can be analog or digital.

[0129] In embodiments, the moving body is an animal.

[0130] A moving body can be any element that can come into contact with a moving part. Such a moving body is, for example, an animal or an object. Preferably, the moving body is a bird or a bat.

[0131] In certain variants, the method 100 which is the subject of the present invention, as represented in FIG. 1, comprises a step 195 of emitting an audible and / or visual alert when a moving body is detected.

[0132] Such an audible alert is emitted, for example, by a loudspeaker located near the structure.

[0133] Such a visual alert is emitted, for example, by a flash lamp located near the structure.

[0134] The emission of an audible alert and / or a stop command may be sequential and determined as a function of the distance from the moving body or the direction of movement of the moving body. For example, if the moving body is at a distance greater than a predetermined limit distance, an audible alert is emitted while if the moving body is at a distance less than said limit distance, a stop command is emitted. In certain variants, the method 100 which is the subject of the present invention, as represented in FIG. 1, a step 185 of pattern detection.

[0135] The detection step 185 is carried out, for example, by implementing a set of computer instructions, forming computer software, executed by a computing device, such as a processor 210 as described with reference to FIG. 2.

[0136] During this detection step 185, a pattern representative of a movement of a moving body is detected, such as a distance, a speed or a movement of approach or distance from the structure. Depending on the pattern detected, the processing carried out may vary.

[0137] Thus, as understood, the method 100 which is the subject of the present invention may comprise:

[0138] - a step 180 of temporal integration, by a calculation device, of results of the processing step,

[0139] - a step 185 of detecting, in the integrated results, a pattern representative of a risk of collision between the moving body and the moving part.

[0140] In embodiments, step 190 of issuing a command to stop the moving part can be performed based on the detected pattern.

[0141] In embodiments, the network is a convolutional neural network.

[0142] In variants, the method 100 which is the subject of the present invention comprises a step 300 of training a neural network.

[0143] This training step 300 implements, for example, a set of computer instructions executed by a computing device, such as a processor 210 as described with reference to FIG. 2.

[0144] To carry out such training, a training data set is assembled, such a set comprising a plurality of image sequences, corresponding to the truncated images implemented by the trained model, associated with a predetermined class (“detection of a moving body”, “non-detection of a moving body”, for example).

[0145] An example of such a training dataset is a batch of images (120 x 120 pixels, grayscale, JPG compression) stored in two folders. One contains nearly 50,000 images of birds in flight, especially raptors, corresponding to a positive class of moving body detection. The other contains nearly 50,000 images of other moving objects (wind turbine blades, airplane, drone, clouds, insects, vegetation, etc.) corresponding to a negative class of moving body detection.

[0146] This training dataset is provided to a machine learning architecture, such as a convolutional neural network.

[0147] Such a convolutional neural network has, for example, five successive layers of artificial neurons. In variants, such a convolutional neural network may include five convolutional layers, followed by max pooling, batch normalization, and dropout layers, and dense layers for classification.

[0148] The result of this training step 300 is the provision of a trained model, capable of assigning a class to a sequence of images.

[0149] In Figure 2, we observe schematically a particular embodiment of the device 200 for detecting a body in motion near a structure, which comprises:

[0150] - a means 240 for capturing a video stream representative of the environment near the structure,

[0151] - a means 210 for decomposing the video stream into an initial sequence of at least two successive images,

[0152] - a means 210 for subtracting a first image and a second image from the initial sequence,

[0153] - a means 210 for detecting a movement representative of the presence of a moving body as a function of the subtraction carried out,

[0154] - a means 210 for defining representative coordinates of a search box of the moving body as a function of coordinates of the detected movement,

[0155] - a means 210 for constituting a sequence of truncated images by selecting, in the initial sequence of images, an area of ​​the images restricted to the search box,

[0156] - a means 210 for processing, by a trained neural network, the sequence of truncated images to recognize a moving body in the sequence of images.

[0157] In variants, the device 200 which is the subject of the invention may further comprise, when the structure comprises a mobile part, a means 260 for transmitting a command to stop the mobile part when a moving body is detected.

[0158] Variants of implementation of the means of the device 200 are presented with reference to FIG. 1. Such means are also described below.

[0159] Generally, shown in Figure 2, which is not to scale, is a block diagram illustrating an exemplary computer system with which an embodiment may be implemented. In the example of Figure 2, a computer system 205 and instructions for implementing the disclosed technologies in hardware, software, or a combination of hardware and software are shown schematically, for example, as boxes and circles, at the same level of detail that is commonly used by those of ordinary skill in the art to which this disclosure relates to communicate about computer architecture and computer system implementations.

[0160] The computer system 205 includes an input / output (I / O) subsystem 220 that may include a bus and / or one or more other communication mechanisms for communicating information and / or instructions between components of the computer system 205 over electronic signal paths. The input / output subsystem 220 may include an input / output controller, a memory controller, and at least one input / output port. The electronic signal paths are shown schematically in the drawings, for example, as lines, one-way arrows, or two-way arrows.

[0161] At least one processor 210, or computing device, is coupled to the I / O subsystem 220 to process information and instructions. The processor 210 may include, for example, a general-purpose microprocessor or microcontroller and / or a special-purpose microprocessor such as an embedded system or graphics processing unit (GPU) or a digital signal processor or an ARM processor. The processor 210 may include an integrated arithmetic logic unit (ALU) or may be coupled to a separate ALU.

[0162] The computer system 205 includes one or more memories 225, such as a main memory, which is coupled to the I / O subsystem 220 to electronically digitally store data and instructions to be executed by the processor 210. The memory 225 may include volatile memory such as various forms of random access memory (RAM) or any other dynamic storage device. The memory 225 may also be used to store temporary variables or other intermediate information during the execution of the instructions to be executed by the processor 210. Such instructions, when stored in a non-transitory computer-readable storage medium accessible to the processor 210, may transform the computer system 205 into a special purpose machine that is customized to perform the operations specified in the instructions.

[0163] The computer system 205 further includes non-volatile memory such as a read-only memory (ROM) 230 or other static storage device coupled to the I / O subsystem 220 for storing information and instructions for the processor 210. The ROM 230 may include various forms of programmable ROM (PROM) such as erasable PROM (EPROM) or electrically erasable PROM (EEPROM). A persistent storage unit 215 may include various forms of non-volatile random access memory (NVRAM), such as FLASH memory, or solid state storage, a magnetic disk, or an optical disk such as a CD-ROM or DVD-ROM and may be coupled to the I / O subsystem 220 for storing information and instructions.Memory 215 is an example of a non-transitory computer-readable medium that can be used to store instructions and data that, when executed by processor 210, cause execution of computer-implemented methods for performing the techniques of this document.

[0164] The instructions in memory 225, ROM 230, or storage 215 may comprise one or more sets of instructions that are organized into modules, methods, objects, functions, routines, or calls. The instructions may be organized as one or more computer programs, operating system services, or application programs, including mobile applications. The instructions may comprise an operating system and / or system software; one or more libraries to support multimedia, programming, or other functions; data protocol instructions or stacks to implement TCP / IP, HTTP, or other communication protocols; file format processing instructions to parse or render files encoded using HTML, XML, JPEG, MPEG, or PNG;user interface instructions for rendering or interpreting commands for a graphical user interface (GUI), command-line interface, or text-based user interface;application software such as an office suite, Internet access applications, design and manufacturing applications, graphics applications, audio applications, software engineering applications, educational applications, games, or miscellaneous applications. The instructions may implement a web server, a web application server, or a web client. The instructions may be organized as a presentation layer, an application layer, and a data storage layer such as a relational database system using Structured Query Language (SQL) or no SQL, an object store, a graph database, a flat file system, or other data storage.;

[0165] The computer system 205 may be coupled via the I / O subsystem 220 to at least one output device 235. In one embodiment, the output device 235 is a digital computer display. Exemplary displays that may be used in various embodiments include a touchscreen or a light-emitting diode (LED) display or a liquid crystal display (LCD) or an e-paper display. The computer system 205 may include one or more other types of output devices 235, instead of or in addition to a display device. Examples of other output devices 235 include printers, ticket printers, plotters, projectors, sound cards or video cards, speakers, buzzers or piezoelectric devices or other audible devices, LED or LCD lamps or indicators, haptic devices, actuators, or servos.

[0166] At least one input device 240 is coupled to the I / O subsystem 220 to communicate signals, data, command selections, or gestures to the processor 210. Examples of input devices 240 include touchscreens, microphones, digital still and video cameras, alphanumeric and other keys, keyboards, graphics tablets, image scanners, joysticks, clocks, switches, buttons, dials, sliders.

[0167] Another type of input device is a control device 245, which may perform cursor control or other automated control functions such as navigating a graphical interface on a display screen, alternatively or in addition to input functions. The control device 245 may be a touchpad, mouse, trackball, or cursor direction keys to communicate direction information and control selections to the processor 210 and to control movement of the cursor on the display 235. The input device may have at least two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), which allows the device to specify positions in a plane.Another type of input device is a wired, wireless, or optical control device, such as a joystick, wand, console, steering wheel, pedal, gear shift mechanism, or other type of control device. An input device 240 may include a combination of several different input devices, such as a video camera and a depth sensor.

[0168] In another embodiment, the computer system 205 may include an Internet of Things (IoT) device in which one or more of the output device 235, the input device 240, and the control device 245 are omitted. Or, in such an embodiment, the input device 240 may include one or more cameras, motion detectors, thermometers, microphones, seismic detectors, other sensors or detectors, measuring devices, or encoders, and the output device 235 may include a special-purpose display such as a single-line LED or LCD display, one or more indicators, a display panel, a meter, a valve, a solenoid, an actuator, or a servomotor.

[0169] The output device 235 may include hardware, software, firmware, and interfaces to generate position report packets, notifications, pulse or heartbeat signals, or other recurring data transmissions that specify a position of the computer system 205, alone or in combination with other application-specific data, directed to the host 250 or server 255.

[0170] The computer system 205 may implement the techniques described herein using custom hardwired logic, at least one ASIC (Application-specific integrated circuit) or FPGA (Field-programmable gate array), firmware, and / or program instructions or logic that, when loaded and used or executed in combination with the computer system, cause or program the computer system to operate as a special-purpose machine. In one embodiment, the techniques described herein are executed by the computer system 205 in response to the processor 210 executing at least one sequence of at least one instruction contained in the main memory 225.These instructions may be read into main memory 225 from another storage medium, such as memory 215. Execution of the instruction sequences contained in main memory 225 causes processor 210 to execute the process steps described herein. In other embodiments, hard-wired circuits may be used instead of or in combination with software instructions.

[0171] The term "storage medium," as used herein, means any non-transitory medium that stores data and / or instructions that enable a machine to operate in a specific manner. These storage media may include non-volatile media and / or volatile media. Non-volatile media include, for example, optical or magnetic disks, such as memory 215. Volatile media include dynamic memory, such as memory 225. Common forms of storage media include, for example, a hard disk drive, a solid-state drive, a flash drive, a magnetic data storage medium, any optical or physical data storage medium, a memory chip, etc.

[0172] Storage media are distinct from transmission media, but may be used in conjunction with them. Transmission media are involved in the transfer of information between storage media. For example, transmission media include coaxial cables, copper wires, and optical fibers, including the wires that make up a bus of the I / O subsystem 220. Transmission media may also take the form of acoustic or light waves, such as those generated during radio and infrared data communications.

[0173] Various forms of media may be involved in transporting at least one sequence of at least one instruction to the processor 210 for execution. For example, the instructions may initially be transported on a magnetic disk or solid-state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and send the instructions over a communications link such as a fiber optic or coaxial cable or a telephone line using a modem. A modem or router local to the computer system 205 may receive the data over the communications link and convert the data into a format that can be read by the computer system 205.For example, a receiver such as a radio frequency antenna or an infrared detector may receive the data carried in a wireless or optical signal and suitable circuitry may provide the data to the I / O subsystem 220, for example by placing the data on a bus. The I / O subsystem 220 transports the data to the memory 225, from which the processor 210 retrieves and executes the instructions. The instructions received by the memory 225 may optionally be stored on the memory 215 before or after execution by the processor 210.

[0174] The computer system 205 may also include a communication interface 260 coupled to a bus 220. The communication interface 260 provides a bidirectional data communication coupling to the one or more network links 265 that are directly or indirectly connected to at least one communication network, such as a network 270 or a public or private cloud on the Internet. For example, the communication interface 260 may be an Ethernet network interface, an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem for providing a data communication connection to a corresponding type of communication line, for example, an Ethernet cable or a metallic cable of any type or a fiber optic line or a telephone line.Network 270 broadly represents a local area network (LAN), a wide area network (WAN), a campus network, an Internet network, or any combination thereof. Communications interface 260 may include a LAN card to provide a data communications connection to a compatible LAN, or a cellular radiotelephone interface that is wired to send or receive cellular data according to cellular radiotelephone wireless network standards, or a satellite radio interface that is wired to send or receive digital data according to satellite wireless network standards. In any such implementation, communications interface 260 sends and receives electrical, electromagnetic, or optical signals over signal paths that carry digital data streams representing various types of information.

[0175] The network link 265 typically provides electrical, electromagnetic, or optical data communication directly or via at least one network to other data devices, using, for example, satellite, cellular, W-Fi, or BLUETOOTH technology. For example, the network link 265 may provide a connection through a network 270 to a host computer 250.

[0176] Further, the network link 265 may provide a connection via the network 270 or to other computing devices via interconnecting devices and / or computers that are operated by an Internet Service Provider (ISP) 275. The ISP 275 provides data communication services via a global packet data communication network represented by the Internet 280. A server computer 255 may be coupled to the Internet 280. The server 255 broadly represents any computer, data center, virtual machine or virtual computing instance with or without a hypervisor, or computer running a containerized program system such as DOCKER or KUBERNETES. The server 255 may represent an electronic digital service that is implemented using more than one computer or instance and that is accessed and used by transmitting web service requests,Uniform Resource Locator (URL) strings with parameters in Hypertext Transfer Protocol (HTTP) payloads, Application Programming Interface (API) calls, application service calls, or other service calls. The computer system 205 and the server 255 may form elements of a distributed computing system that includes other computers, a processing cluster, a server farm, or other organization of computers that cooperate to perform tasks or run applications or services. The server 255 may have one or more sets of instructions that are organized as modules, methods, objects, functions,routines or calls. Instructions may be organized as one or more computer programs, operating system services, or application programs, including mobile applications. Instructions may include an operating system and / or system software; one or more libraries to support multimedia, programming, or other functions; instructions or data protocol stacks to implement TCP / IP (for "Transmission control protocol / Internet protocol"), HTTP, or other communication protocols; file format processing instructions to parse or render files encoded using HTML (for "Hypertext markup language"), XML (for "Extensible markup language"),JPEG (for Joint Photographie Experts Group), MPEG (for Moving picture experts group), or PNG (for Portable Networks Graphie); user interface instructions for rendering or interpreting commands for a graphical user interface (GUI), command-line interface, or text-based user interface; application software such as an office suite, Internet access applications, design and manufacturing applications, graphics applications, audio applications, software engineering applications, educational applications, games, or miscellaneous applications. The server 255 may include a web application server that hosts a presentation layer,an application layer and a data storage layer such as a relational database system using Structured Query Language (SQL) or no SQL, an object store, a graph database, a flat file system, or any other data storage.

[0177] The computer system 205 may send messages and receive data and instructions, including program code, via the network(s), network link 265, and communications interface 260. In the Internet example, a server 255 may transmit requested code for an application program via the Internet 280, ISP 275, local area network 270, and communications interface 260. The received code may be executed by the processor 210 as it is received, and / or stored in memory 215, or other non-volatile memory for later execution.

[0178] The execution of instructions as described in this section may implement a process as an instance of a running computer program consisting of program code and its current activity. Depending on the operating system (OS), a process may consist of multiple threads that execute instructions concurrently. In this context, a computer program is a passive collection of instructions, while a process may be the actual execution of those instructions. Multiple processes may be associated with the same program; for example, having multiple instances of the same program open often means that more than one process is running. Multitasking may be implemented to allow multiple processes to share the processor 210.Although each processor 210 or processor core executes only one task at a time, the computer system 205 may be programmed to implement multitasking to allow each processor to switch between currently executing tasks without having to wait for each task to complete. In one embodiment, the switches may be performed when tasks perform input / output operations, when a task indicates that it can be switched, or upon hardware interrupts. Time sharing may be implemented to allow rapid response to user interactive applications by rapidly performing context switches to give the appearance of simultaneous execution of multiple processes.In one embodiment, for security and reliability reasons, an operating system may prevent direct communication between independent processes, by providing strictly mediated and controlled interprocess communication functionality.

Claims

CLAIMS 1. Method (100) for detecting a moving body near a structure, characterized in that it comprises: - a step (105) of capturing a video stream representative of the environment near the structure, - a step (110) of decomposition, by a calculation device, of the video stream into an initial sequence of at least two successive images, - a step (125) of subtraction, by a calculation device, of a first image and a second image from the initial sequence, - a step (150) of detecting, by a calculation device, a movement representative of the presence of a body in motion as a function of the subtraction carried out, - a step (155) of defining, by a calculation device, coordinates representative of a search box of the moving body as a function of coordinates of the detected movement, - a step (170) of constituting, by a calculation device, a sequence of truncated images by selection, in the initial sequence of images, of a zone of images restricted to the search box, - a step (175) of processing, by a trained neural network, the sequence of truncated images to recognize a moving body in the sequence of images.

2. Method (100) according to claim 1, which further comprises, when the structure comprises a moving part, a step (190) of transmitting a command to stop the moving part when a moving body is detected.

3. Method (100) according to one of claims 1 or 2, in which the moving body is an animal.

4. Method (100) according to claim 1, which comprises: - a step (180) of temporal integration, by a calculation device, of results of the processing step, - a step (185) of detecting, in the integrated results, a pattern representative of a risk of collision between the moving body and the moving part.

5. Method (100) according to claim 4, in which the step (190) of issuing a command to stop the moving part is carried out as a function of the detected pattern.

6. Method (100) according to one of claims 1 to 5, which comprises a step (195) of emitting an audible and / or visual alert depending on the detected pattern.

7. Method (100) according to one of claims 1 to 6, which comprises, downstream of the subtraction step (125), a thresholding step (130) applied to the result of the subtraction step.

8. Method (100) according to claim 7, which comprises, downstream of the thresholding step (130), an erosion step (135) applied to the result of the thresholding step.

9. Method (100) according to claim 8, which comprises, downstream of the erosion step (135), a dilation step (140) applied to the result of the erosion step.

10. Method (100) according to claim 9, which comprises, downstream of the expansion step (140), a segmentation step (145) applied to the result of the expansion step.

11. Method (100) according to one of claims 1 to 10, in which the network is a convolutional neural network.

12. Device (200) for detecting a moving body near a structure, characterized in that it comprises: - a means (240) for capturing a video stream representative of the environment near the structure, - a means (210) for decomposing the video stream into an initial sequence of at least two successive images, - means (210) for subtracting a first image and a second image from the initial sequence, - a means (210) for detecting a movement representative of the presence of a moving body as a function of the subtraction carried out, - a means (210) for defining coordinates representative of a search box of the moving body as a function of coordinates of the detected movement, - a means (210) for constituting a sequence of truncated images by selecting, in the initial sequence of images, an area of ​​the images restricted to the search box, - means (210) for processing, by a trained neural network, the sequence of truncated images to recognize a moving body in the sequence of images.

13. Device (200) according to claim 12, which further comprises, when the structure comprises a movable part, means (260) for transmitting a command to stop the movable part when a moving body is detected.

Images