Event camera based object drop detection method, apparatus, medium, and device

Abstract

Embodiments of the present application provide an event camera-based object falling detection method, device, medium and equipment. The method comprises: acquiring an event stream output by an event camera, the event stream containing event information of a plurality of continuous events, the event information including occurrence coordinates and occurrence time of an event; generating a feature map corresponding to the event stream according to the occurrence coordinates and occurrence time corresponding to each event, the feature map describing time characteristics and / or position characteristics of the events contained in the event stream; performing clustering recognition on feature values corresponding to each coordinate in the feature map to determine position information of at least one target region of a target object existing in the feature map; and determining a motion trajectory of the target object based on position information of each target region in the feature maps corresponding to at least one continuous event stream. The technical solution of the embodiments of the present application improves the accuracy of falling object recognition and ensures the recognition effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to an object falling detection method and device based on event camera, medium and equipment. BACKGROUND

[0002] The target detection of falling object refers to detecting the object falling freely from high altitude to the ground, and restoring and visualizing the trajectory of the target object. In the current technical solution, the target detection of falling object is usually based on a traditional camera. When detecting, the traditional camera needs to be exposed for a certain period of time. If the speed of the moving object is fast, it will produce obvious motion blur, lose more information of the appearance and motion of the object, and it is difficult to identify. In addition, the traditional camera also has the problem of insufficient contrast, which makes it difficult to effectively obtain a clear image in a challenging lighting environment, hindering the acquisition of information of the target area. Therefore, how to improve the accuracy of falling object recognition and ensure the recognition effect has become a technical problem to be solved. SUMMARY

[0003] Embodiments of the present application provide an object falling detection method and device based on event camera, medium and equipment, which can at least improve the accuracy of falling object recognition and ensure the recognition effect.

[0004] Other characteristics and advantages of the present application will become apparent from the following detailed description, or will be learned by practice of the present application.

[0005] According to an aspect of an embodiment of the present application, an object falling detection method based on an event camera is provided, which comprises:

[0006] Obtaining an event stream output by an event camera, the event stream containing event information of a plurality of continuous events, the event information including occurrence coordinates and occurrence time of the events;

[0007] Generating a feature map corresponding to the event stream according to the occurrence coordinates and occurrence time of each event, the feature map describing the time characteristics and / or position characteristics of the events contained in the event stream;

[0008] Performing clustering recognition on the feature values corresponding to each coordinate in the feature map to determine the position information of at least one target area in which a target object exists in the feature map;

[0009] Determining the motion trajectory of the target object based on the position information of each target area in the feature map corresponding to at least one continuous event stream.

[0010] According to an aspect of an embodiment of the present application, an object falling detection device based on an event camera is provided, which comprises:

[0011] an acquisition module configured to acquire an event stream of an event camera output, the event stream containing event information of a plurality of continuous events, the event information comprising occurrence coordinates and occurrence time of the events;

[0012] a feature map generation module configured to generate a feature map corresponding to the event stream according to the occurrence coordinates and the occurrence time of each of the events, the feature map describing time characteristics and / or position characteristics of the events contained in the event stream;

[0013] a moving target identification module configured to perform clustering identification according to feature values corresponding to each coordinate in the feature map, to determine position information of at least one target region in which a target object exists in the feature map;

[0014] a trajectory processing module configured to determine a motion trajectory of the target object based on position information of each target region in the feature map corresponding to at least one continuous event stream.

[0015] According to an aspect of some embodiments of the present application, there is provided a computer readable medium having stored thereon a computer program which, when executed by a processor, implements the event camera based object falling detection method as described in the above embodiments.

[0016] According to an aspect of some embodiments of the present application, there is provided an electronic device comprising: one or more processors; and a storage device configured to store one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the event camera based object falling detection method as described in the above embodiments.

[0017] According to an aspect of some embodiments of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the event camera based object falling detection method as described in the above embodiments.

[0018] In some embodiments provided in the present application, an event stream output by an event camera is acquired, the event stream containing a plurality of continuous events, and the event information of each event includes the occurrence coordinates and the occurrence time of the event. A feature map corresponding to the event stream is generated according to the occurrence coordinates and the occurrence time of each event. The feature map can be used to describe the time feature and / or the position feature of the events contained in the event stream. Thus, the position information of at least one target region of the target object in the feature map is determined by clustering and identifying each coordinate corresponding to the feature value in the feature map. Then, the motion trajectory of the target object is determined based on the position information of each target region in the feature map corresponding to the continuous at least one event stream. Thus, the motion trajectory of the falling object can be quickly and accurately identified, and the identification effect is ensured.

[0019] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application. It is apparent that the accompanying drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. In the drawings:

[0021] Figure 1 A flowchart of an event camera-based object falling detection method according to an embodiment of the present application is shown;

[0022] Figure 2 A block diagram of an event camera-based object falling detection device according to an embodiment of the present application is shown;

[0023] Figure 3 A structural schematic diagram of a computer system of an electronic device suitable for implementing embodiments of the present application is shown. DETAILED DESCRIPTION

[0024] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example implementations to those skilled in the art.

[0025] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the application can be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known structures, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the application.

[0026] The block diagrams in the drawings show only the functionality of the features and can not imply that the functionality must be implemented in the order shown. In some embodiments, functional elements can be implemented in hardware, software, or a combination of both hardware and software. Further, the described features can be combined in any suitable manner in one or more embodiments.

[0027] The flow diagrams shown in the various figures, which can also be referred to as flow charts, are examples of sequences of operations that can be performed by, for example, software programs, hardware, or a combination of both. In the context of software, the operations can be stored on or transmitted over computer-readable media as computer-executable instructions, such as in the form of code segments, program commands, or the like. In the context of hardware, the operations can be implemented by circuitry, such as one or more integrated circuits, ASICs, or the like. In some embodiments, the operations can be performed in an order different than those described, and / or operations can be omitted, repeated, or combined.

[0028] Figure 1 A flow diagram of an event camera based object fall detection method according to an embodiment of the application is shown. The method can be applied to a terminal device, including but not limited to one or more of a smartphone, a tablet computer, a portable computer, a desktop computer, and can also be applied to a server, such as a physical server or a cloud server, etc., which is not particularly limited by the present application.

[0029] Reference Figure 1 As shown, the event camera based object fall detection method includes at least steps S110 to S140, which are described in detail as follows (the method is taken as an example applied to a terminal device, hereinafter referred to as a terminal):

[0030] In step S110, an event stream output by an event camera is obtained, the event stream containing event information of a plurality of consecutive events, the event information including occurrence coordinates and occurrence time of the event.

[0031] In this embodiment, the raw data output by the event camera is an event, and a single event can be represented as: e = [x, y, t, p], where x, y represent the coordinates of the event e, t is the time of the event e, and p is the polarity of the event e, p = 1 indicating an increase in brightness, and p = -1 indicating a decrease in brightness. When the change in the logarithmic value of the brightness of a pixel point after the last event response exceeds a certain threshold c, the event camera will generate an event at this pixel position.

[0032] The terminal can acquire the event stream output by the event camera in a fixed time length or a fixed number of events, for example, events in thirty milliseconds as an event stream, or ten thousand consecutive events as an event stream, etc. Those skilled in the art can select a corresponding event stream acquisition method according to prior experience, and the present application does not make special limitations thereto.

[0033] It is worth noting that, in order to make the event information in the event effective in subsequent processing, there can be some overlap between the events in adjacent event streams, and the overlap ratio can range from 1% to 99%, for example, 1%, 20%, 50%, 80%, or 90%, etc. The above numbers are only exemplary, and the present application does not make special limitations thereto.

[0034] In step S120, a feature map corresponding to the event stream is generated according to the occurrence coordinates and occurrence time of each event.

[0035] In this embodiment, the terminal can obtain the occurrence coordinates and occurrence time of each event from the event stream, determine the time and number of events occurring at each coordinate position, and generate a feature map corresponding to the event stream, which can be used to describe the time and / or position characteristics of the events contained in the event stream.

[0036] It should be noted that, for the same event stream, one or more feature maps can be generated, and different feature maps can describe different characteristics of the event stream.

[0037] In an embodiment of the present application, when generating the feature map, the terminal can count the number of events occurring at the same coordinate and the corresponding time information according to the occurrence coordinates and occurrence time of each event. Then, the terminal can generate a feature map of the average time accumulation corresponding to the event stream according to the number of events occurring at the same coordinate and the sum of the time information of the events occurring.

[0038] Specifically, the feature value at each coordinate in the feature map of the average time accumulation can be calculated by the following formula:

[0039]

[0040] Where L is the number of events occurring at (i, j) pixel, t is the time information, and ξ is the set of events at (i, j) pixel. ij ij

[0041] It should be noted that t can be the time information after normalization processing, which can be normalized according to the following formula: t = (T - T0) / (T1 - T0)​​原始 -T min ) / (T max -T min ), wherein T 原始 is the occurrence time of the event, T max is the timestamp of the last occurred event in the event stream, and T min is the timestamp of the earliest occurred event in the event stream.

[0042] Alternatively, the terminal can generate a feature map of the latest time update corresponding to the event stream according to the latest time of the events occurring on the same coordinate.

[0043] Specifically, the feature value of each coordinate in the feature map of the latest time accumulation can be calculated according to the following formula:

[0044] T ij = max(t):t∈ξ ij ,

[0045] wherein t is time information, and for the same reason, it can also be time after the above normalization processing, and ξ ij is the set of events on the (i, j) pixel.

[0046] Thus, the feature maps obtained by the above two processing methods are statistical features related to the triggering time of the events. In an example, after the feature value of each coordinate is calculated, the determined feature value can be normalized for subsequent processing.

[0047] Specifically, the feature value can be normalized according to the following formula:

[0048]

[0049] wherein Δt is the time span of the event stream, i.e., the maximum timestamp minus the minimum timestamp in the event stream.

[0050] In an embodiment of the present application, after the feature map of the average time accumulation and / or the feature map of the latest time accumulation corresponding to the event stream is generated, the method further comprises:

[0051] According to the feature value corresponding to each coordinate in the feature map, the average value of the timestamps of the occurred events, and the time span of the event stream, removing the background and the noise points from the feature map.

[0052] In this embodiment, the background and part of the noise points in the feature map can be removed by the following formula:

[0053]

[0054] wherein T avg(i,j) represents the average value of the event timestamp on the (i,j) pixel or in a certain neighborhood range of the (i,j) pixel, and the neighborhood range can be adjusted according to the actual application scenario. When p(i,j)≤0, it is considered that the event is generated by the background; when p(i,j)>λ, it is considered that the event is generated by the moving object, and λ is a threshold value set according to the environment, λ>0.

[0055] Therefore, by removing the background and noise points in the feature map in the above manner, the accuracy of the subsequent clustering result can be improved, and the accuracy of the subsequent motion trajectory of the target object is further improved.

[0056] Alternatively, the terminal can generate a 0 / 1 feature map corresponding to the event stream according to the coordinates of the events that have occurred, wherein the coordinates corresponding to the positions where the events have occurred are 1, and otherwise are 0.

[0057] Further alternatively, the terminal can generate a feature map of the number of events accumulated according to the number of events occurring at the same coordinate. Specifically, the number of events occurring at each pixel point can be multiplied by a fixed amplification coefficient to obtain the feature value of the corresponding position in the feature map, and the specific formula is as follows:

[0058] T ij =L ij *μ,

[0059] Wherein, L ij is the number of events generated at the (i,j) pixel, and μ is the amplification coefficient, which can be pre-set by those skilled in the art.

[0060] It should be understood that those skilled in the art can determine the generation method of the feature map according to actual implementation needs, which can include one or more of the above generation methods, and the present application does not make special limitations thereto.

[0061] In an embodiment of the present application, before clustering and identifying each coordinate corresponding to the feature value in the feature map to determine the position information of at least one target region of the target object in the feature map, the method further comprises:

[0062] The feature values in the feature map are convolved by using a convolution kernel, an activation function and a corresponding activation threshold, so as to remove isolated points from the feature map.

[0063] In an embodiment, the convolution kernel can be a matrix with all elements being 1, and the convolution kernel can be adjusted and set according to the application scenario in actual application.

[0064] In an embodiment, specifically, the feature values in the feature map can be convolved by using the following activation function:

[0065]

[0066] wherein, η is a threshold value (i.e., an activation threshold) determined according to the event camera performance parameter and the environment setting and the target detection object, and is generally small, and can be obtained by experience value through several experiments. The convolution kernel is a matrix with all elements being 1, and the matrix size is 2*η+1. For example, when η=1, the size of the convolution kernel is 3*3, and the corresponding convolution kernel is:

[0067] In other embodiments, other activation functions can also be used by those skilled in the art, and the present application does not make special limitations on this.

[0068] In an embodiment of the present application, after the corresponding feature map is generated, the gray value corresponding to each feature value in the feature map can be obtained to visualize the feature map. In an example, the normalized feature value can be multiplied by 255 to determine the corresponding gray value, so as to visualize the feature map.

[0069] In another example, the gray value corresponding to each feature value can be obtained in combination with the chronological order, for example, the later the event occurs, the larger or smaller the corresponding gray value is, and the two are in a monotonically increasing or monotonically decreasing relationship. Thus, according to the visualized feature map, the chronological order of the event occurrence can be determined according to the light and dark relationship of the pixel points.

[0070] Please continue to refer to Figure 1 In step S130, clustering recognition is performed according to the feature value corresponding to each coordinate in the feature map to determine the position information of at least one target region of the target object in the feature map.

[0071] In this embodiment, the terminal can use a clustering algorithm to cluster the feature value corresponding to each coordinate in the feature map to identify the position information of the target region of the target object (i.e., the falling object) from the feature map. In an example, a box can be used to mark the position of the target object in the feature map, and the corner point information of the box can be determined as the position information.

[0072] In an embodiment of the present application, clustering recognition is performed according to the feature value corresponding to each coordinate in the feature map to determine the position information of at least one target region of the target object in the feature map, comprising:

[0073] According to the pre-set neighborhood radius and the minimum number of sample points in the neighborhood radius, the DBSACN density clustering algorithm is used to cluster the feature value corresponding to each coordinate in the feature map, so that the DBSACN density clustering algorithm feeds back the position information of at least one target region of the target object.

[0074] In this embodiment, the DBSCAN density clustering algorithm parameters eps and min_samples can be preset, where eps is the neighborhood radius of the algorithm scanning, and min_samples is the minimum number of sample points in the neighborhood radius. Specifically, starting from an unvisited point, all nearby points within eps (including eps) can be found.

[0075] If the number of nearby points is ≥ min_samples, the current point and its nearby points form a cluster, and the starting point is marked as visited. Then recursively process all unmarked points in the cluster in the same way to expand the cluster. If the number of nearby points is less than min_samples, the point is temporarily marked as a noise point. If the cluster is sufficiently expanded, i.e., all points in the cluster are marked as visited, other unvisited points are processed in the same way, and finally the clustering of points is achieved.

[0076] According to the clustering result, the points with different labels fed back by the algorithm can be framed, and the position information is as follows:

[0077]

[0078] Where pt1 and pt2 are a pair of opposite corners of the annotated rectangular frame. After determining the position of the rectangular frame, the rectangular frame can be visualized in the feature map.

[0079] It should be understood that other clustering methods can also be selected by those skilled in the art according to actual implementation needs, which are not specially limited in the present application.

[0080] Please continue to refer to Figure 1 In step S140, the motion trajectory of the target object is determined based on the position information of each target region in the feature map corresponding to the continuous at least one event stream.

[0081] In this embodiment, it should be understood that the process of object falling needs to last for a period of time. If the time span of the event stream is too small, the position information of each target region in the feature map corresponding to the continuous at least one event stream can be combined to determine the motion trajectory of the target object, for example, the position information of each target region in the feature map corresponding to three continuous event streams can be combined, and so on. The terminal can map the position information of each target region in the feature map corresponding to the at least one event stream to the same graph, and determine the motion trajectory of the target object by curve fitting according to the position information and display it.

[0082] In an embodiment of the present application, the motion trajectory of the target object is determined based on the position information of each target region in the feature map corresponding to the continuous at least one event stream, and the determination includes:

[0083] The center position of each target region is determined according to the position information of each target region in the feature map corresponding to the continuous at least one event stream.

[0084] The motion trajectory of the target object is generated by interpolating the center positions of the target regions and fitting a curve.

[0085] In this embodiment, the terminal can determine the center position of each target region according to the position information of each target region in the feature map corresponding to the at least one event stream, and generate the motion trajectory of the target object by interpolating the center positions of the target regions and fitting a curve. Thus, the linearity of the generated motion trajectory can be ensured by interpolating the center positions and generating the motion trajectory, thereby improving the accuracy of the motion trajectory.

[0086] Thus, the object falling detection method based on the event camera provided in the embodiments of the present application is mainly based on the event stream, can fully utilize the time information of the events, has a smaller required calculation amount, has a higher calculation efficiency, can be used for real-time processing, and improves the detection efficiency. Moreover, the analysis based on the event stream enables the small-volume moving target to be accurately detected, and the event camera detects the target region and only captures the motion information, thereby improving the privacy of the environment.

[0087] The device embodiments of the present application are described below, which can be used to execute the object falling detection method based on the event camera in the above-mentioned embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the above-mentioned embodiments of the object falling detection method based on the event camera.

[0088] Figure 2 A block diagram of an object falling detection device based on an event camera according to an embodiment of the present application is shown.

[0089] Referring to Figure 2 The object falling detection device based on an event camera according to an embodiment of the present application includes:

[0090] The acquisition module 210 is configured to acquire an event stream output by an event camera, wherein the event stream contains event information of a plurality of continuous events, and the event information includes the occurrence coordinates and occurrence time of the events.

[0091] The feature map generation module 220 is configured to generate a feature map corresponding to the event stream according to the occurrence coordinates and occurrence time of each event, wherein the feature map describes the time characteristics and / or position characteristics of the events contained in the event stream.

[0092] The motion target recognition module 230 is configured to perform clustering recognition on the feature values corresponding to each coordinate in the feature map, and determine position information of at least one target region in which a target object exists in the feature map.

[0093] The trajectory processing module 240 is configured to determine a motion trajectory of the target object based on the position information of each target region in the feature map corresponding to the continuous at least one event stream.

[0094] In an embodiment of the present application, the feature map generation module 220 is configured to: determine the number of events occurring at the same coordinate and time information of the events based on the occurrence coordinates and occurrence times of the events; generate a feature map of average time accumulation corresponding to the event stream based on the number of events occurring at the same coordinate and the sum of the time information of the events; and / or generate a feature map of latest time update corresponding to the event stream based on the latest time of the events occurring at the same coordinate; and / or generate a feature map of event number accumulation corresponding to the event stream based on the number of events occurring at the same coordinate; and / or generate a 0 / 1 feature map corresponding to the event stream based on the coordinates of the events, wherein the coordinates corresponding to the events are set to 1, and otherwise 0.

[0095] In an embodiment of the present application, the feature map generation module 220 is further configured to remove background and noise from the feature map based on the feature values corresponding to each coordinate in the feature map, the average of the time stamps of the events, and the time span of the event stream.

[0096] In an embodiment of the present application, the feature map generation module 220 is further configured to perform convolution processing on the feature values in the feature map by using a convolution kernel, an activation function, and a corresponding activation threshold, so as to remove isolated points from the feature map, wherein the convolution kernel is a matrix with all elements being 1.

[0097] In an embodiment of the present application, the motion target recognition module 230 is configured to perform clustering recognition on the feature values corresponding to each coordinate in the feature map by using a DBSACN density clustering algorithm based on a preset neighborhood radius and a minimum number of sample points in the neighborhood radius, so that the DBSACN density clustering algorithm feeds back the position information of at least one target region in which a target object exists.

[0098] In an embodiment of the present application, the trajectory processing module 240 is configured to determine the center positions of each target region based on the position information of each target region in the feature map corresponding to the continuous at least one event stream; and perform interpolation based on the center positions of each target region, fit a curve, and generate a motion trajectory of the target object.

[0099] In an embodiment of the present application, the feature map generation module 220 is further configured to: acquire a gray value corresponding to each feature value in the feature map according to the feature value corresponding to each coordinate in the feature map, so as to perform visual processing on the feature map.

[0100] Figure 3 A structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application is shown.

[0101] It should be noted that, Figure 3 The computer system of the electronic device shown is only an example, and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0102] As Figure 3 shown, the computer system includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 302 or loaded from a storage portion 308 into a random access memory (RAM) 303, such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in the RAM 303. The CPU 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0103] The following components are connected to the I / O interface 305: an input portion 306 including a keyboard, a mouse, and the like; an output portion 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage portion 308 including a hard disk, and the like; and a communication portion 309 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication portion 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as necessary. A removable recording medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is attached to the drive 310 as necessary, so that a computer program read therefrom is installed in the storage portion 308 as necessary.

[0104] In particular, the processes described above with reference to the flow charts can be implemented as a computer software program in accordance with embodiments of the present application. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising computer instructions for performing the methods illustrated by the flow charts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 309 and / or installed from the removable media 311. When the computer program is executed by the central processing unit (CPU) 301, various functions defined in the system of the present application are performed.

[0105] It should be noted that the computer readable medium shown in the embodiments of the present application can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium may, for example, be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, the computer readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In this application, the computer readable signal medium can include a data signal carried in a baseband or as part of a carrier wave, in which the computer readable program is carried. Such a propagated data signal can take any of a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The computer program contained in the computer readable medium can be transmitted using any suitable medium, including, but not limited to, wireless, wired, optical, electromagnetic, infrared, or any suitable combination thereof.

[0106] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or combinations of hardware and software.

[0107] The units described in the embodiments of the present application can be implemented by software, or by hardware, or by a combination of software and hardware. The units described may

[0108] As another aspect, the present application also provides a computer readable medium, which can be included in the electronic device described in the above embodiments, or can exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the method described in the above embodiments.

[0109] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, the division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into a plurality of modules or units.

[0110] From the above description of the embodiments, those skilled in the art will readily appreciate that the example embodiments described herein can be implemented by software and / or by hardware coupled with software. Accordingly, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, or the like) or on a network, and includes a number of instructions for causing a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to perform the methods according to the embodiments of the present application.

[0111] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. It is intended that the application be limited only by the scope of the claims, which will be construed in accordance with the principles of patent law including 35 U.S.C. § 112(f). All references cited herein are incorporated by reference in their entirety and for all purposes.

[0112] It should be understood that the application is not limited to the precise construction and compositions described above and shown in the attached drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application is limited only by the claims that follow, which are to be construed in the broadest sense permissible.

Claims

1. An event camera based object fall detection method, characterized in that, The method comprises: obtaining an event stream of an event camera output, the event stream containing a plurality of continuous events, and the event information including the occurrence coordinates and the occurrence time of the events; generating a feature map corresponding to the event stream according to the occurrence coordinates and the occurrence time of each event, the feature map describing the time characteristics and / or position characteristics of the events contained in the event stream; generating a feature map corresponding to the event stream according to the occurrence coordinates and the occurrence time of each event, comprising: determining the number of events occurring at the same coordinates and the time information of the events occurring at the same coordinates according to the occurrence coordinates and the occurrence time of each event; generating a feature map of average time accumulation corresponding to the event stream according to the number of events occurring at the same coordinates and the sum of the time information of the events occurring at the same coordinates; and / or generating a feature map of the latest time update corresponding to the event stream according to the latest time of the events occurring at the same coordinates; and / or generating a feature map of event number accumulation corresponding to the event stream according to the number of events occurring at the same coordinates; and / or generating a 0 / 1 feature map corresponding to the event stream according to the coordinates of the events, wherein the coordinates of the events correspond to a position of 1, otherwise 0; performing clustering recognition on the feature values corresponding to each coordinate in the feature map to determine the position information of at least one target region of the target object existing in the feature map, comprising: performing clustering recognition on the feature values corresponding to each coordinate in the feature map using a DBSACN density clustering algorithm according to the pre-set neighborhood radius, the minimum number of sample points in the neighborhood radius, so that the DBSACN density clustering algorithm feeds back the position information of at least one target region of the target object existing; determining the motion trajectory of the target object based on the position information of each target region in the feature maps corresponding to the at least one continuous event stream, comprising: determining the center position of each target region according to the position information of each target region in the feature maps corresponding to the at least one continuous event stream; interpolating according to the center position of each target region to fit a curve to generate the motion trajectory of the target object.

2. The method of claim 1, wherein, After generating the feature map of average time accumulation and / or the feature map of the latest time accumulation corresponding to the event stream, the method further comprises: removing background and noise from the feature map according to the feature values corresponding to each coordinate in the feature map, the average value of the time stamp of the events, and the time span of the event stream.

3. The method of claim 1, wherein, Before performing clustering recognition on the feature values corresponding to each coordinate in the feature map to determine the position information of at least one target region of the target object existing in the feature map, the method further comprises: performing convolution processing on the feature values in the feature map using a convolution kernel, an activation function and a corresponding activation threshold to remove isolated points from the feature map.

4. The method according to any one of claims 1-3, characterized in that, After generating a feature map corresponding to the event stream according to the occurrence coordinates and the occurrence time of each event, the method further comprises: obtaining the gray value corresponding to each feature value according to the feature value corresponding to each coordinate in the feature map to perform visual processing on the feature map.

5. An event camera based object fall detection apparatus, characterized in that, The method comprises: an acquisition module configured to acquire an event stream output by an event camera, the event stream comprising event information of a plurality of continuous events, the event information comprising occurrence coordinates and occurrence time of the events; a feature map generation module configured to generate a feature map corresponding to the event stream according to the occurrence coordinates and the occurrence time of each event, the feature map describing time characteristics and / or position characteristics of the events included in the event stream; generating a feature map corresponding to the event stream according to the occurrence coordinates and the occurrence time of each event comprises: determining the number of events occurring at the same coordinates and time information of the events according to the occurrence coordinates and the occurrence time of each event; generating an average time accumulation feature map corresponding to the event stream according to the number of events occurring at the same coordinates and the sum of the time information of the events; and / or generating a latest time update feature map corresponding to the event stream according to the latest time of the events occurring at the same coordinates; and / or generating an event number accumulation feature map corresponding to the event stream according to the number of events occurring at the same coordinates; and / or generating a 0 / 1 feature map corresponding to the event stream according to the coordinates of the events, wherein the coordinates of the events correspond to a position of 1, otherwise 0; a moving target recognition module configured to perform clustering recognition on the feature values corresponding to each coordinate in the feature map to determine position information of at least one target region in which a target object exists in the feature map, comprising: performing clustering recognition on the feature values corresponding to each coordinate in the feature map using a DBSACN density clustering algorithm according to a pre-set neighborhood radius and a minimum number of sample points in the neighborhood radius, so that the DBSACN density clustering algorithm feeds back the position information of at least one target region in which a target object exists; a trajectory processing module configured to determine a motion trajectory of the target object based on the position information of each target region in the feature maps corresponding to the continuous at least one event stream, comprising: determining the center positions of each target region according to the position information of each target region in the feature maps corresponding to the continuous at least one event stream; performing interpolation according to the center positions of each target region to fit a curve to generate the motion trajectory of the target object.

6. A computer readable medium having stored thereon a computer program, characterized in that, The computer program is executed by a processor to implement the event camera-based object falling detection method of any one of claims 1 to 3.

7. An electronic device, comprising: The method comprises: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the event camera-based object falling detection method of any one of claims 1 to 3.

Images