Lightning monitoring device and method
By setting a counter for each event pixel in the event camera and using depth-first search, the problems of frame rate limiting and high power consumption of traditional cameras are solved, and low-power, fast-responsive lightning detection is achieved to accurately extract the flash path.
Patent Information
- Application Number
- CN202411188288.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-11-15
AI Technical Summary
In lightning monitoring, traditional CMOS cameras have difficulty capturing lightning images in time and in full. In complex environments, event cameras have caused a large number of pulse events due to pixel circuit design, making it difficult to judge the flash path.
By setting a counter for each event pixel in the event camera, the pixels are effectively activated based on the change in the count value of the ON and OFF events and the time difference, and the flash path is extracted through depth-first search.
It realizes low-power and fast-responsive lightning detection, which can effectively capture lightning details and extract flash paths, improving the accuracy of detection.
Smart Images

Figure CN118962262A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of lightning monitoring, and in particular to a lightning monitoring device and method implemented by an event camera. Background Art
[0002] Lightning monitoring and protection is an important branch of meteorology and electrical engineering, which involves the detection and analysis of lightning activities and the design of protective measures.
[0003] There are many types of lightning, mainly (1) Intra-Cloud (IC): Lightning that occurs inside a cloud, which is the most common type. (2) Cloud-to-Ground (CG): Lightning that travels from a cloud to the ground, posing a direct threat to humans and facilities. (3) Inter-Cloud (IC): Lightning that occurs between different clouds.
[0004] To detect lightning, the main detection technologies are divided into: optical monitoring, using cameras or light sensors to capture the light signals generated by lightning. Acoustic monitoring, using the thunder caused by lightning to locate the lightning position. Electromagnetic field monitoring, locating and analyzing lightning activity by detecting the electromagnetic radiation generated by lightning. Ground network, establishing a ground sensor network, such as the Lightning Location System (LLS), to accurately measure the location, intensity and frequency of lightning. Satellite monitoring, using sensors on satellites to monitor lightning activity worldwide from space.
[0005] In the direction of optical detection, traditional complementary metal oxide semiconductor (CMOS) cameras are limited by frame rate and the ultra-high generation speed of lightning, so they can capture lightning images in a timely and complete manner and truly restore the lightning scene.
[0006] In addition, due to the high power consumption and cost of CMOS cameras, it is difficult to widely deploy them in different scenarios, and there is a lack of conditions for large-scale deployment, which is urgently needed for lightning detection applications.
[0007] Currently, there is a new optical sensor, namely the event camera or dynamic vision sensor. If the event camera is used directly for lightning monitoring, due to various real-world environments and the pixel circuit design of the event camera circuit, such as the reflection or illumination of buildings, clouds or air, or high-contrast changes in ambient light, which may cause leakage of surrounding pixels, not only will the lightning path activate the event pixels, but the pixels around these activated event pixels will also generate a large number of pulse events (for example, 120,000 to 300,000 events). Based on the large number of events directly output by the event camera, it is difficult to directly determine whether lightning has occurred, and the lightning path cannot be effectively extracted. Summary of the invention
[0008] In order to alleviate or partially alleviate the above technical problems, the solution of the present invention is as follows:
[0009] On the one hand, the present invention discloses a lightning monitoring device, including an event camera, wherein the event camera includes event pixels, and the event pixels are configured to generate ON events or OFF events; and when the event pixel generates an ON event, the count value of the corresponding counter is incremented by one; when the event pixel generates an OFF event, the count value of the corresponding counter is decremented by one; when an event pixel generates an ON event, it is determined whether the count value of the counter corresponding to the event pixel is lower than a second threshold within a preset time period, and if so, the event pixel is indicated as a valid activation event pixel, otherwise the event pixel is not indicated as a valid activation event pixel.
[0010] Furthermore, each event pixel is provided with a corresponding counter.
[0011] Further, when the count value of the counter corresponding to the event pixel reaches the first threshold, it is determined whether the count value of the counter corresponding to the event pixel is lower than the second threshold within a preset time period.
[0012] Further, when the count value of the counter corresponding to the event pixel reaches the first threshold value within a preset period starting from 0, it is determined whether the count value of the counter corresponding to the event pixel is lower than the second threshold value within a preset time period.
[0013] Further, when the count value of the counter corresponding to the event pixel decreases for the first time, it is determined whether the count value of the counter corresponding to the event pixel is lower than a second threshold within a preset time period.
[0014] Further, after the event pixel is indicated as a valid activation event pixel, if the recovery period is reached, the count value of the counter corresponding to the event pixel is reset to zero.
[0015] Further, when the count value of the counter corresponding to the event pixel reaches the first threshold, if the waiting period is reached, the count value of the counter corresponding to the event pixel is reset to zero.
[0016] Further, when the event pixel generates an ON event, and the time difference between the ON event and the last ON event generated by the event pixel is within a set time range, the count value of the counter corresponding to the event pixel is incremented by one.
[0017] Furthermore, when the event pixel generates an OFF event, and the time difference between the OFF event and the last OFF event generated by the event pixel is within a set time range, the count value of the counter corresponding to the event pixel is reduced by one.
[0018] Furthermore, according to all pixels in the event camera that are indicated as valid activation events, event pixels activated by the lightning path are extracted through a depth-first search method.
[0019] On the other hand, the present invention discloses a lightning monitoring method, characterized in that it includes the following steps: detecting lightning through an event camera, wherein the event camera includes event pixels, and the event pixels are configured to generate ON events or OFF events; when the event pixel generates an ON event, the count value of the counter corresponding to the event pixel is incremented by one; when the event pixel generates an OFF event, the count value of the counter corresponding to the event pixel is decremented by one; when an event pixel generates an ON event, determining whether the count value of the counter corresponding to the event pixel is lower than a second threshold within a preset time period, and if so, the event pixel is indicated as a valid activation event pixel, otherwise the event pixel is not indicated as a valid activation event pixel.
[0020] Furthermore, each event pixel is provided with a corresponding counter.
[0021] Further, when the count value of the counter corresponding to the event pixel reaches the first threshold, it is determined whether the count value of the counter corresponding to the event pixel is lower than the second threshold within a preset time period.
[0022] Further, when the count value of the counter corresponding to the event pixel reaches the first threshold value within a preset period starting from 0, it is determined whether the count value of the counter corresponding to the event pixel is lower than the second threshold value within a preset time period.
[0023] Further, when the count value of the counter corresponding to the event pixel decreases for the first time, it is determined whether the count value of the counter corresponding to the event pixel is lower than a second threshold within a preset time period.
[0024] Further, after the event pixel is indicated as a valid activation event pixel, if the recovery period is reached, the count value of the counter corresponding to the event pixel is reset to zero.
[0025] Further, when the count value of the counter corresponding to the event pixel reaches the first threshold, if the waiting period is reached, the count value of the counter corresponding to the event pixel is reset to zero.
[0026] Further, when the event pixel generates an ON event, and the time difference between the ON event and the last ON event generated by the event pixel is within a set time range, the count value of the counter corresponding to the event pixel is incremented by one.
[0027] Furthermore, when the event pixel generates an OFF event, and the time difference between the OFF event and the last OFF event generated by the event pixel is within a set time range, the count value of the counter corresponding to the event pixel is reduced by one.
[0028] Furthermore, according to all pixels in the event camera that are indicated as valid activation events, event pixels activated by the lightning path are extracted through a depth-first search method.
[0029] The technical solution of the present invention has one or more of the following beneficial technical effects:
[0030] (1) A new lightning detection system and method are proposed to solve the lightning monitoring problem effectively and with low power consumption.
[0031] (2) Low power consumption, fast response speed, and the ability to capture lightning details and extract lightning paths.
[0032] In addition, other beneficial effects of the present invention will be mentioned in the specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 It is a schematic diagram of the event generated by the event pixel in the pixel array of the actual measurement event camera;
[0034] Figure 2 It is a schematic diagram of the pixels of the actual activation event of the lightning path;
[0035] Figure 3 is a schematic diagram of the change trend of the count value of the counter corresponding to the event pixel;
[0036] Figure 4 This is an ideal pixel diagram of the activation event along the lightning path;
[0037] Figure 5 It is a schematic diagram of event pixels activated by a lightning activity;
[0038] Figure 6This is a processed schematic diagram of the lightning path activation event pixels. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0040] In order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order.
[0041] An event camera, like a normal camera, usually comprises a two-dimensional pixel array. Without loss of generality, the event camera pixel array comprises event pixels. An event pixel can be any pixel of the event camera.
[0042] Figure 1 It is a schematic diagram of the event generated by the event pixel in the pixel array of the event camera after actual measurement. The present invention adopts a brand-new optical sensor, namely the event camera, to capture the lightning path. After actual testing, the event pixel generates an event characterized by first generating a number of ON events and then generating a number of OFF events. Among them, the ON event is an event with a positive polarity (for example, p=1), and the OFF event is an event with a negative polarity (for example, p=-1).
[0043] Generally, different lightning has different parameters. 95% of the front phase (front duration, 2kA to peak) lasts about 1.8 microseconds (the first stroke), and 95% of the half-peak to 2kA duration is usually 30 microseconds, which means that lightning activity is an extremely high-speed behavior, which is extremely difficult to capture for ordinary cameras.
[0044] The present invention comprises a method for detecting the presence of lightning and further preferably extracting the path of the lightning.
[0045] Figure 2 It is a schematic diagram of the pixels of the actual activated event of the lightning path, and exemplarily gives the information (x, y, p, t) contained in the event (also called pulse event) generated by each pixel, where (x, y) is the coordinate of the pixel, p represents the polarity, which can be used to distinguish between ON events and OFF events, and t represents the timestamp when the event occurs.
[0046] During the measurement process, it was found that the lightning path would activate the event pixel, and the event pixel would output an event or a pulse event. The event pixel of the event camera would generate a first number of ON events and then a second number of OFF events, wherein the first number is often less than the second number.
[0047] To this end, the present invention sets a first counter for the event pixel, and the first counter records the count value of the counter corresponding to the event pixel, that is, the count value of the event pixel. When the event pixel generates an ON event, the count value of the counter corresponding to the event pixel is incremented by 1, and if the event pixel generates an OFF event, the count value of the counter corresponding to the event pixel is decremented by 1.
[0048] Furthermore, only if the time difference between two OFF events or / and ON events of the same polarity generated by the event pixel is within a set time range, the count value of the counter corresponding to the event pixel can be incremented or decremented by 1.
[0049] However, not all pixels that generate events are generated by the lightning path. Pixels that generate events can be divided into the following two types: event pixels activated by factors outside the lightning path and event pixels activated by the lightning path.
[0050] Figure 3 It is a schematic diagram of the counting value change trend of the counter corresponding to the event pixel.
[0051] For example, when the count value of the counter corresponding to the event pixel reaches (in this embodiment, is greater than) the first threshold, it is determined that within the preset time period T d The count value of the counter corresponding to the event pixel is lower than the second threshold. If it is lower than the second threshold, the event pixel is a valid activation event pixel, otherwise the event pixel is an ineffective activation event pixel. The event pixel is a valid activation event pixel, which means that the lightning path or factors outside the lightning path are effectively imaged at the event pixel position.
[0052] Alternatively, when the count value of the counter corresponding to the event pixel reaches the first threshold value within the preset period T1 from 0, it is determined that the event pixel is within the preset time period T d Whether the count value of the counter corresponding to the event pixel is lower than the second threshold.
[0053] Alternatively, when the count value of the counter corresponding to the event pixel decreases for the first time, it is determined that the event pixel has been detected within the preset time period T. d In this embodiment, the rising condition of the ON event is no longer concerned, but only the falling interval of the OFF event is concerned. Therefore, the preset time period T d It can be compared with the preset time period T in the previous embodiment. d Small.
[0054] Preferably, after an event pixel is indicated as a valid activation event pixel, if the recovery period T r , the count value of the counter corresponding to the event pixel can be automatically reset to zero.
[0055] Preferably, when the count value of the counter corresponding to the event pixel reaches the first threshold, if the waiting period T is reached w , the count value of the counter corresponding to the event pixel can be automatically reset to zero.
[0056] For example, the waiting period T w and recovery period T r They can be equal or unequal.
[0057] In one example, the preset time period T d Equal to 25 milliseconds.
[0058] For example, the preset period T1 is equal to 10 milliseconds. Although the present invention gives some parameter examples by way of example, the present invention is not limited thereto and is not limited thereto. In addition, the logic judgments about size listed in the present invention can be implemented based on opposite equivalent means, which is a transformation that is essentially the same as the present invention.
[0059] For example, the first threshold is 2, and the second threshold is -3. In one embodiment, if the count value of the counter corresponding to the event pixel reaches 2, it is determined whether the count value of the event pixel in the preset time period is lower than -3. If the count value of the event pixel in the preset time period is lower than -3, it is considered that the event pixel is activated by the lightning path, otherwise, the possibility that the event pixel generates an event due to lightning is excluded.
[0060] Through the above scheme, only by setting a counter for each pixel, the effective activation event pixels can be effectively and efficiently identified. The effective activation event pixels can be used as a basis for whether lightning occurs. For example, if the number of effective activation event pixels reaches a preset number within a certain time period, it can be identified as the detection of lightning activity.
[0061] Through the above scheme, the present invention can identify event pixels activated by the lightning path and exclude event pixels activated by factors outside the lightning path to the greatest extent possible.
[0062] Preferably, after obtaining effective activation event pixels or determining that lightning activity is detected, the lightning path can be extracted by a depth-first search scheme. The depth-first search method itself is common technical knowledge in the art, and the present invention will not elaborate on it.
[0063] The multiple pulse events generated by the pixel array constitute a pulse sequence and output it. How to efficiently process the pulse sequence and extract the lightning path is a difficult problem. From the above description, it is not difficult to know that the present invention proposes a solution for efficiently marking and extracting the lightning path.
[0064] Figure 4 This is a schematic diagram of an ideal activation event pixel of a lightning path. This result shows the ideal effective activation event pixel or lightning path extraction result.
[0065] However, in actual environments, among the identified valid activation event pixels, there may still be event pixels activated by factors outside the lightning path.
[0066] Optionally, in extracting the lightning path, pixels (valid activation event pixels or activation event pixels) constituting the slice or block can be directly eliminated, and do not constitute valid activation event pixels of the final lightning path. For example, valid activation event pixels within the block can be set as deactivation event pixels.
[0067] Figure 5 This is a schematic diagram of event pixels activated by a lightning activity. In practical applications, lightning will hit objects such as walls, buildings, and trees, and sometimes the same event pulse will be generated. In this example, lightning activates a large area, but this large area contains a large number of event pixels that are activated due to factors outside the lightning path. If the causes of these pixels are not distinguished, the lightning path will be misjudged.
[0068] Figure 6 is right Figure 5 Schematic diagram of lightning path after processing. In this example, most of the event pixels in the aforementioned large area are determined as event pixels not activated by the lightning path through some processing methods, and the remaining event pixels are determined as event pixels activated by the lightning path, thereby obtaining Figure 6 As mentioned above, this is more conducive to the accuracy of detecting lightning paths in some practical application scenarios.
[0069] In order to better illustrate the present invention, numerous specific details are provided in the above specific embodiments. It should be understood by those skilled in the art that the present invention can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present invention.
[0070] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A lightning monitoring device, comprising an event camera, characterized in that: The event camera includes event pixels configured to generate an ON event or an OFF event; When an event pixel generates an ON event, the count value of the corresponding counter increases by one; When an event pixel generates an OFF event, the count value of the corresponding counter is reduced by one; When an event pixel generates an ON event, it is determined whether the count value of the counter corresponding to the event pixel is lower than a second threshold within a preset time period. If so, the event pixel is indicated as a valid activation event pixel, otherwise the event pixel is not indicated as a valid activation event pixel.
2. The lightning monitoring device according to claim 1, characterized in that: When the count value of the counter corresponding to the event pixel reaches the first threshold, it is determined whether the count value of the counter corresponding to the event pixel is lower than the second threshold within a preset time period.
3. The lightning monitoring device according to claim 1, characterized in that: When the count value of the counter corresponding to the event pixel reaches the first threshold value within a preset period starting from 0, it is determined whether the count value of the counter corresponding to the event pixel is lower than the second threshold value within a preset time period.
4. The lightning monitoring device according to claim 1, characterized in that: When the count value of the counter corresponding to the event pixel decreases for the first time, it is determined whether the count value of the counter corresponding to the event pixel is lower than a second threshold within a preset time period.
5. The lightning monitoring device according to any one of claims 1 to 4, characterized in that: After the event pixel is indicated as a valid activation event pixel, if the recovery period is reached, the count value of the counter corresponding to the event pixel is set to zero.
6. The lightning monitoring device according to claim 2 or 3, characterized in that: When the count value of the counter corresponding to the event pixel reaches the first threshold, if the waiting period is reached, the count value of the counter corresponding to the event pixel is reset to zero.
7. The lightning monitoring device according to claim 5, characterized in that: When the event pixel generates an ON event, and the time difference between the ON event and the last ON event generated by the event pixel is within the set time range, the count value of the counter corresponding to the event pixel is incremented by one.
8. The lightning monitoring device according to claim 5, characterized in that: When the event pixel generates an OFF event, and the time difference between the OFF event and the last OFF event generated by the event pixel is within the set time range, the count value of the counter corresponding to the event pixel is reduced by one.
9. The lightning monitoring device according to claim 8, characterized in that: According to all the event pixels indicated as valid activation events in the event camera, event pixels activated by the lightning path are extracted through a depth-first search method.
10. A lightning monitoring method, characterized in that: The steps include: detecting lightning by an event camera, wherein the event camera comprises event pixels, the event pixels being configured to generate an ON event or an OFF event; When an event pixel generates an ON event, the count value of the counter corresponding to the event pixel increases by one; When an event pixel generates an OFF event, the count value of the counter corresponding to the event pixel is reduced by one; When an event pixel generates an ON event, it is determined whether the count value of the counter corresponding to the event pixel is lower than a second threshold within a preset time period. If so, the event pixel is indicated as a valid activation event pixel, otherwise the event pixel is not indicated as a valid activation event pixel.
