Lightning Monitoring Device, System, Method, and Storage Medium

Abstract

This application is applicable to the technical field of power systems, and provides a lightning monitoring device, system and method, as well as a storage medium. The device includes: a dynamic vision sensor for collecting event data; a calculation module for obtaining lightning information according to the event data, where the lightning information includes whether lightning occurs; and a communication module for sending the lightning information to a central station.

Description

Technical Field

[0001] This application belongs to the technical field of power systems, and particularly relates to a lightning monitoring device, system, method, and computer-readable storage medium. Background Art

[0002] Lightning poses a great threat to the safe operation of transmission lines and often causes accidents. Especially in mountainous areas and other places with inconvenient transportation, it adds a lot of difficulties to patrol and find faults. Accurately locating the position where the transmission line is struck by lightning and then performing fast and accurate maintenance is of great significance to the safe operation of the power system.

[0003] The lightning location system currently used in the power grid is a complete set of fully automatic, large-area, high-precision, and real-time lightning monitoring systems, which mainly consists of three parts: detection stations, data processing and system control centers (central stations), and lightning information systems. The detection stations detect very low frequency / low frequency (VLF / LF) electromagnetic waves, identify cloud-to-ground lightning strokes through certain waveform discrimination conditions, analyze and extract information such as the arrival time, azimuth angle, and signal intensity of the return stroke electromagnetic waves, and send them to the central station for positioning calculation. The central station comprehensively calculates the occurrence time and position of the cloud-to-ground lightning stroke based on the direction-finding method and the time-of-arrival method, and estimates the return stroke current intensity through the current inversion model. After completing the above calculation and analysis work, the positioning information is transmitted to the lightning information system for subsequent processing.

[0004] The positioning accuracy and detection efficiency of the above lightning location system are not high enough. The positioning error range is more than 500m, and the detection efficiency of lightning return stroke discharges is below 90%. Therefore, it is difficult to accurately locate the lightning strike point with the help of the lightning location system. Summary of the Invention

[0005] Embodiments of this application provide a lightning monitoring device, system, method, and computer-readable storage medium, which can solve the problem that it is difficult to accurately locate the lightning strike point in the related art lightning location system.

[0006] In a first aspect, an embodiment of this application provides a lightning monitoring device, which includes: a dynamic vision sensor for collecting event data; a calculation module for obtaining lightning information according to the event data, where the lightning information includes whether lightning occurs; and a communication module for sending the lightning information to the central station.

[0007] In a second aspect, an embodiment of this application provides a lightning monitoring system, which includes a plurality of the lightning monitoring devices described in the first aspect above and a central station; the central station is used to perform fusion processing on the lightning information reported by the lightning monitoring devices to obtain the lightning strike coordinates.

[0008] In a third aspect, an embodiment of the present application provides a lightning monitoring method, the method comprising: collecting event data through a dynamic visual sensor; obtaining lightning information based on the event data, the lightning information including whether lightning occurs; and sending the lightning information to a central station.

[0009] In a fourth aspect, an embodiment of the present application provides a lightning monitoring method, the method comprising: receiving lightning information from a lightning monitoring device, the lightning information including whether lightning occurs, and at least one of lightning-related event data and a restored image reconstructed based on the event data; when lightning information sent by at least two lightning monitoring devices is associated with the same lightning strike process, determining restored images of multiple perspectives of the lightning strike process based on the associated lightning information; synthesizing the restored images of multiple perspectives using multi-perspective synthesis technology to obtain a synthesized image; and identifying the landing point of the synthesized image to obtain the coordinates of the lightning strike.

[0010] In a fifth aspect, an embodiment of the present application provides a central station, including a memory, a processor, and a computer program stored in the memory and executable by the processor, and when the processor executes the computer program, the lightning monitoring method described in the third aspect above is implemented.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a lightning monitoring device, the lightning monitoring device executes the lightning monitoring method described in the third aspect above.

[0012] In a seventh aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a central station, the central station executes the lightning monitoring method described in the fourth aspect above.

[0013] Compared with the prior art, the embodiments of the present application have the following beneficial effects: each pixel of the dynamic visual sensor can independently detect changes in light intensity. When a pixel detects that the change in light intensity exceeds a set threshold, an event will be output, which includes a timestamp, the coordinates of the pixel, and the polarity of the event. The dynamic visual sensor simulates the visual perception system of organisms, and has the characteristics of high temporal resolution, less data redundancy, low power consumption, and high dynamic range. It can meet the detection needs of lightning with short duration and drastic changes in light intensity. At the same time, the lightning monitoring device has strong adaptability to the environment and can be arranged in remote locations, on the top of buildings, etc. Together with the central station, it forms a lightning monitoring system, which can effectively monitor the power grid in real time. Lightning information can be identified through event data collected by the dynamic visual sensor, and a restored image can be reconstructed based on the event data to identify more accurate lightning coordinates, thereby improving the accuracy of lightning monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for description in the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0015] Figure 1 is a schematic structural diagram of a lightning monitoring system provided by an embodiment of the present application;

[0016] Figure 2 is a schematic structural diagram of a lightning monitoring device provided by an embodiment of the present application;

[0017] Figure 3 is a schematic diagram of the change curve of current with time during a lightning strike in a specific example of the present application;

[0018] Figure 4 is a schematic diagram of the change curve of the number of events with time during a lightning strike in a specific example of the present application;

[0019] Figure 5 is a restored image of lightning in a specific example of the present application;

[0020] Figure 6 is a schematic flowchart of a lightning monitoring method provided by an embodiment of the present application;

[0021] Figure 7 is a schematic flowchart of a lightning monitoring method provided by another embodiment of the present application. Detailed implementation manners

[0022] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are presented to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0023] It should be understood that when used in the specification and appended claims of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0024] It should also be understood that the term "and / or" used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0025] As used in the specification of this application and the appended claims, the term "if" may be construed as "when", "once", "in response to determining", or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be construed as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" depending on the context.

[0026] In addition, in the description of the specification of this application and the appended claims, the terms "first", "second", "third", etc. are only used for differential description and should not be construed as indicating or implying relative importance.

[0027] Reference to "one embodiment" or "some embodiments" or the like described in the specification of this application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0028] As Figure 1 As shown, the lightning monitoring system provided by the embodiment of this application includes a plurality of lightning monitoring devices 1 and a central station 2. The central station 2 is communicatively connected to each lightning monitoring device 1. The specific structure of the lightning monitoring device 1 refers to the description of subsequent embodiments.

[0029] The lightning monitoring device 1 is used to send lightning information to the central station 2, and the lightning information includes whether lightning occurs. The lightning monitoring device 1 can report lightning information periodically or can be event-triggered to report lightning information, that is, it reports only when lightning is detected.

[0030] The lightning information may further include at least one of relevant event data, a restored image obtained from the event data, and a landing point coordinate calculated by the lightning monitoring device 1 itself for the central station 2 to calculate the lightning strike coordinate. In addition, the lightning information may further include a timestamp, an identifier of the lightning monitoring device 1, positioning, etc.

[0031] The central station 2 is used to perform fusion processing on the lightning information reported by the lightning monitoring device 1 to obtain the lightning strike coordinates. For example, the central station 2 can determine whether multiple lightning monitoring devices 1 monitor the same lightning strike process according to the time stamp. If so, it can determine the restored images of multiple perspectives of this lightning strike process based on the lightning information reported by these lightning monitoring devices 1. Combining the positioning information of these lightning monitoring devices 1, the restored images are synthesized using multi-perspective synthesis technology (for example, when the perspective is 2, binocular stereo vision technology is used) to obtain a synthesized image, and then the lightning strike point is identified from the synthesized image, and the lightning strike coordinates are calculated, thereby improving the calculation accuracy of the lightning strike coordinates.

[0032] If only one lightning monitoring device 1 monitors the lightning strike process and the lightning information reported by this lightning monitoring device 1 includes the strike point coordinates, the central station 2 can directly use the strike point coordinates as the lightning strike coordinates.

[0033] If only one lightning monitoring device 1 monitors the lightning strike process and the lightning information reported by this lightning monitoring device 1 does not include the strike point coordinates, the central station 2 can identify the lightning strike coordinates based on the event data in the lightning information and / or the restored image obtained from the event data. The specific calculation process is similar to the process of the lightning monitoring device 1 calculating the strike point coordinates by itself, and the specific details can refer to the description of the subsequent embodiments.

[0034] Figure 2 The block diagram of a part of the structure of the lightning monitoring device 1 provided by the embodiment of the present application is shown. Refer to Figure 2 , the lightning monitoring device 1 includes a dynamic vision sensor 10, a calculation module 20, and a communication module 30 that are connected in sequence. Those skilled in the art can understand that Figure 2 the structure of the lightning monitoring device 1 shown in

[0035] does not limit the lightning monitoring device 1, and it may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements. Figure 2 The following specifically introduces each component of the lightning monitoring device 1 in combination with

[0036] The dynamic vision sensor 10, which can also be referred to as a neuromorphic vision sensor or an event camera, is a vision sensor that mimics the visual perception system of living organisms. Each pixel of the dynamic vision sensor 10 can independently detect changes in light intensity. When a pixel detects a change in light intensity exceeding a set threshold, it outputs an event, which includes a timestamp, the coordinates of that pixel, and a polarity. Specifically, an event can be represented by a four-dimensional vector e = [t, x, y, p], where t is the timestamp indicating the time when the event is output, x and y represent the coordinates of the pixel, and p represents the polarity, specifically referring to an increase or decrease in light intensity. For example, p = +1 indicates a positive polarity, and this event can also be called an ON event, indicating that the light intensity of this pixel has increased beyond the threshold; p = -1 indicates a negative polarity, and this event can also be called an OFF event, indicating that the light intensity of this pixel has decreased beyond the threshold.

[0037] The pixels of the dynamic vision sensor 10 include a logarithmic photoreceptor, a differential amplifier circuit, and two comparators. The logarithmic photoreceptor uses a logarithmic light intensity perception model to convert the optical signal into a voltage signal, which is then transmitted to the input terminals of the two comparators after passing through the differential amplifier circuit. The other input terminals of the two comparators are the threshold voltages of the upper limit and the lower limit respectively. When the voltage exceeds the threshold, an event with the corresponding polarity is output.

[0038] When a traditional vision sensor is imaging, the aperture is opened, and each pixel of the sensor converts the optical signal into an electrical signal. After the imaging is completed, the aperture is closed, and the electrical signals of each pixel are read one by one to form a frame of image. Each pixel in the image reflects the light intensity during imaging. Traditional vision sensors work synchronously. Since both imaging and reading take time, the frame rate of traditional vision sensors is limited, that is, the temporal resolution is limited. The frame rate of common vision sensors is generally dozens or hundreds of hertz. Special high-speed cameras can achieve higher frame rates, but due to the very limited time for imaging and reading, high-speed cameras require special materials and designs, are bulky and costly, and are generally limited to experimental applications.

[0039] The dynamic vision sensor 10 does not have the process of reading each pixel one by one like traditional vision sensors. The pixels therein only feedback on changes in light intensity and work asynchronously. Therefore, it has a very high temporal resolution and features less data redundancy, low power consumption, and a high dynamic range.

[0040] Due to the fast speed of the lightning strike process and the large change in light intensity, the bright light generated by lightning will illuminate the entire surrounding area. As a result, a large number of events are generated in a short period of time. When the number of events in a short period of time exceeds the transmission bandwidth of the event camera, it may be impossible to read out all the events occurring in all rows / columns within the limited time. Only the "rows / columns that are queued first" can be read out as much as possible. The event data that is not read in time is lost, which is reflected in the restored image as striped imaging. The light intensity of the lightning channel itself changes more dramatically, and the light intensity of the illuminated surrounding area changes relatively slowly. Therefore, in order to reduce the occurrence of the above-mentioned event data loss, the threshold voltage of the dynamic vision sensor 10 can be increased. The specific threshold voltage of the dynamic vision sensor 10 can be set to a first value higher than the default value, thereby reducing the sensitivity of the pixel to the triggering event, reducing the events generated in the illuminated area, and retaining the events generated by the lightning channel. Increasing the threshold voltage of the dynamic vision sensor 10 can also reduce interference from other moving objects in the environment, such as raindrops.

[0041] At the same time, since lightning travels very fast, the event data generated by lightning is a high-frequency signal. In order to further reduce the amount of events and ease the pressure on the transmission bandwidth, the built-in filter of the dynamic vision sensor 10 is set to a high-pass filter to filter out low-frequency signals and retain high-frequency lightning signals.

[0042] The calculation module 20 is used to obtain lightning information according to the event data, and the lightning information includes whether lightning occurs. The calculation module 20 is the control center of the lightning monitoring device 1, and can run the program stored in the memory (not shown in the figure) to perform various functions and process data. The calculation module 20 can be a central processing unit (CPU), and the calculation module 20 can also be other general-purpose processors, digital computer vision devices (Digital Signal Processor, DSP), application-specific integrated circuits (Application Specific Integrated Circuit, ASIC), field-programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In some embodiments, the calculation module 20 may include an AI (Artificial Intelligence, artificial intelligence) processor, which is used to process computing operations related to machine learning.

[0043] The lightning monitoring device 1 may further include a memory for storing an operating system, application programs, a boot loader (BootLoader), data, and other programs, such as program codes of computer programs. The memory may also be used to temporarily store data required for and generated during the execution of programs. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a flash memory, a hard disk, a multimedia card, a card-type memory, etc. The memory may include a storage unit disposed inside the lightning monitoring device 1, such as the hard disk of the lightning monitoring device 1, and / or a removable external storage unit, such as a mobile hard disk, a USB flash drive, a smart media card (SMC), a secure digital (SD) card, etc.

[0044] The communication module 30 is used to connect to and communicate with the central station through wired communication (such as optical fiber, etc.) and / or wireless communication (such as cellular communication, satellite communication, etc.). For example, the communication module 30 may send lightning information to the central station 2. The communication module 30 may also implement the positioning and timing of the lightning monitoring device 1 through a network.

[0045] The lightning monitoring device 1 may further include a power module for supplying power to the lightning monitoring device 1. To adapt to different environments, especially harsh environments, the power module is generally a solar power module capable of independent operation.

[0046] Specifically, the calculation module 20 may calculate the change rate of the number of events based on the number of events; and determine whether lightning occurs based on the change rate.

[0047] During a lightning strike, the light intensity is positively correlated with the current, both showing a sharp increase first and then a relatively slower decrease, presenting the characteristic of a steep rise and gentle fall. The number of events corresponding to the positive polarity increases rapidly within a very short time, and then the number of events with negative polarity increases at a relatively slower speed, and the number of events with negative polarity is generally less than the number of events with positive polarity. The event data itself may not carry light intensity information, but the number of events is positively correlated with the light intensity. For example, the curve of the current changing with time during a lightning strike is as Figure 3 shown, and the curve of the number of events changing with time is as Figure 4 shown. It can be seen that there is an obvious positive correlation between the two.

[0048] Based on this, the calculation module 20 can determine whether lightning occurs. Since the number of events is positively correlated with the size of the lightning current, in order to eliminate the influence of the lightning current size, the rate of change of the number of events is selected instead of the number of events itself for judgment. The judgment can be made only based on the rate of change of the number of events with positive polarity, or it can be judged by combining the rate of change of the number of events with positive polarity and the rate of change of the number of events with negative polarity. Specifically, the rate of change of the number of events can be calculated in very small time units (such as microsecond level or even nanosecond level), and the rate of change of the number of events can be compared with the set threshold value. If it is greater than the threshold value, it is judged that lightning occurs.

[0049] Alternatively, the computing module 20 can restore the event data to obtain a restored image, and use the restored image to determine the lightning information, which also includes the coordinates of the lightning landing point. The event data itself is a vector, which does not match the "one frame image" that we traditionally observe and process. Therefore, the event data can be restored to obtain a restored image, and then the restored image can be processed. In some embodiments, the event data can also be processed directly, for example, by using brain-like computing to process the event data.

[0050] Event data can be accumulated in the time domain to obtain a restored image. The pixel values ​​in the restored image are used to represent the number and polarity of the accumulated events. Compared with traditional images, its pixel values ​​cannot accurately represent the ambient light intensity and may lose some light intensity information. However, for lightning channels with drastic changes in light intensity, it can fully display their morphology.

[0051] For example, the restored image of lightning obtained by reconstruction is as follows Figure 5 As shown, it can be seen that the restored image retains the morphology of the lightning channel well.

[0052] The duration of accumulation may be fixed. For example, a fixed duration may be set according to the duration of lightning, and the event data may be restored at fixed intervals to obtain a restored image.

[0053] Alternatively, the duration of accumulation may be variable. Specifically, the time interval to be restored may be determined with reference to the change rate of the number of events, and then the event data in the time interval may be reconstructed to obtain a restored image.

[0054] The processing of the restored image may include preprocessing, segmentation (may be omitted), classification (may be omitted) and position recognition.

[0055] Preprocessing may include noise reduction, enhancement, etc. to improve the image quality for subsequent processing. The specific processing method is not limited and traditional image processing and / or neural networks can be used. For example, spatio-temporal correlation filtering (such as Gaussian filtering, bilateral filtering, etc.) and neural networks (such as generative adversarial networks, etc.) can be used for noise filtering, and wavelet transform can be used for enhancement. Specifically, the restored image is convolved with wavelet basis functions to obtain detail coefficients and approximation coefficients at different scales. According to the detail coefficients and approximation coefficients, the signal is reconstructed through inverse wavelet transform to obtain the restored image with enhanced details.

[0056] The edge of the restored image can be extracted by traditional image processing methods, such as corner detection algorithms, Gaussian mixture models, feature detection algorithms, etc. Or, a segmentation model can be used to segment the restored image, which is not limited here.

[0057] A lightning classification model can be used to identify the type of lightning. Or, traditional image processing methods, such as optical flow estimation, feature extraction and fusion, etc., can be used to classify lightning. The types of lightning can include non-lightning and lightning. When the restored image is identified as non-lightning, it means that no lightning appears; when the restored image is identified as lightning, it means that lightning appears. In addition, lightning can include more specific types, such as linear, chain-shaped, etc.

[0058] A lightning position model can be used to identify the position of lightning. Or, traditional image processing methods, such as object tracking, object detection, etc., can be used to determine the position of lightning. The lightning position includes the coordinates of the strike point (which can also be called the end point) of the lightning, and in addition, it can also include the starting coordinates of the lightning. The number of starting points of linear lightning is 1. If there is no bifurcation, the number of end points is 1. If there is bifurcation, the number of end points is greater than 1. The number of starting points and end points of chain-shaped lightning is greater than 1.

[0059] Both the lightning classification model and the lightning position model include a feature extraction part and a regression analysis part. The feature extraction parts of the two models can be independent or shared. The regression analysis part of the lightning classification model outputs classification labels, and the lightning position model outputs the detection frames and their coordinates of the starting and ending points of the lightning channel. The specific network architectures of the lightning classification model and the lightning position model are not limited here. For example, at least one of convolutional neural networks, vision transformers, etc. can be adopted.

[0060] Optionally, the calculation module 20 is further configured to generate a curve of the lightning current over time according to the change of the number of events over time in the event data, and use the lightning model to process the restored image and the curve to obtain lightning information. The lightning model can include at least one of the lightning classification model and the lightning position model.

[0061] During a lightning strike, the number of brightening events collected by the dynamic vision sensor 10 is positively correlated with the current of the lightning. The number of events with positive polarity during a lightning strike can be extracted, and an empirical model between the number of events, light intensity and current can be established, thereby generating a curve of the change of the lightning current over time based on the change of the number of events over time. Specifically, multiple experiments can be conducted, and the experimental data can be fitted to calculate the model parameters, so as to convert the curve of the change of the number of events over time into the curve of the change of the lightning current over time. The change curve reflects the time domain information of the current change during a lightning strike, and the introduction of the change curve can improve the accuracy of the output results of the lightning model.

[0062] Optionally, the input of the lightning model includes at least one of a change curve and a wavelet transform result thereof. The basis function of the wavelet transform includes at least one of a Morlet wavelet basis function and a Daubechies wavelet basis function.

[0063] The change curve reflects the information in the time domain, and the time-frequency analysis can be performed on it through wavelet transform. Since lightning signals are usually short-term transient signals with sudden and non-stationary characteristics, the basis function needs to have a high time and frequency resolution. Specifically, the change curve of the number of events or lightning current over time can be used as a time series signal for wavelet transform to obtain the wavelet transform result of the change curve. The change curve and its wavelet transform result can be drawn as a two-dimensional image and input into the lightning model, thereby further improving the accuracy of the output result of the lightning model.

[0064] Optionally, the lightning information may also include lightning current. By integrating a curve of the change of the lightning current over time, the total current of the lightning strike process may be obtained.

[0065] Through the implementation of this embodiment, each pixel of the dynamic vision sensor can independently detect changes in light intensity. When a pixel detects that the light intensity change exceeds the set threshold, an event will be output, which includes a timestamp, the coordinates of the pixel, and the polarity of the event. The dynamic vision sensor simulates the visual perception system of organisms, and has the characteristics of high temporal resolution, less data redundancy, low power consumption, and high dynamic range. It can meet the detection needs of lightning with short time and drastic changes in light intensity. At the same time, the lightning monitoring device has strong adaptability to the environment and can be arranged in remote locations, on the top of buildings, etc. Together with the central station, it forms a lightning monitoring system, which can effectively monitor the power grid in real time. The event data collected by the dynamic vision sensor can identify lightning information, and the restored image can be reconstructed based on the event data to identify more accurate lightning coordinates, thereby improving the accuracy of lightning monitoring.

[0066] The lightning monitoring method provided by the embodiments of the present application can be implemented as a computer software program. For example, an embodiment of the present application provides a computer program product, which includes a computer program carried on a computer-readable medium. The computer program contains program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network, and / or installed from a removable external storage unit. When the computer program is executed by the computing module 20 / central station, various functions defined in the lightning monitoring method provided by the embodiments of the present application are implemented.

[0067] Figure 6 The schematic flowchart of the lightning monitoring method provided by an embodiment of the present application is shown. As an example but not a limitation, this method can be applied to the above-mentioned lightning monitoring device, and for specific details, reference can be made to the relevant content of the above embodiment.

[0068] S11: Collect event data through a dynamic vision sensor.

[0069] S12: Obtain lightning information according to the event data.

[0070] The lightning information includes whether lightning occurs.

[0071] The change rate of the number of events can be calculated according to the number of events; whether lightning occurs can be determined according to the change rate.

[0072] Alternatively, the event data can be restored to obtain a restored image, and the lightning information can be determined by using the restored image. The lightning information also includes the landing coordinates of the lightning.

[0073] Specifically, a change curve of lightning current over time can be generated according to the change of the number of events in the event data, and the lightning information can be obtained by processing the restored image and the change curve using a lightning model. The lightning model can include a lightning position model for identifying the landing coordinates.

[0074] The input of the lightning model includes at least one of the change curve and its wavelet transform result. The basis functions of the wavelet transform include at least one of the Morlet wavelet basis function and the Daubechies wavelet basis function.

[0075] S13: Send the lightning information to the central station.

[0076] Figure 7 The schematic flowchart of the lightning monitoring method provided by another embodiment of the present application is shown. As an example but not a limitation, this method can be applied to the above-mentioned central station, and for specific details, reference can be made to the relevant content of the above embodiment.

[0077] S21: Receive the lightning information from the lightning monitoring device.

[0078] The lightning information includes at least one of whether lightning occurs, event data related to lightning, and a restored image reconstructed based on the event data.

[0079] S22: When the lightning information sent by at least two lightning monitoring devices is associated with the same lightning strike process, determine the restored images of multiple perspectives of the lightning strike process according to the associated lightning information.

[0080] S23: Use multi-perspective synthesis technology to synthesize the restored images of multiple perspectives to obtain a synthesized image.

[0081] S24: Perform strike point recognition on the synthesized image to obtain the lightning strike coordinates.

[0082] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The order of execution of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0083] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above division of each functional unit and module is used as an example. In practical applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0084] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the foregoing method embodiments can be implemented.

[0085] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above method embodiments of this application, a computer program can be used to instruct relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the photographing device / lightning monitoring device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0086] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0087] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0088] In the embodiments provided in this application, it should be understood that the disclosed device / network device and method can be implemented in other ways. For example, the device / network device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0089] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0090] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included within the protection scope of the present application.

Claims

1. A lightning monitoring device, characterized in that: The device comprises: A dynamic vision sensor, used to collect event data of target pixels, wherein the target pixels are pixels whose light intensity changes exceed a set threshold within the monitoring range of the dynamic vision sensor, and the event data of each target pixel includes a timestamp, pixel coordinates and polarity of the pixel; A calculation module, used to extract the number of events with positive polarity and / or negative polarity in the lightning strike process from the event data, and obtain a curve of the number of events in the event data changing over time by fitting; convert the curve of the number of events changing over time into a curve of the lightning current changing over time; determine the lightning information of the monitoring range according to the curve of the lightning current changing over time, and the lightning information includes indication information of whether lightning occurs in the monitoring range; The communication module is used to send the lightning information to a central station; the central station is used to receive lightning information from multiple lightning monitoring devices and determine the lightning strike coordinates based on the received lightning information.

2. The device according to claim 1, characterized in that The calculation module is specifically used to reconstruct an image using the event data to obtain a restored image, and to determine lightning information using the restored image, wherein the lightning information also includes the coordinates of the lightning landing point.

3. The device according to claim 2, characterized in that The calculation module is also used to obtain the lightning information based on the restored image and the curve of the change of the lightning current over time according to the lightning model.

4. The device according to claim 3, characterized in that The input of the lightning model includes at least one of the variation curve and its wavelet transform result.

5. The device according to claim 4, characterized in that The basis function of the wavelet transform includes at least one of a Morlet wavelet basis function and a Daubechies wavelet basis function.

6. The device according to claim 3, characterized in that The lightning model includes a lightning position model, and the lightning position model is used to identify the falling point coordinates.

7. The device according to any one of claims 1 to 6, characterized in that: The calculation module is specifically used to calculate the change rate of the number of events according to the number of events; and determine whether lightning occurs according to the change rate.

8. The device according to any one of claims 1 to 6, characterized in that: The threshold voltage of the dynamic vision sensor is set to a first value higher than a default value, and the built-in filter of the dynamic vision sensor is set to a high-pass filter.

9. A lightning monitoring system, characterized in that: The system comprises a plurality of lightning monitoring devices according to any one of claims 1 to 8, and comprises a central station; The central station is used to continuously receive lightning information from multiple lightning monitoring devices, and process the lightning information reported by the multiple lightning monitoring devices to obtain the coordinates of the lightning strike.

10. The system according to claim 9, characterized in that The lightning information also includes at least one of lightning-related event data and a restored image reconstructed according to the event data, and the central station is specifically used for: When the lightning information sent by at least two of the plurality of lightning monitoring devices is associated with the same lightning strike process, determining restored images of the lightning strike process from multiple perspectives according to the lightning information of the at least two lightning monitoring devices; Using a multi-view synthesis technology to synthesize the restored images of the multiple viewpoints to obtain a synthesized image; The lightning strike coordinates are obtained by performing landing point recognition on the composite image.

11. A lightning monitoring method, characterized in that: Applied to a first lightning monitoring device, the first lightning monitoring device includes a dynamic visual sensor, a computing module and a communication module, the method includes: Collecting event data of target pixels through the dynamic vision sensor, wherein the target pixels are pixels whose light intensity changes within the monitoring range of the dynamic vision sensor exceed a set threshold, and the event data of each target pixel includes a timestamp, pixel coordinates and polarity of the pixel; Utilizing the calculation module, extracting the number of events with positive polarity and / or negative polarity in the lightning strike process from the event data, and fitting to obtain a curve of the number of events in the event data changing over time; converting the curve of the number of events changing over time into a curve of the lightning current changing over time; determining lightning information of the monitoring range according to the curve of the lightning current changing over time, wherein the lightning information includes indication information of whether lightning occurs in the monitoring range; The lightning information is sent to a central station through the communication module; the central station is used to receive lightning information from multiple lightning monitoring devices and determine the lightning strike coordinates according to the received lightning information, and the multiple lightning monitoring devices include the first lightning monitoring device.

12. A lightning monitoring method, characterized in that: Applied to a central station, the method comprises: Continuously receiving lightning information from multiple lightning monitoring devices, the lightning information of each lightning monitoring device includes indication information of whether lightning occurs in the monitoring range corresponding to the lightning monitoring device; the lightning information of each lightning detection device also includes event data of target pixel points collected by the lightning monitoring device through a dynamic visual sensor and / or a restored image reconstructed according to the event data; the target pixel points are pixel points within the monitoring range of the dynamic visual sensor whose light intensity changes exceed a set threshold, and the event data of each target pixel point includes the timestamp, pixel coordinates and polarity of the pixel point; In a case where the lightning information sent by at least two lightning monitoring devices is associated with the same lightning strike process, determining restored images of the lightning strike process from multiple perspectives according to the lightning information of the at least two lightning monitoring devices; Using a multi-view synthesis technology to synthesize the restored images of the multiple viewpoints to obtain a synthesized image; The strike point is identified on the composite image to obtain the strike coordinates of the lightning strike process.

13. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to claim 11 or 12 is implemented.

Images