A fast lightning recognition method based on brightness change information of image sequences

Abstract

The present application provides a method for rapid lightning identification based on brightness change information of an image sequence, which includes the following steps: obtaining an image at the current time and converting the image into a first grayscale image; obtaining N consecutive frames of contrast images within a preset period of time before the current time and converting the N frames of contrast images into N frames of second grayscale images; wherein the N frames of second grayscale images include a second grayscale image corresponding to the previous frame of the image at the current time, recorded as a third grayscale image; obtaining a candidate image based on the brightness entropy change value and its standard deviation between the first grayscale image and the N frames of second grayscale images; and judging whether there is a lightning channel in the candidate image based on the morphology of the highlight area. The advantages of the present application are that it can quickly and accurately identify the occurrence of lightning events, realize automatic and rapid identification of lightning events, solve the problem of false triggering that is prone to occur in optical observation of lightning, and effectively improve the observation efficiency and data quality.

Description

Technical Field

[0001] The present application relates to the field of meteorological detection, and in particular to a lightning recognition method based on image sequences. Background Art

[0002] Lightning often accompanies severe convective weather events (such as thunderstorms, hail, typhoons, and squall lines). Real-time monitoring and early warning, along with protective measures, can effectively reduce casualties and property damage. Lightning generates physical signals such as sound, light, electricity, and magnetism. Currently, the most widely used method for automated observation of lightning events involves the collection, analysis, and processing of electromagnetic signals. However, electromagnetic signal observation is susceptible to interference, requires high precision in multi-station time synchronization, and can lead to both systematic and random errors in positioning results. With the continuous advancement of various technologies and the continuous upgrading of application requirements, the demand for lightning detection accuracy is becoming increasingly stringent. In some application scenarios, detection results based on electromagnetic signals no longer meet the required accuracy.

[0003] Optical observation is another method for lightning observation. This method directly captures the optical radiation signals of the lightning channel. It can intuitively provide accurate and reliable information about the lightning channel and lightning location. It is a very important lightning observation method. Many important discoveries in lightning physics research have come from analyzing optical observation results. Optical data is an important basis for verifying and evaluating the quality of lightning electromagnetic detection results.

[0004] Lightning is an instantaneous, rapidly changing electrical discharge event, with some flashes lasting only a few milliseconds. Therefore, high-quality optical observation of lightning requires high-speed filming. While the advent of commercial high-speed cameras has addressed the technical bottleneck of high-speed photography, automated optical observation of lightning remains challenging, primarily due to the instantaneous and sporadic nature of lightning. Triggered recording, which records only when lightning occurs, rather than continuously, is an appropriate detection method. This requires reliable and accurate triggering conditions.

[0005] Currently, most lightning analysis studies based on optical observations use semi-automatic or manual triggering to acquire data. Automatic triggering relies primarily on electromagnetic or light intensity signals, which often results in frequent false triggering and results in a large number of images without lightning. Furthermore, high frame rates result in extremely high data volumes, requiring extensive effort to screen out images showing lightning before use. This significantly reduces observation efficiency and data acquisition rates. Existing traditional algorithms for lightning optical image recognition are based on image morphology, which is computationally intensive and inadequate for fast, real-time recognition. Patent No. CN116797823A discloses a lightning channel image recognition method that uses an edge detection algorithm to obtain lightning outlines and then identifies lightning channels through threshold segmentation and morphological processing. This type of lightning recognition method based on image morphology is targeted at single images and can achieve pixel-level recognition accuracy, aiming to identify all lightning channels within the image. Due to the algorithmic complexity and high computational effort, it is suitable for applications where computational time is not a priority. Therefore, it is only suitable for accurately identifying lightning channels within recorded lightning images, but is not suitable for real-time determination of lightning events.

[0006] In summary, there is a need in the art to provide an accurate and fast image-based lightning event recognition method to overcome the shortcomings of the existing technology. Summary of the Invention

[0007] The present application provides a lightning recognition method based on image sequences. This method is not based on a single frame image but on continuous frame images acquired over a period of time, which can solve the problems existing in the prior art. The purpose of the present application is achieved through the following technical solutions.

[0008] In a first aspect, an embodiment of the present application provides a lightning recognition method based on an image sequence, which uses real-time image information to quickly determine whether a lightning event has occurred and provide a reliable trigger signal for optical observation of lightning. The method includes the following steps:

[0009] Step 1: Obtain the image at the current time, obtain the image at the current time, and convert the image into a first grayscale image;

[0010] Step 2: Obtain N consecutive contrast images within a preset time period before the current time, and convert the N contrast images into N second grayscale images; wherein the N second grayscale images include the second grayscale image corresponding to the previous frame of the image at the current time, which is recorded as the third grayscale image;

[0011] Step 3: Obtain a candidate image based on the brightness entropy change value and its standard deviation between the first grayscale image and N frames of second grayscale images; and

[0012] Step 4: Determine whether there is a lightning channel in the candidate image based on the morphology of the highlighted area of ​​the candidate image.

[0013] According to the lightning recognition method based on image sequences provided in one embodiment of the present application, step 3: obtaining a candidate image based on the brightness entropy change value and the standard deviation between the first grayscale image and N frames of second grayscale images includes the following steps:

[0014] Step 31: Calculate the brightness entropy of the first grayscale image and each frame of the second grayscale image;

[0015] Step 32: Divide the N frames of second grayscale images into M groups according to the time sequence of shooting, with the number of second grayscale images in each group being n, and calculate the average change value V of the brightness entropy of each group of second grayscale images;

[0016] Step 33: calculating the standard deviation σ of the average change value V of the M groups of second grayscale images according to the average change value V of the brightness entropy of each group of second grayscale images;

[0017] Step 34: Calculate the brightness entropy change value Vi between the first grayscale image and the third grayscale image; and

[0018] Step 35: Determine whether Vi is greater than or equal to a preset first threshold. If yes, execute step 4; if no, the process ends.

[0019] According to the lightning recognition method based on image sequence provided by one embodiment of the present application, step 4: judging whether there is a lightning channel in the candidate image based on the morphology of the highlight area includes the following steps;

[0020] Step 41: Obtain each sudden change pixel in the first grayscale image whose brightness change exceeds a preset second threshold compared with the same position in the third grayscale image;

[0021] Step 42: setting the mutation pixels connected in position as a region of interest; wherein the mutation pixels may form multiple independent regions of interest;

[0022] Step 43: Select a large area of ​​interest as the target area;

[0023] Step 44: Determine whether the target area meets the linear feature. If yes, go to step 45; if no, go to step 46.

[0024] Step 45: determining that the image at the current time includes lightning, and then the processing ends; and

[0025] Step 46: Determine that there is no lightning in the image at the current time, and then the process ends.

[0026] According to the lightning recognition method based on image sequence provided by one embodiment of the present application, the brightness entropy calculation formula of each frame of grayscale image is:

[0027] ,in ,

[0028] Where E is the brightness entropy of the image, x(i,j) refers to the brightness value of the pixel at position (i,j) in the image, and the brightness value range is [0, 255].

[0029] According to the lightning recognition method based on image sequences provided by one embodiment of the present application, the calculation formula for the average change value V of the brightness entropy of each group of second grayscale images is: , where Ei is the brightness entropy of the i-th image.

[0030] According to the lightning recognition method based on image sequences provided by one embodiment of the present application, the formula for calculating the standard deviation σ of the average change value V of the brightness entropy of M groups of second grayscale images based on the average change value V of the brightness entropy of each group of second grayscale images is:

[0031] σ= ,in , is the average entropy change of group i in M ​​groups.

[0032] According to the lightning recognition method based on image sequences provided in one embodiment of the present application, step 44: determining whether the target area meets the linear feature includes the following steps:

[0033] Step 441: forming a circumscribed quadrilateral of the target area with the top, bottom, left, and right vertices of the target area, and obtaining the aspect ratio D of the circumscribed quadrilateral; and

[0034] Step 442: Determine whether the aspect ratio D is greater than the preset aspect ratio d. If yes, go to step 45; if no, go to step 46.

[0035] According to the image sequence-based lightning recognition method provided by one embodiment of the present application, the first threshold is Aσ, where A is a preset coefficient.

[0036] The image sequence-based lightning recognition method disclosed in this application can be used in conjunction with an existing lightning channel observation system. The lightning channel observation system includes a shooting unit and a control component. The control component is electrically connected to the shooting unit. The shooting unit takes pictures and transmits the images to the control component. When the control component uses the image sequence-based lightning recognition method of an embodiment of the present application to analyze the images of the shooting unit and determines that a lightning event has occurred, the control component records the time when the lightning event occurred and stores the images captured by the shooting unit. The control component includes a chassis, a processor, a GPS antenna, a GPS timing unit, and a power supply. The processor is disposed in the chassis and is electrically connected to the shooting unit. The GPS antenna is mounted on the outside of the chassis, the GPS timing unit is mounted in the chassis, the GPS antenna is electrically connected to the GPS timing unit, and the GPS timing unit is electrically connected to the processor. The power supply is disposed in the chassis and is connected to the processor.

[0037] In a second aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein when the one or more computer instructions are executed by the processor, the image sequence-based lightning recognition method shown in the first aspect above is implemented.

[0038] In a third aspect, an embodiment of the present application provides a computer storage medium for storing a computer program, wherein the computer program enables a computer to implement the lightning recognition method based on image sequences as shown in the first aspect above when executed.

[0039] In a fourth aspect, an embodiment of the present application provides a computer program product, comprising: a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, causes the one or more processors to perform the steps of the image sequence-based lightning identification method of the first aspect.

[0040] The advantages of the lightning recognition method based on image sequences according to the embodiments of the present application are that it can quickly and accurately identify the occurrence of lightning events, avoiding the multi-level convolution and matrix algorithms required for image morphology with huge computational complexity (the basic calculation method of the convolution algorithm is to slide pixel matrices of different sizes on the image for calculation, and the computational complexity increases exponentially with the increase of image pixels). The computational complexity is only one thousandth of that of the morphological algorithm, and lightning events can be identified within milliseconds, solving the problem of false triggering that is prone to occur in lightning optical observations, realizing automatic and rapid identification of lightning events, improving the efficiency of obtaining lightning optical data and the observation quality, and truly realizing fully automatic optical observation of lightning; it is not only beneficial to lightning scientific research, but also has high application value in lightning accident investigation, lightning activity warning, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Other features, objects and advantages of the present application will become more apparent through the detailed description of non-limiting embodiments of the present application with reference to the following drawings.

[0042] Figure 1 A flowchart of a lightning recognition method based on image sequences according to one embodiment of the present application is shown.

[0043] Figure 2 Shown as Figure 1 The flowchart shown is for determining whether a target area meets the linear feature in a lightning recognition method based on an image sequence according to one embodiment of the present application.

[0044] Figure 3 A lightning channel observation system based on an image sequence-based lightning identification method according to an embodiment of the present application is shown.

[0045] Figure 4 The third grayscale image converted from the previous frame image at the current moment is shown.

[0046] Figure 5 The first grayscale image of the image conversion at the current moment is shown. DETAILED DESCRIPTION

[0047] The following describes the specific implementation methods of the present application in conjunction with the accompanying drawings and examples. Through the contents of this specification, those skilled in the art can clearly and completely understand the technical solutions, technical problems solved, and technical effects produced by the present application. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. In addition, for ease of description, only the parts relevant to the present application are shown in the accompanying drawings.

[0048] It should be noted that the structures, proportions, sizes, etc. depicted in the drawings of the specification are only used to match the contents recorded in the specification for technical personnel in this field to understand and read, and are not used to limit the conditions for the implementation of this application. Therefore, they have no substantive technical significance. Any structural modification, change in proportional relationship or adjustment of size should fall within the scope of the technical content disclosed in this application without affecting the efficacy and purpose that can be achieved by this application.

[0049] References such as "first", "second", "the" and similar words do not indicate a quantitative limitation and may indicate the singular or plural. The terms "include", "comprising", "having" and any variations thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The terms "connect", "connected", "coupled" and similar words involved in this application are not limited to physical or mechanical connections, but may also include direct or indirect electrical connections.

[0050] Firstly, Figure 1 FIG. 1 shows a flow chart of a lightning recognition method based on an image sequence according to an embodiment of the present application. Figure 1 As shown in FIG, the lightning recognition method based on image sequence includes multiple steps:

[0051] Step 1: Acquire an image at the current time and convert the image into a first grayscale image; wherein the method of converting the image into a grayscale image includes but is not limited to: an average method, a weighted method, or a maximum / minimum method; for example: Figure 5 A first grayscale image of the image conversion at the current moment is shown;

[0052] Step 2: Obtain N consecutive contrast images within a preset period of time before the current time, and convert the N contrast images into N second grayscale images; wherein the N second grayscale images include the second grayscale image corresponding to the previous frame of the image at the current time, which is recorded as the third grayscale image; for example: Figure 4 shows a third grayscale image converted from the previous frame image at the current moment;

[0053] Step 3: Obtain a candidate image based on the brightness entropy change value and its standard deviation between the first grayscale image and N frames of second grayscale images; and

[0054] Step 4: Determine whether there is a lightning channel in the candidate image based on the morphology of the highlighted area.

[0055] According to the lightning recognition method based on image sequences provided in one embodiment of the present application, step 3: obtaining a candidate image based on the brightness entropy change value and the standard deviation between the first grayscale image and N frames of second grayscale images includes the following steps:

[0056] Step 31: Calculate the brightness entropy of the first grayscale image and each frame of the second grayscale image;

[0057] Step 32: Divide the N frames of second grayscale images into M groups according to the time sequence of shooting, with the number of second grayscale images in each group being n, and calculate the average change value V of the brightness entropy of each group of second grayscale images;

[0058] Step 33: calculating the standard deviation σ of the average change value V of the M groups of second grayscale images according to the average change value V of the brightness entropy of each group of second grayscale images;

[0059] Step 34: Calculate the brightness entropy change value Vi between the first grayscale image and the third grayscale image; and

[0060] Step 35: Determine whether Vi is greater than or equal to a preset first threshold. If yes, execute step 4; if no, the process ends.

[0061] According to the lightning recognition method based on image sequence provided by one embodiment of the present application, step 4: judging whether there is a lightning channel in the candidate image based on the morphology of the highlight area includes the following steps;

[0062] Step 41: Obtain each sudden change pixel in the first grayscale image whose brightness change exceeds a preset second threshold compared with the same position in the third grayscale image;

[0063] Step 42: setting the mutation pixels connected in position as a region of interest; wherein the mutation pixels may form multiple independent regions of interest;

[0064] Step 43: Select a large area of ​​interest as the target area;

[0065] Step 44: Determine whether the target area meets the linear feature. If yes, go to step 45; if no, go to step 46.

[0066] Step 45: determining that the image at the current time includes lightning, and then the processing ends; and

[0067] Step 46: Determine that there is no lightning in the image at the current time, and then the process ends.

[0068] According to the lightning recognition method based on image sequence provided by one embodiment of the present application, the brightness entropy calculation formula of each frame of grayscale image is:

[0069] ,in ,

[0070] Where E is the brightness entropy of the image, x(i,j) refers to the brightness value of the pixel at position (i,j) in the image, and the range of brightness values ​​is [0, 255].

[0071] According to the lightning recognition method based on image sequences provided by one embodiment of the present application, the calculation formula for the average change value V of the brightness entropy of each group of second grayscale images is:

[0072] , where Ei is the brightness entropy of the i-th image.

[0073] According to the lightning recognition method based on image sequences provided by one embodiment of the present application, the formula for calculating the standard deviation σ of M groups of second grayscale images based on the average change value V of the brightness entropy of each group of second grayscale images is:

[0074] σ= ,in , is the average entropy change of group i in M ​​groups.

[0075] According to the above-mentioned embodiment of the present application, a lightning recognition method based on an image sequence is provided, wherein the first threshold is Aσ, where A is a preset coefficient, for example, a positive integer such as 2, 3, etc.; the second threshold is twice the brightness of the pixel point in the third grayscale image, and the maximum value is 255.

[0076] Figure 2 Shown as Figure 1 The flowchart shown in FIG. 1 is a method for determining whether a target area meets the linear feature in a lightning recognition method based on an image sequence according to an embodiment of the present application. Figure 2 As shown, step 44: determining whether the target area meets the linear feature includes the following steps:

[0077] Step 441: forming a circumscribed quadrilateral of the target area with the top, bottom, left, and right vertices of the target area, and obtaining the aspect ratio D of the circumscribed quadrilateral; and

[0078] Step 442: Determine whether the aspect ratio D is greater than a preset aspect ratio d, preferably, d is greater than 10; if “yes”, execute step 45; if “no”, execute step 46.

[0079] According to the image sequence-based lightning recognition method provided by one of the above-mentioned embodiments of the present application, determining whether the target area meets the linear feature refers to determining whether the shape of the target area is linear. Commonly used methods are edge detection (based on algorithms such as Canny and Sobel) and Hough Transform. However, the computational complexity of such algorithms will increase exponentially with the improvement of image resolution and the increase of edge complexity. Therefore, they are not suitable for the scenarios targeted by the present application where it is necessary to quickly determine whether lightning has occurred. However, linear feature judgment based on aspect ratio can greatly improve the judgment speed. When the aspect ratio value is large, the target area can be determined to be linear.

[0080] According to the image sequence-based lightning recognition method provided by the above-mentioned embodiment of the present application, the preset aspect ratio d has a value range of 20-50. Preferably, the value of the aspect ratio d is 20.

[0081] The image sequence-based lightning recognition method disclosed in this application can be used in conjunction with an existing lightning channel observation system. Figure 3 FIG shows a lightning channel observation system based on an image sequence-based lightning recognition method according to an embodiment of the present application. Figure 3 As shown, the lightning channel observation system includes a shooting unit 901 and a control component 902. The control component 902 is electrically connected to the shooting unit 901. The shooting unit 901 shoots and transmits the image to the control component 902. The control component 902 pre-stores a continuous frame image sequence of a certain length and analyzes the received image sequence using the lightning recognition method based on image sequence according to an embodiment of the present application. When a lightning event is determined to have occurred, the control component 902 records the time of the lightning event and formally stores the image sequence captured by the shooting unit 901. The control component 902 includes a chassis 9021, a processor 9022, a GPS antenna 9023, a GPS timing unit 9024, and a power supply 9025. The processor 9022 is disposed in the chassis 9021 and is electrically connected to the shooting unit 901. The GPS antenna 9023 is mounted outside the chassis 9021, and the GPS timing unit 9024 is mounted inside the chassis 9021. The GPS antenna 9023 is electrically connected to the GPS timing unit 9024, and the GPS timing unit 9024 is electrically connected to the processor 9022. A power supply 9025 is provided inside the chassis 9021 and connected to the processor 9022.

[0082] In a second aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein when the one or more computer instructions are executed by the processor, the image sequence-based lightning recognition method shown in the first aspect above is implemented.

[0083] In a third aspect, an embodiment of the present application provides a computer storage medium for storing a computer program, wherein the computer program enables a computer to implement the lightning recognition method based on image sequences as shown in the first aspect above when executed.

[0084] In a fourth aspect, an embodiment of the present application provides a computer program product, comprising: a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, causes the one or more processors to perform the steps of the image sequence-based lightning identification method of the first aspect.

[0085] The advantages of the lightning recognition method based on image sequences according to the implementation mode of the present application are: it can identify the occurrence of lightning events in real time, quickly (millisecond level), and accurately, solve the problems of reliable triggering of optical lightning observation and automatic and rapid identification of lightning events, improve the efficiency of obtaining lightning optical data, and truly realize fully automatic optical observation of lightning. It is not only beneficial to lightning scientific research, but also has high application value in lightning accident investigation, lightning activity warning, etc.

[0086] Although the present application has been described and illustrated with reference to specific embodiments of the present application, these descriptions and illustrations are not intended to limit the present application. It will be clearly understood by those skilled in the art that various changes may be made and that equivalent elements may be substituted within the embodiments without departing from the scope of protection of the present application as defined by the claims. Due to variables in the manufacturing process, etc., there may be differences between the technical reproduction in the present application and the actual device. There may be other embodiments of the present application that are not specifically described. The description and illustrations should be regarded as illustrative, not restrictive, and modifications may be made according to the purpose and spirit of the present application, all of which are within the scope of protection of the claims. Although the methods disclosed herein have been described with reference to specific operations performed in a specific order, it should be understood that these operations may be recombined, subdivided, or arranged to form equivalent methods without departing from the teachings of the present application. Therefore, unless specifically indicated herein, the order and grouping of operations do not limit the present application.

Claims

1. A lightning fast identification method based on image sequence brightness change information, characterized in that: It involves several steps: Step 1: Get the image at the current time and convert the image into the first grayscale image; Step 2: Obtain N consecutive contrast images within a preset period of time before the current time, and convert the N contrast images into N second grayscale images; wherein the N second grayscale images include the second grayscale image corresponding to the previous frame of the image at the current time, which is recorded as the third grayscale image; Step 3: Obtain a candidate image based on the brightness entropy change value and its standard deviation between the first grayscale image and N frames of second grayscale images; The following steps are involved: Step 31: Calculate the brightness entropy of the first grayscale image and each frame of the second grayscale image; Step 32: Divide the N frames of second grayscale images into M groups according to the time sequence of shooting, with the number of second grayscale images in each group being n, and calculate the average change value V of the brightness entropy of each group of second grayscale images; Step 33: calculating the standard deviation σ of the average change value V of the M groups of second grayscale images according to the average change value V of the brightness entropy of each group of second grayscale images; Step 34: Calculate the brightness entropy change value Vi between the first grayscale image and the third grayscale image; and Step 35: Determine whether Vi is greater than or equal to a preset first threshold. If yes, proceed to step 4; if no, the process ends. The first threshold is Aσ, where A is a preset coefficient. as well as Step 4: Determine whether there is a lightning channel in the candidate image based on the morphology of the highlighted area.

2. The lightning fast identification method based on image sequence brightness change information according to claim 1, characterized in that: Step 4: Determining whether there is a lightning channel in the candidate image based on the morphology of the highlight area includes the following steps; Step 41: Obtain each sudden change pixel in the first grayscale image whose brightness change exceeds a preset second threshold compared with the same position in the third grayscale image; Step 42: setting the mutation pixels connected in position as a region of interest; wherein the mutation pixels form a plurality of independent regions of interest; Step 43: Select a large area of ​​interest as the target area; Step 44: Determine whether the target area meets the linear feature. If yes, go to step 45; if no, go to step 46. Step 45: determining that the image at the current time includes lightning, and then the processing ends; and Step 46: Determine that there is no lightning in the image at the current time, and then the process ends.

3. The lightning fast identification method based on image sequence brightness change information according to claim 1, characterized in that: The calculation formula for the brightness entropy of each frame of grayscale image is: ,in , Where E is the brightness entropy of the image, x(i,j) refers to the brightness value of the pixel at position (i,j) in the image, and the brightness value range is [0, 255].

4. The lightning fast identification method based on image sequence brightness change information according to claim 3, characterized in that: The calculation formula for the average change value V of the brightness entropy of each group of second grayscale images is: , where Ei is the brightness entropy of the i-th image.

5. The lightning fast identification method based on image sequence brightness change information according to claim 4 is characterized in that: The calculation formula for calculating the standard deviation σ of M groups of second grayscale images according to the average change value V of the brightness entropy of each group of second grayscale images is: σ= ,in , is the average entropy change of group i in M ​​groups.

6. The lightning fast identification method based on image sequence brightness change information according to claim 2, characterized in that: Step 44: Determining whether the target area meets the linear feature includes the following steps: Step 441: Using the top, bottom, left, and right sides of the target area as vertices, form a circumscribed quadrilateral of the target area, and obtain the aspect ratio D of the circumscribed quadrilateral; as well as Step 442: Determine whether the aspect ratio D is greater than the preset aspect ratio d. If yes, go to step 45; if no, go to step 46.

7. The lightning fast identification method based on image sequence brightness change information according to claim 6, characterized in that: The preset value range of the aspect ratio d is 20-50.

Images