Method and device for counting flashing light points

Abstract

To provide a blinking light spot counting method and device which can accurately count the number of blinking light spots from a dynamic image in such a situation that the blinking light spots that may move are scattered.SOLUTION: A blinking light spot counting device (200) generates a light spot image Fh from dynamic image data made of a prescribed first frame number (n) (301), determines whether or not a light spot is a blinking light spot for each of the plurality of light spots obtained by substantially overlapping dynamic images for each second frame number (p) of the first frame number (304), determines the same blinking light spot if the first and second blinking light spots are within a prescribed distance on the basis of position information of the first blinking light spot obtained for each second frame number and position information of the second blinking light spot obtained previously (309), and counts these blinking light spots as one when the blinking light spots are the same blinking light spot, and counts as the different blinking light spots in other cases (310).SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a method and apparatus for counting the number of potentially moving flashing light points. [Background technology]

[0002] The number of glowing fireflies is a typical example of an indicator of the state of a rural ecosystem or natural environment. However, it is not easy to count the glowing fireflies, and it is especially difficult to count fireflies that are not flying. Therefore, in the case of Genji fireflies and Heike fireflies, a method is usually adopted in which people visually count the area within two hours after sunset when the fireflies are flying (see Non-Patent Document 1).

[0003] Methods for counting the number of individuals using images have also been proposed. For example, Patent Document 1 discloses an individual counting device that divides an image capture screen into multiple small regions and calculates the number of individuals on the entire screen from the sum of the numbers of individuals present in the small regions. Patent Document 2 also discloses a method for removing bright spot noise when capturing a video of a low-luminance subject by cumulatively adding up image data received multiple times with the same or different exposure times and counting the number of individuals of the subject based on the added image data.

[0004] Furthermore, Patent Document 3 discloses a method of capturing video of people passing through a predetermined passage, identifying the direction of movement from the video image, and counting the number of people passing through. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 5-108822 [Patent Document 2] Japanese Patent Application Laid-Open No. 2005-215711 [Patent Document 3] Japanese Patent Application Laid-Open No. 2005-148863 [Non-patent literature]

[0006] [Non-Patent Document 1] Monitoring Site 1000 Satochi Survey Manual "Firefly" ver.3.1 (Feb. 2015) Published by the Biodiversity Center of the Ministry of the Environment, Nature Conservation Bureau Summary of the Invention [Problem to be solved by the invention]

[0007] However, the methods for counting the number of individuals disclosed in the above Patent Documents 1 and 2 are based on the premise that the subject is captured on the image screen. Therefore, when capturing a video of a blinking individual such as a firefly, the presence of a light spot may or may not occur depending on the captured frame, making it impossible to accurately count the number of individuals from the image screen.

[0008] Furthermore, the counting method disclosed in Patent Document 3 requires a device to move the individuals to be counted in a certain direction, and therefore cannot be applied to situations where individuals are scattered in a specified area.

[0009] The present invention has been devised in consideration of the above circumstances, and its object is to provide a flashing light spot counting method and device that can accurately count the number of flashing light spots from a moving image in a situation where flashing light spots that may move are scattered. [Means for solving the problem]

[0010] In order to achieve the above-mentioned object, a flashing light spot counting device according to one embodiment of the present invention is a device that counts flashing light spots from moving image data consisting of frame images, and comprises a first memory unit that stores moving image data consisting of a predetermined first number of frames, a second memory unit that stores information about the flashing light spots, and a processor that counts flashing light spots based on the information about the flashing light spots, wherein the processor determines whether each of a plurality of light spots obtained by sequentially superimposing moving images for each second number of frames out of the first number of frames is a flashing light spot, stores position information of light spots determined to be flashing light spots in the second memory unit, and registers the first and second flashing light spots as the same flashing light spot in the second memory unit if they are within a predetermined distance based on the position information of the first flashing light spot obtained for each second number of frames and the position information of the second flashing light spot obtained previously, and counts the same flashing light spots as one based on the registration information in the second memory unit, and counts flashing light spots that are further away from the predetermined distance as separate flashing light spots. According to one embodiment of the present invention, for each of the multiple light spots obtained by sequentially superimposing moving images for each of the second number of frames, it is possible to determine whether the light spot is a flashing light spot based on the number of frames that include the light spot and the number of frames that do not include the light spot. According to one embodiment of the present invention, the device may further include a display unit, and if the first and second flashing light points are determined to be the same flashing light point, the first and second flashing light points may be linked and displayed on the display unit, and if they are determined to be separate flashing light points, they may be displayed as separate light points on the display unit. According to an embodiment of the present invention, the flashing light points may be the blinking points of a luminous firefly. In order to achieve the above-mentioned object, a flashing light spot counting method according to one embodiment of the present invention is a method in which a processor counts flashing light spots from moving image data consisting of frame images, storing moving image data consisting of a predetermined first number of frames in a first memory unit, sequentially superimposing the moving images for each second number of frames out of the first number of frames to extract multiple light spots, determining for each of the multiple light spots whether the light spot is a flashing light spot or not, storing position information of light spots determined to be flashing light spots in a second memory unit, and based on the position information of the first flashing light spot obtained for each second number of frames and the position information of the second flashing light point obtained previously, registering the first and second flashing light spots as the same flashing light spot in the second memory unit if they are within a predetermined distance, and counting the same flashing light spots as one based on the registration information in the second memory unit, and counting flashing light spots that are further away from the predetermined distance as separate flashing light spots. According to one embodiment of the flashing light spot counting method of the present invention, for each of the multiple light spots obtained by sequentially superimposing moving images for each of the second frame numbers, it is possible to determine whether the light spot is a flashing light spot based on the number of frames that include the light spot and the number of frames that do not include the light spot. According to one embodiment of the flashing light spot counting method of the present invention, the number of flashing light spots can be determined by counting the number of flashing light spots multiple times for each of the first frame numbers and statistically processing the number of flashing light spots obtained by the multiple counts. According to the flashing light spot counting method according to an embodiment of the present invention, the flashing light spots may be blinking spots of luminous fireflies. In order to achieve the above-mentioned object, one embodiment of the present invention provides a program that causes a computer to function as a device that counts flashing light spots from moving image data consisting of frame images, and is characterized in that the program implements the following functions on the computer: storing moving image data consisting of a predetermined first number of frames in a first memory unit; sequentially superimposing moving images for each second number of frames out of the first number of frames to extract multiple light spots; determining for each of the multiple light spots whether or not the light spot is a flashing light spot; storing position information of light spots determined to be flashing light spots in a second memory unit; determining that the first and second flashing light spots are the same flashing light spot if they are within a predetermined distance based on the position information of the first flashing light spot obtained for each second number of frames and the position information of the second flashing light point obtained previously; registering the first and second flashing light spots as the same flashing light spot if they are within a predetermined distance; and counting the same flashing light spots as one and flashing light spots that are further away from the predetermined distance as separate flashing light spots based on the registration information in the second memory unit. [Effects of the Invention]

[0011] According to one embodiment of the present invention, a determination is made as to whether each of a plurality of light spots extracted by sequentially superimposing moving images is a blinking light spot, and whether they are the same blinking light spot is determined based on the position information of a first blinking light spot in the superimposed image and the position information of a second blinking light spot in a previously obtained superimposed image. This makes it possible to extract blinking light spots and determine whether they are the same light spot or multiple light spots even when multiple light spots are captured due to the movement of a single light spot, thereby enabling more accurate counting of the number of blinking light spots. Furthermore, according to one embodiment of the present invention, for each of a plurality of light spots, it is determined whether the light spot is a flashing light spot based on the number of frames that include the light spot and the number of frames that do not include the light spot, so that it is possible to accurately determine whether the light spot is a flashing light spot simply by comparing the number of frames. According to one embodiment of the present invention, if the flashing light points are determined to be the same, the flashing light points are linked and displayed on the display unit, and if the flashing light points are determined to be separate, they are displayed as separate light points, making it easier for the user to recognize the movement of the flashing light points. According to one embodiment of the present invention, it is possible to accurately count the number of fireflies from a moving image of the luminescent fireflies. According to one embodiment of the present invention, the number of flashing light points is counted multiple times and the obtained number of flashing light points is statistically processed, thereby excluding abnormal values ​​and more accurately determining the number of flashing light points. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a schematic configuration of a flashing light point counting device according to an embodiment of the present invention; [Figure 2] FIG. 2 is a diagram illustrating a flow of a blinking light point counting method according to the present embodiment. [Figure 3] 1 is a flowchart illustrating an example of a flashing light point counting method according to an embodiment of the present invention. [Figure 4] 4 is a flowchart showing an example of an identical light spot determination process in FIG. 3. [Figure 5] 10A and 10B are schematic diagrams illustrating an example of frame superposition in the blinking light point counting method according to the present embodiment. [Figure 6] 10A and 10B are diagrams illustrating examples of displays of the flashing light point counting device according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, the configuration and operation of a flashing light point counting device according to an embodiment of the present invention will be described in detail with reference to the drawings.

[0014] As shown in FIG. 1, an imaging unit 101 captures a moving image of blinking light spots G1, G2, G3, etc. as subjects. A blinking light spot counting device 200 according to this embodiment is connected to the imaging unit 101 via a wired or wireless interface 201. The blinking light spot counting device 200 is implemented on a computer and includes a processor 202 that implements a blinking light spot determination unit and a same light spot determination unit (described later), a moving image memory 203 (first storage unit), a program memory 204, an input unit 205, a display unit 206, and a memory 207 (second storage unit) that registers various data. The processor 202 executes a program in the program memory 204 to implement a blinking light spot counting function, including the functions of the blinking light spot determination unit and the same light spot determination unit (described later).

[0015] The video memory 203 stores video data input from the imaging unit 101 through the interface 201. The video data may be in a known file format with a constant frame rate. In the following description, it is assumed that a plurality of (m) blinking light points G1, G2, G3, ... Gm are captured in the video.

[0016] The input unit 205 is an input device such as a keyboard, etc. The display unit 206 is a monitor, and as will be described later, the blinking light points and the movement loci of the blinking light points are displayed on the screen together with the counted number of blinking light points.

[0017] When counting the number of fireflies, images are taken from sunset to night, so it is desirable to use a low-noise, high-sensitivity video camera as the camera 101. This makes it possible to accurately count the number of fireflies that emit faint flashing light that cannot be seen with the naked eye from the video images.

[0018] <Outline of flashing light counting process> Next, an outline of the counting process of the flashing light point counting device according to this embodiment will be described with reference to FIG.

[0019] In FIG. 2, frame images F0 to F1 for N seconds are stored in the video memory 203. nare read in sequentially, and light points having a luminance equal to or greater than a predetermined threshold are extracted from each frame image F to generate light point images Fh0 to Fh n are generated sequentially.

[0020] Each time a light spot image for a predetermined number p of frames is generated, the light spot images for the p frames are superimposed, and for each light spot, if the number of frames in which it appears is equal to or less than a predetermined threshold Hth, it is determined to be a blinking light spot. For example, if p=18 and Hth=5, a light spot that is lit in 5 or fewer frames out of 18 frames and not lit in the other frames is determined to be a blinking light spot. The magnitude of the predetermined threshold Hth can be determined based on the frame rate of the moving image, the number of frames p, the blinking period of the blinking light spot, the blinking pattern, etc. Alternatively, the predetermined threshold Hth can be determined based on the proportion of the target blinking light spot that appears in a predetermined number of frames (e.g., p frames). As will be described in detail later, the blinking light spot determination is a process of removing non-blinking light spots, thereby generating a blinking light spot superimposed image Fs consisting only of blinking light spots. In Figure 2, the blinking light spot determination is performed for each p frames, and the blinking light spot superimposed images Fs0 to Fs m are generated sequentially.

[0021] Next, the flashing light spot superimposed image Fs0~Fs m Each time each of the overlapping images Fs is generated, a determination is made as to whether it is a movement of one flashing light spot or a different flashing light spot. i When generated, the past superimposed image Fs i-1 The coordinates (position) of each light spot are compared between the two, and if the movement is less than a predetermined distance, it is determined to be the same flashing light spot (same individual). If they are the same flashing light spot, they are counted as one, otherwise they are counted as separate light spots. In this way, the number of flashing light spots (individuals) is counted from the video image over N seconds.

[0022] The above blinking light point counting process is performed every N seconds, and the results are tallied or statistically processed to determine the appropriate number of blinking light points. In practice, it is desirable to perform the counting process multiple times every N seconds, remove the highest and lowest values, and then take the average or median of the remaining values. An example of the blinking light point counting process performed every N seconds will be described below with reference to Figures 3 to 5.

[0023] <Flashing light detection> In FIG. 3, the processor 202 reads frame images F0 to F1 for N seconds from the video memory 203. n and extracting light points with brightness above a predetermined threshold from each frame image F to create light point images Fh0 to Fh n are generated sequentially (step 301).

[0024] Next, the processor 202 generates the light point images Fh0 to Fh n is superimposed every predetermined number p of frames, and light spots G in the superimposed image are sequentially extracted (step 303). It is determined whether the number of frames in which the extracted light spot G shines among the p frames is equal to or less than a predetermined threshold Hth (step 304), where the predetermined threshold Hth is a value greater than 0 and less than p. If the light spot G shines in more frames than the predetermined threshold Hth (NO in step 304), it is determined that the light spot G is not a flashing light spot, and the process returns to step 303 to extract the next light spot G. If the light spot G is equal to or less than the predetermined threshold Hth (YES in step 304), it is determined that the light spot G is a flashing light spot, and its position information (coordinates) within the screen is registered in memory 207 (step 305).

[0025] The above steps 303 to 305 are executed for all light spots G (NO in step 306). When the processing for all light spots has been completed (YES in step 306), processor 202 registers the coordinates and frame numbers registered as blinking light spots as a blinking light spot superimposed image Fs in memory 207 (step 307). Processor 202 then returns to step 302 and repeats the processing of steps 303 to 307 for each p frames (NO in step 308). When the processing has been completed for all n frames (YES in step 308), processor 202 executes the same light spot determination (step 309) described below, and then counts each blinking light spot determined to be the same individual in the series of blinking light spot superimposed images Fs as one, thereby determining the total number of blinking light spots captured during n frames (N seconds) (step 310).

[0026] <Identical light spot determination> 4, processor 202 sequentially reads out from memory 207 the blinking light spot superimposed image Fs(i) registered in step 307 and the blinking light spot superimposed image Fs(i-1) previously registered (step 401). Note that i = 2,...m, and when i = 1, the same Fs(i) is used as the previous blinking light spot image Fs(i-1). By referencing the coordinates of the blinking light spots stored in memory 207, processor 202 determines whether there are any new blinking light spots included in the blinking light spot superimposed image Fs(i) that are not included in the previous blinking light spot superimposed image Fs(i-1) (step 402). If there are no new blinking light spots included (NO in step 402), the process returns to step 401, where the process of sequentially reading out from memory 207 the next registered blinking light spot superimposed image Fs(i+1) and the previously registered blinking light spot superimposed image Fs(i) is repeated.

[0027] If a new light spot is present in the blinking light spot superimposed image Fs(i) (YES in step 402), the processor 202 measures the distance Δ between the coordinates of the new light spot and the closest previous light spot in terms of the number of pixels in the image, and determines whether the distance Δ is equal to or less than a predetermined threshold Δth (step 403). The predetermined threshold Δth is, for example, 100 pixels. If the new light spot appears at a position within the predetermined threshold Δth from the previous nearby light spot (YES in step 403), the processor 202 determines that the same blinking light spot (the same individual) has moved from its previous position, and registers the two nearby light spots as the same individual in the memory 207 (step 404). As described above, steps 401 to 404 are repeated until the same light spot determination is performed for all flashing light spot superimposed images Fs (NO in step 405), and when all are completed (YES in step 405), processor 202 returns to step 310 in Figure 3 described above, where multiple flashing light spots determined to be the movement of the same individual are counted as one, and the total number of flashing light spots captured during n frames (N seconds) is determined.

[0028] The above process (steps 301 to 310) is performed every N seconds, and the valid number of blinking light points can be determined by aggregating or statistically processing the results. That is, the number of blinking light points appearing in the video is estimated by performing the above-mentioned blinking light point determination and identical light point determination on n frames of video corresponding to N seconds. For example, when observing the number of fireflies, if N=30 seconds, it is possible to obtain counting results similar to those obtained by a survey in which a person observes a field for 30 seconds and counts the number of fireflies.

[0029] As shown in FIG. 5, among the light spots G1 to G5 captured in the light spot image Fh, light spot G3 is determined to be non-blinking noise by the blinking light spot determination and is excluded from the count. Among the light spots G1, G2, G4, and G5 determined to be blinking light spots, light spot G5 is a new light spot that appeared in the blinking light spot superimposed image Fs(i). Therefore, the distance Δ between light spot G4 and the previous blinking light spot superimposed image Fs(i-1)n is measured in pixels and compared with a predetermined threshold Δth to determine whether the blinking light spots G4 and G5 are the same individual. For example, if the distance Δ between the blinking light spots G4 and G5 is Δth = 100 pixels or less, it can be determined that the same individual has moved and appeared as separate images. If they are determined to be the same individual, these two light spots G4 and G5 are counted as one. Similarly, if the distance Δ between light spot G5 and light spot G6 (not shown) is 100 pixels or less, these two light spots are also counted as one. In this way, the light points G4, G5, and G6 are judged to be a single flashing light point that has moved, and even if there are three light points, they are counted as one.

[0030] As shown in Fig. 6, the display unit 206 of the flashing light spot counting device 200 displays the positions of the flashing light spots detected from the captured video based on the result of the same light spot determination. According to the example of Fig. 5, the flashing light spots G1 and G2 are displayed as separate, independent light spots, while the flashing light spots G4, G5, and G6 are determined to be a single moving light spot, and are therefore displayed as connected by a straight line, for example. This allows the user to see the movement trajectory of the single flashing light spot from G4 to G6. [Explanation of symbols]

[0031] 101 Imaging unit 200 Flashing light point counter 201 Interface 202 processors 203 Video Memory 204 program memory 205 Input section 206 Display section 207 Memory

Claims

1. A device for counting flashing light points from moving image data consisting of frame images, a first storage unit for storing moving image data consisting of a predetermined first number of frames; a second storage unit for storing information about the flashing light points; a processor for counting flashing light points based on the information about the flashing light points; wherein the processor comprises: For each of a plurality of light spots obtained by sequentially superimposing the moving images for each second number of frames among the first number of frames, it is determined whether the light spot is a blinking light spot, and position information of the light spot determined to be a blinking light spot is stored in the second storage unit; based on the position information of the first blinking light point obtained for each second number of frames and the position information of the second blinking light point obtained previously, if the first and second blinking light points are within a predetermined distance, register them as the same blinking light point in the second storage unit; Based on the registered information in the second storage unit, the same flashing light points are counted as one flashing light point, and the flashing light points that are apart from the predetermined distance are counted as separate flashing light points. A flashing light point counting device characterized by:

2. The flashing light spot counting device described in claim 1, characterized in that for each of the multiple light spots obtained by sequentially superimposing moving images for each second number of frames, it is determined whether the light spot is a flashing light spot based on the number of frames that include the light spot and the number of frames that do not include the light spot.

3. Further comprising a display unit, A flashing light point counting device as described in claim 1 or 2, characterized in that if the first and second flashing light points are determined to be the same flashing light point, the first and second flashing light points are linked and displayed on the display unit, and if they are determined to be separate flashing light points, they are displayed on the display unit as separate light points.

4. 4. The flashing light spot counting device according to claim 1, wherein the flashing light spots are the blinking spots of luminous fireflies.

5. A method for counting flashing light points from moving image data consisting of frame images by a processor, comprising: storing moving image data consisting of a predetermined first number of frames in a first storage unit; sequentially superimposing a moving image for each second number of frames among the first number of frames to extract a plurality of light points; For each of the plurality of light spots, it is determined whether the light spot is a flashing light spot, and position information of the light spot determined to be a flashing light spot is stored in a second storage unit; based on the position information of the first blinking light point obtained for each second number of frames and the position information of the second blinking light point obtained previously, if the first and second blinking light points are within a predetermined distance, register them as the same blinking light point in the second storage unit; Based on the registered information in the second storage unit, the same flashing light points are counted as one flashing light point, and the flashing light points that are apart from the predetermined distance are counted as separate flashing light points. A method for counting flashing light points.

6. The flashing light spot counting method according to claim 5, characterized in that for each of a plurality of light spots obtained by sequentially superimposing moving images for each of the second number of frames, it is determined whether the light spot is a flashing light spot based on the number of frames that include the light spot and the number of frames that do not include the light spot.

7. Counting the number of flashing light points a plurality of times for each of the first number of frames; 7. The method for counting flashing light points according to claim 5, wherein the number of flashing light points obtained by the multiple counting operations is statistically processed to determine the number of flashing light points.

8. 8. The method for counting flashing light points according to claim 5, wherein the flashing light points are the blinking points of a luminous firefly.

9. A program that causes a computer to function as a device that counts flashing light points from moving image data consisting of frame images, a function of storing moving image data consisting of a predetermined first number of frames in a first storage unit; a function of sequentially superimposing a moving image for each second number of frames among the first number of frames to extract a plurality of light points; a function of determining whether each of the plurality of light spots is a flashing light spot; a function of storing position information of the light point determined to be a blinking light point in a second storage unit; a function of determining that the first and second flashing light points are the same flashing light point if they are within a predetermined distance based on the position information of the first flashing light point obtained for each second number of frames and the position information of the second flashing light point obtained before that; a function of registering the first and second flashing light points as the same flashing light point in the second storage unit if the first and second flashing light points are within a predetermined distance from each other; a function of counting the same flashing light points as one flashing light point and counting the flashing light points that are apart from the predetermined distance as separate flashing light points based on the registered information of the second storage unit; A program characterized by realizing the above on the computer.

Images