Fish counting system, method of playing back captured video in fish counting system, and program

Abstract

To enable improvement in reliability of counting results while reducing playback time of captured videos, and to enable reduction of burden on operators in checking counting results.SOLUTION: A fish counting system is provided, comprising a counting unit for counting the number of fish based on a captured video of a fluid containing fish, a display unit for playing back the captured video, and an adjustment unit for adjusting the playback speed of the captured video. The adjustment unit changes playback speed of the captured video according to the number of fish counted by the counting unit when the display unit plays back the captured video.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a fish counting system, a method for playing back video captured in the fish counting system, and a program. [Background technology]

[0002] For example, Patent Document 1 discloses a technique for counting the number of passing fish using a plurality of cameras. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-202674 Summary of the Invention [Problem to be solved by the invention]

[0004] After counting fish, a user may want to play back the video on the screen to confirm the counting results. In this case, if the video is played back at the same speed (frame rate) as when it was shot, the time required to play back the entire video will be equal to or longer than when it was shot. This is because the user may need to stop the video midway through playback to check whether the counting was performed properly. In other words, in this case, the playback time of the entire video will be longer. On the other hand, if the playback speed of the video is uniformly increased regardless of the number of counted fish in order to shorten the playback time of the entire video, for example, when there are many fish on the playback screen, the operator may miss fish that should be checked, making it difficult to correct the counting results as needed. This leads to a decrease in the reliability of the counting results.

[0005] Furthermore, if a manual instruction is required from the worker to trigger the stopping of the video playback for confirmation, the worker will be required to input the stop instruction in addition to checking the counting results, which increases the burden on the worker.

[0006] The present invention has been made to solve the above-mentioned problems, and its purpose is to provide a fish counting system, a method for playing back filmed video in a fish counting system, and a program that can improve the reliability of counting results while shortening the playback time of the entire filmed video and reduce the burden on workers who check the counting results. [Means for solving the problem]

[0007] A fish counting system according to one aspect of the present invention is a fish counting system comprising a counting unit that counts the number of fish based on a filmed video of a fluid containing fish, and further comprising a display unit that plays back the filmed video, and an adjustment unit that adjusts the playback speed of the filmed video, and when the display unit plays back the filmed video, the adjustment unit changes the playback speed of the filmed video in accordance with the number of fish counted by the counting unit.

[0008] Another aspect of the present invention relates to a method for playing back filmed videos in a fish counting system, which is a method for playing back filmed videos in a fish counting system that has a counting unit that counts the number of fish based on filmed videos of a fluid containing fish, and includes adjusting the playback speed when displaying and playing back the filmed videos on a display unit, and playing back the filmed videos at the adjusted playback speed, where adjusting the playback speed includes changing the playback speed of the filmed videos in accordance with the number of fish counted by the counting unit.

[0009] A program according to yet another aspect of the present invention is a program for causing a computer to execute the above-described playback method. [Effects of the Invention]

[0010] While shortening the playback time of the entire captured video, it is possible to improve the reliability of the counting results and reduce the burden on the worker who checks the counting results. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is an explanatory diagram showing the general configuration of a fish counting system according to one embodiment of the present invention; [Figure 2] 3 is an explanatory diagram illustrating a schematic view of the imaging area of ​​the imaging unit of the fish counting system. FIG. [Figure 3] FIG. 10 is an explanatory diagram showing an example of setting a playback waiting time according to the number of counted fish. [Figure 4] 10 is a graph showing the relationship between the number of fish and the playback waiting time. [Figure 5] FIG. 10 is an explanatory diagram showing an example of a screen for confirming counting results. [Figure 6] 10 is a flowchart showing a procedure for modified playback. [Figure 7] FIG. 10 is an explanatory diagram schematically showing the playback time when a captured video in which the number of fish tails is zero in all frames is played back for normal playback and modified playback. [Figure 8] FIG. 10 is an explanatory diagram showing a schematic representation of the playback time when a captured video in which two of the frames have one fish tail is played back in normal playback and modified playback. [Figure 9] This is an explanatory diagram that shows a schematic representation of the playback time when playing back a filmed video in which the number of fish tails is one or more in any frame among multiple frames and the number of fish tails is zero in other frames, for both normal playback and modified playback. [Figure 10] 10 is a flowchart showing a procedure for modified playback when the manual restart mode is set. [Figure 11] FIG. 10 is an explanatory diagram showing another example of the counting result confirmation screen. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.

[0013] [1. Fish counting system configuration] 1 is an explanatory diagram showing the general configuration of a fish counting system 10 according to this embodiment. The fish counting system 10 includes an imaging unit 1, an input unit 2, a display device 3, and a processing device 4.

[0014] (1-1. Imaging unit) The imaging unit 1 is composed of a camera (video camera) capable of capturing video at, for example, 30 fps (frames per second). The imaging unit 1 captures the flow path of fish F, for example, in a river, the ocean, or a fish farm. For example, as shown in FIG. 1, when a first cage and a second cage are directly connected and fish F, such as tuna, are moved from the first cage to the second cage, the imaging unit 1 is installed near the entrance of the second cage to capture the flow path of fish F. Data of the video (captured video) captured by the imaging unit 1 is sent to the processing device 4. Note that the installation location of the imaging unit 1 is not limited to near the entrance of the second cage. For example, the imaging unit 1 may be installed near the exit of the first cage.

[0015] (1-2. Input section) The input unit 2 is configured with input devices such as a mouse, keyboard, touchpad, or touch panel, and is used by a user (e.g., a fish farmer) to give various instructions to the processing device 4. In other words, the input unit 2 accepts various instructions from the user. When the input unit 2 is configured with a touch panel, the input unit 2 is disposed so as to overlap the display screen of the display device 3.

[0016] (1-3.Display device) The display device 3 includes, for example, a display unit 31 and a display control unit 32. The display unit 31 is configured, for example, with a liquid crystal panel, and displays information provided by the processing device 4. For example, the display unit 31 plays back and displays a captured video that is read from a memory unit 42 (described later) of the processing device 4 and output to the display device 3. In other words, the fish counting system 10 includes a display unit 31 that plays back a captured video.

[0017] The display control unit 32 is configured by, for example, a central processing unit called a CPU (Central Processing Unit), and controls the display of various information on the display unit 31. The display unit 31 also displays information input by the input unit 2 (for example, text information input via a keyboard), a mouse pointer, and the like.

[0018] The display device 3 may be configured integrally with the processing device 4. In this case, the function of the display control unit 32 may be provided to a processing control unit 44 of the processing device 4, which will be described later.

[0019] (1-4. Processing equipment) The processing device 4 performs various processes, such as counting the number of fish in the fluid, based on the video captured by the imaging unit 1. Here, the fluid refers to a liquid containing water as a main component, such as seawater or freshwater. The processing device 4 is configured by a computer or a server (including a cloud server). The processing device 4 includes a counting unit 41, a memory unit 42, an information output unit 43, a processing control unit 44, and an adjustment unit 45.

[0020] The counting unit 41 is configured with a computing device or processor specialized for real-time image processing, such as a GPU (Graphics Processing Unit). The counting unit 41 has a neural network that has been trained in advance by deep learning or the like. This allows the counting unit 41 to input video data captured by the imaging unit 1, recognize (analyze) specific types of fish F through image recognition processing, count the number of recognized fish F (number of fish), and output the counting results. In other words, the fish counting system 10 includes a counting unit 41 that counts the number of fish F based on video data of a fluid containing fish F.

[0021] 2 is an explanatory diagram schematically showing the image capturing area of ​​the imaging unit 1. Here, a part of the edge of the rectangular image captured by the imaging unit 1 is defined as a first area Ar1, and the other edge is defined as a second area Ar2. The first area Ar1 is defined as the side closer to the first fish cage. In this example, when a fish F that was in the first area Ar1 moves to the second area Ar2, the counting unit 41 counts (adds) the fish F.

[0022] Specifically, the counting unit 41 extracts fish F from each of a plurality of images (e.g., images of each frame) included in the time series of the video by image recognition processing. Then, the counting unit 41 assigns an individual identification label to the extracted fish F to identify the individual fish F based on the position of the fish F in past images. Thereafter, when a fish F with the same label as the label assigned to the fish F in the first area Ar1 reaches the second area Ar2, the counting unit 41 counts the fish F in the image at that time. Note that this processing is one example, and various algorithms can be adopted as long as they can count the number of fish F.

[0023] The photographing area in FIG. 2 includes a third area Ar3 between the first area Ar1 and the second area Ar2. The third area Ar3 is a rectangular area surrounded by the first area Ar1 and the second area Ar2. By positioning the third area Ar3 in this manner, a fish F that leaves the first area Ar1 does not immediately reach the second area Ar2, but is photographed at the position of the third area Ar3 in multiple images. Therefore, the counting unit 41 can also recognize fish F in the third area Ar3. Furthermore, by recognizing the same fish F in the first area Ar1, the third area Ar3, and the second area Ar2, it is possible to accurately track the fish F that arrives from the first area Ar1 to the second area Ar2, thereby improving the accuracy of fish counting. Of course, the third area Ar3 can also be omitted.

[0024] 1 is a memory that stores various data in addition to the operating program of the processing device 4, and is configured to include, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk, an SSD (Solid State Drive), etc. For example, information such as data of the captured video acquired by the imaging unit 1 and the analysis results (counting results) of the counting unit 41 is stored in the storage unit 42.

[0025] The information output unit 43 outputs information such as the analysis results (counting results) of the counting unit 41 to the outside (for example, the display device 3 or an external terminal), and also outputs a signal to instruct the imaging unit 1 to change the imaging conditions. Such information output unit 43 is configured to have a communication interface for communicating with the outside via wired or wireless communication.

[0026] The processing control unit 44 is composed of, for example, a CPU, and controls the operation of each unit of the processing device 4. For example, when the input unit 2 instructs the processing control unit 44 to record the captured video acquired by the imaging unit 1, the processing control unit 44 performs a process of recording the captured video (storing it in the memory unit 42). Furthermore, when an instruction to correct the number of fish is input by the input unit 2, the processing control unit 44 also functions as a correction unit that corrects the fish count result obtained by the analysis in the counting unit 41 based on the correction instruction. Furthermore, the processing control unit 44 also has the function of a timer (timekeeping unit) that measures time.

[0027] The adjustment unit 45 is configured by, for example, a CPU. The adjustment unit 45 adjusts the playback speed of the captured video when the display unit 31 plays back and displays the captured video based on instructions received by the input unit 2. That is, the fish counting system 10 includes an adjustment unit 45 that adjusts the playback speed of the captured video. The function of the adjustment unit 45 may be possessed by the above-mentioned processing control unit 44 or the display control unit 32 of the display device 3. A method for adjusting the playback speed of the captured video by the adjustment unit 45 will be described below.

[0028] [2. How to adjust the playback speed of recorded videos] (2-1. Overview of adjustment method and playback wait time setting) First, the playback speed, reference speed, and reference time of a captured video are defined as follows. When a captured video containing multiple frames is played back, the number of frames played back per second is referred to as the playback speed. The playback speed is also referred to as the frame rate. For example, when a captured video is played back (displayed) at 30 frames per second, the playback speed is expressed as 30 fps. In this case, the playback time per frame is 1 second / 30 frames ≒ 0.033 seconds (= 33 msec). In this embodiment, playing back a captured video at a constant playback speed of 30 fps is referred to as "normal playback." The playback speed (30 fps) in normal playback is referred to as the "reference speed." The playback time at the reference speed (33 msec) is referred to as the "reference time."

[0029] In this embodiment, when the display unit 31 plays back the captured video, the adjustment unit 45 changes the playback speed of the captured video in accordance with the number of fish counted by the counting unit 41. Specifically, the adjustment unit 45 changes the playback speed by adjusting the playback wait time, which indicates the playback stop time for each frame included in the captured video, in accordance with the number of counted fish. Playing back the captured video at a changed playback speed in this way is called "altered playback," as opposed to the normal playback described above.

[0030] Here, the playback waiting time is set based on the following parameters. The standard waiting time that serves as the reference for the playback stop time of each frame included in the captured video is defined as T0 (msec). The standard waiting time T0 is set to, for example, 3 msec. Therefore, the standard waiting time T0 is 1 / 10 or less of the reference time (33 msec) mentioned above, and is sufficiently shorter than the reference time. The standard waiting time T0 of 3 msec is the minimum waiting time that can be realized by the system of this embodiment. In other words, the standard waiting time T0 of 3 msec corresponds to the maximum playback speed that can be realized by the system of this embodiment, that is, the maximum speed.

[0031] The number of fish counted in each frame by the counting unit 41 is defined as N fish, where N is an integer equal to or greater than 0. The set waiting time set as the playback stop time for each fish is defined as Ts (msec). The set waiting time Ts is set to, for example, 66 msec. Note that the above numerical example is merely an example and can be changed as appropriate.

[0032] If the playback wait time corresponding to the number of fish in each frame is TA (msec), the playback wait time TA is set based on the standard wait time T0, the set wait time Ts, and the number of counted fish N. Specifically, the playback wait time TA is set as TA = T0 + Ts × N. For example, for a frame in which no fish are counted, N = 0, so TA = T0 = 3 msec. For a frame in which only one fish is counted, TA = T0 + Ts × 1 = 3 + 66 × 1 = 69 msec. For a frame in which three fish are counted, TA = T0 + Ts × 3 = 3 + 66 × 3 = 201 msec. In this way, a playback wait time TA corresponding to the number of counted fish is set for each frame.

[0033] FIG. 3 shows an example of setting the playback wait time according to the number of counted fish (number of fish in an area). From this figure, it can be seen that the more fish counted in an area, the longer the playback wait time. The relationship between the number of counted fish and playback wait time is shown in a graph in FIG. 4. Note that the above area refers to the counting area. The counting area refers to, for example, the second area Ar2 shown in FIG. 2, but it may be at least one of the first area Ar1 to the third area Ar3.

[0034] An upper limit is set for the playback wait time TA. This is to prevent the playback time in modified playback from easily exceeding the playback time in normal playback due to an excessive increase in the playback wait time TA. The upper limit, the maximum wait time TAmax, is set to, for example, 3000 msec. For example, if the number of counted fish is 46, the playback wait time TA is TA = T0 + Ts × 46 = 3 + 66 × 46 = 3039 msec, which exceeds the maximum wait time TAmax (3000 msec). Therefore, once the number of counted fish exceeds 46, the playback wait time TA is fixed to the maximum wait time TAmax. The maximum wait time TAmax is not limited to 3000 msec and can be changed as appropriate.

[0035] (2-2. Screen for checking counting results) When the input unit 2 receives an instruction from the user to display a screen for confirming the counting results, the processing control unit 44 of the processing device 4 causes the display unit 31 to display the screen shown in Fig. 5. In this case, the above-mentioned user includes an operator who will be checking the counting results. The operator may be the same person as the aquaculture farmer, or may be a different person (for example, a manager entrusted with management by the aquaculture farmer).

[0036] The display screen of FIG. 5 has a count result display area R1, an image display area R2, a correction area R3, and a fish number change display area R4.

[0037] The counting result display area R1 displays the number of fish counted by the counting unit 41 of the processing device 4. In the example of Fig. 5, the counting result (cumulative number) up to the current point in time of the captured video being played back and displayed in the video display area R2 (corresponding to the current point P in the fish number change display area R4, described later) is 2024 fish.

[0038] The video display area R2 displays the video (filmed video) of the recording file that was the subject of the fish count as described above. This allows the user to input a correction value for the number of fish in the correction area R3 while viewing the video displayed in the video display area R2. The video display area R2 also displays icons IC for playing the video (multiple images), advancing the screen frame by frame, rewinding the screen frame by frame, etc. Note that these icons IC merely indicate that manual playback instructions are also possible in the fish counting system 10 of this embodiment, which performs automatic playback. In other words, in the fish counting system 10 of this embodiment, which automatically adjusts the playback speed, the display of the icons IC is merely an auxiliary display, and the icons IC may not be displayed.

[0039] The correction area R3 is an area where the user inputs a correction value for the number of fish using the input unit 2. An addition key K1 and a subtraction key K2 are displayed in the correction area R3. When the user clicks (or presses) the addition key K1 once using the input unit 2, the correction value for the number of fish increases by one. Conversely, when the user clicks (or presses) the subtraction key K2 once using the input unit 2, the correction value for the number of fish decreases by one. If the input unit 2 includes a keyboard, the correction value may be input directly by operating the keyboard (e.g., a numeric keypad). The example in FIG. 5 indicates that the correction value for the image of the current frame is 0 fish, and that the correction value for the entire recorded video is +4 fish. The information on the number of fish corrected in the correction area R3 is output from the display device 3 to the processing device 4 and stored in the memory unit 42 of the processing device 4 in association with the captured video (recorded file).

[0040] The fish number change display area R4 displays a graph G1 that shows the change in the number of counted fish over time. The vertical axis Ax1 of the graph G1 corresponds to the number of fish counted per unit time, and the horizontal axis Ax2 corresponds to the elapsed time. If the video recording time is long and does not fit on one screen, a slider bar B1 is displayed in the fish number change display area R4. By sliding the slider bar B1, the count results for any point in time can be displayed. The fish number change display area R4 also displays the current time P, which indicates the time of the image (frame) currently being displayed.

[0041] (2-3. Procedure for changing playback) As described above, when the input unit 2 receives an instruction from the user to display a screen for confirming the counting results, the screen shown in Fig. 5 is displayed on the display unit 31, and the following modified playback process is started. Fig. 6 is a flowchart showing the procedure for modified playback in this embodiment.

[0042] First, the processing control unit 44 reads the image data of the first frame from the captured video from the storage unit 42 (S1). The processing control unit 44 also reads the fish count result for the first frame stored in the storage unit 42 (S2).

[0043] Next, the adjustment unit 45 determines whether or not the fish to be counted are present in the counting area based on the counting result read in S2 in the frame image read in S1 (S3). For example, if the counting result is zero, the adjustment unit 45 determines that the fish to be counted are not present in the counting area. On the other hand, if the counting result is 1 or more, the adjustment unit 45 determines that the fish to be counted are present in the counting area.

[0044] If it is determined in S3 that no fish exist in the counting area (zero fish have been counted), the adjustment unit 45 sets the standard waiting time T0 (e.g., 3 msec) for the first frame as the playback waiting time TA based on the above calculation (S4). Because the standard waiting time T0 is shorter than the reference time (e.g., 33 msec), the playback speed when playing back and displaying the image of the first frame is faster than the reference speed (e.g., 30 fps). Also, as mentioned above, the playback speed is the maximum speed that can be realized by the system.

[0045] On the other hand, if it is determined in S3 that fish are present in the counting area (the number of counted fish is one or more), the adjustment unit 45 sets the playback waiting time TA to the standard waiting time T0 + the set waiting time Ts × the number of fish N based on the above calculation (S5). For example, if the number of counted fish is one, the playback waiting time TA will be 69 msec, as described above, which is longer than the reference time (e.g., 33 msec). Therefore, the playback speed when the image of the first frame is played back and displayed will be slower than the reference speed (e.g., 30 fps).

[0046] Next, the adjustment unit 45 determines whether the playback waiting time TA is equal to or greater than the maximum waiting time TAmax (S6). If the playback waiting time TA is equal to or greater than the maximum waiting time TAmax, the playback waiting time TA is fixed to the maximum waiting time TAmax (S7). If the playback waiting time TA is less than the maximum waiting time TAmax, the process proceeds directly to S8.

[0047] Next, the adjustment unit 45 causes the display unit 31 to display the first frame image (S8). That is, the first frame image is played back. At this time, the display time of the first frame image is, for example, 1 msec. Thereafter, the adjustment unit 45 subtracts 1 msec from the playback wait time TA (S9), and if the playback wait time TA remains (if TA>0 msec), the operations from S8 onwards are repeated. That is, the first frame image is displayed again for 1 msec. By repeating the processes from S8 to S10, the first frame image is displayed for the playback wait time TA.

[0048] When the playback wait time TA for the image of the first frame expires in S10, the adjustment unit 45 accesses the storage unit 42 to determine whether or not a second frame exists in the captured video (S11). If a second frame exists, the process returns to S1, reads the image data of the second frame from the captured video, and repeats the processes from S2 onwards. If a next frame does not exist in S11, that is, if the same processes as above have been performed up to the final frame of the captured video, the modified playback process ends.

[0049] (2-4. Effects of modified playback) Figure 7 shows a schematic diagram of the playback time for normal playback and modified playback when playing back a video in which there are zero fish tails in any frame. Note that the number of frames played back is the same for normal playback and modified playback. In normal playback, the playback speed of each frame is a constant 30 fps, so the total playback time for multiple frames, X1 (msec), is 33 msec x A, where A is the number of frames.

[0050] In contrast, with modified playback, the playback speed is faster than normal playback for all frames. As a result, the total playback time of multiple frames, X2 (msec), is definitely shorter than the playback time X1 in normal playback. In other words, the playback time of the entire recorded video is shortened by (X1 - X2).

[0051] Figure 8 shows a schematic diagram of the playback time for normal playback and modified playback when playing back a video in which two of the frames contain one fish tail. Note that the number of frames played back is the same for normal playback and modified playback. The playback time X1 for normal playback is the same as in Figure 7 (when the number of fish tails is zero).

[0052] In contrast, in the case of modified playback, the playback speed is slower than normal playback for two frames with a mantissa of 1, but the playback speed is faster than normal playback for other frames. Therefore, the total playback time X3 (msec) for multiple frames is longer than the playback time X2 in FIG. 7, but shorter than the playback time X1 in normal playback. That is, the playback time of the entire captured video is shortened by (X1 - X3).

[0053] FIG. 9 schematically shows the playback time when playing back a captured video in which the mantissa of the fish is 1 or more in any frame and the mantissa of the fish is zero in other frames among multiple frames, for both normal playback and modified playback. It is assumed that the number of frames to be played back is the same for normal playback and modified playback. The playback time X (msec) in normal playback is 33 msec × A, where A is the number of frames.

[0054] In contrast, in the case of modified playback, the playback speed is slower than normal playback for frames with a mantissa of 1 or more, but the playback speed is faster than normal playback for other frames (frames with a mantissa of zero). Therefore, the total playback time Y (msec) for multiple frames is shorter than the playback time X in normal playback. That is, the playback time of the entire captured video is shortened by (X - Y).

[0055] For example, if the number of frames with a large mantissa and a playback waiting time TA reaching the maximum waiting time TAmax increases, it is assumed that it will be difficult to achieve Y < X. In this case, by changing the set values of each parameter, such as setting the waiting time Ts shorter than 66 msec or setting the maximum waiting time TAmax shorter than 3000 msec, it becomes possible to achieve Y < X.

[0056] As described above, in the fish counting system 10 of this embodiment, the adjustment unit 45 changes the playback speed of the captured video in accordance with the number of fish counted by the counting unit 41. Furthermore, the method for playing back the captured video in this embodiment includes adjusting the playback speed when the captured video is displayed and played back on the display unit 31, as shown in S4 and S5 of FIG. 6 , and playing back the captured video at the adjusted playback speed, as shown in S8 to S10. Adjusting the playback speed includes changing the playback speed of the captured video in accordance with the number of fish counted by the counting unit 41.

[0057] This allows the playback speed to be increased when the number of counted fish is small (e.g., when the number of fish is zero), thereby shortening the overall playback time of the captured video (see FIGS. 7 to 9). Furthermore, while shortening the overall playback time of the captured video, the playback speed can be decreased when the number of counted fish is large, making it easier for the user (operator) viewing the played-back captured video to visually confirm the count results (see FIGS. 8 and 9). Furthermore, by reducing the playback speed, it is possible to ensure time to correct the count results in the correction region R3 of FIG. 5, if necessary. Correcting the count results improves the reliability of the count results. Furthermore, because the adjustment unit 45 automatically changes the playback speed depending on the number of counted fish, the burden on the operator is reduced compared to when the operator manually changes the playback speed (e.g., stops playback) to check the count.

[0058] In other words, according to the fish counting system 10 and playback speed adjustment method of this embodiment, it is possible to shorten the playback time of the entire captured video while improving the reliability of the counting results and reducing the burden on the worker who checks the counting results.

[0059] When the number of fish counted by counting unit 41 is zero, adjustment unit 45 increases the playback speed above the standard speed (S3, S4). When the number of counted fish is zero, there is no need to check the number of fish even when playing back the captured video. In this case, by increasing the playback speed of the captured video above the standard speed, the overall playback time is reliably shortened compared to playing back the entire captured video at the standard speed (see FIGS. 7 to 9).

[0060] In particular, when the number of fish counted by the counting unit 41 is zero, the adjustment unit 45 increases the playback speed to the maximum speed (S4), thereby maximizing the reduction in the playback time of the entire captured video.

[0061] When the number of fish counted by the counting unit 41 is greater than zero, the adjustment unit 45 slows the playback speed below the standard speed (S3, S5). In this case, more time is ensured for the operator to visually check the captured video to confirm the fish. This makes it easier for the operator to confirm the fish and reduces confirmation errors (overlooking the confirmation of counted fish). It also makes it easier to find omissions in the count and to correct the count to improve the reliability of the counting results.

[0062] In particular, when the number of fish counted by the counting unit 41 is greater than zero, the adjustment unit 45 reduces the playback speed and stops playback of the captured images for a predetermined time (playback waiting time TA) (S5 to S10). In this case, while playback of the captured images is stopped, the operator can easily check all fish.

[0063] In order to facilitate the change of the playback speed, as in this embodiment, it is desirable that the adjustment unit 45 changes the playback speed by adjusting the playback waiting time TA, which indicates the playback stop time of each frame included in the captured video, according to the number of fish (S4, S5).

[0064] (2-5. Setting and changing the playback mode) The input unit 2 shown in FIG. 2 may accept a designation of a playback mode for a captured video and an instruction to resume playback of the captured video after the playback has stopped. Here, the above playback modes include an automatic restart mode and a manual restart mode. As shown in S10 and S11 of FIG. 6, the automatic restart mode is a playback mode in which, if a next frame is available after a playback wait time TA for a predetermined number of frames has elapsed, playback of the next frame is automatically transitioned to (even without a restart instruction being input via the input unit 2). In other words, the playback mode described in this embodiment corresponds to the automatic restart mode. In the automatic restart mode, the operator does not need to operate the input unit 2 to input a restart instruction, which further reduces the burden on the operator.

[0065] On the other hand, the manual resume mode is a playback mode in which, after a playback wait time TA for a predetermined frame has elapsed, playback of the next frame is transitioned to based on a resume instruction received by the input unit 2. FIG. 10 is a flowchart showing the procedure for modified playback when the manual resume mode is set. The procedure in FIG. 10 is the same as that in FIG. 6, except that S10-1 is added between S10 and S11 in the flowchart in FIG. 6. In S10-1, the adjustment unit 45 determines whether a playback resume instruction has been received by the input unit 2. Playback remains stopped until a resume instruction is received, and once a resume instruction is received, the process proceeds to S11. In the manual resume mode, playback remains stopped until the operator inputs a resume instruction, allowing the operator to safely and reliably check the counting results.

[0066] If two types of regeneration modes are set as described above, the operator can selectively set either the automatic regeneration mode or the manual regeneration mode, which improves convenience. Note that the regeneration mode can be set in advance during initial setup.

[0067] The above-mentioned automatic restart mode is referred to as the first automatic restart mode. As a variant, a second automatic restart mode may be set instead of the manual restart mode. The second automatic restart mode is a mode in which playback of the next frame is automatically started after a predetermined time has elapsed since the playback wait time TA for a predetermined frame has elapsed. When the second automatic restart mode is set, a restart instruction from the input unit 2 is not required, and the playback stop time is effectively extended. Therefore, both the effect of reducing the burden on the operator and the effect of allowing the operator to check the counting results safely and reliably are obtained.

[0068] [3. Other examples of display screens] 11 is a schematic diagram showing another example of a display screen that is displayed on the display unit 31 when the input unit 2 receives an instruction from the user to display a screen for confirming the count results. The display screen in FIG. 11 has a video display area R2, a fish number change display area R4, a progress display area R10, and a disturbance display area R20. The manual correction result is displayed as the cumulative adjusted fish count (Total Count) in the video display area R2, overlapping with the video of fish F.

[0069] When the input unit 2 includes a mouse and correction values ​​are input using the mouse, the left click of the mouse can be assigned to "+1" and the right click of the mouse can be assigned to "-1". However, input using a mouse is only valid when the correction mode is the Normal mode, which will be described later. Furthermore, when the correction mode is the Normal mode, it is not possible to select each element displayed on the full screen with a mouse click, except for the correction mode selection button (area R102, which will be described later).

[0070] At the bottom of the video display area R2, instruction icons IC are displayed for playing the video (multiple images), pausing, advancing the screen frame by frame, rewinding the screen frame by frame, displaying a frame at a specified time, and changing the playback point at a specified time.

[0071] Here, if the input unit 2 includes a keyboard, the following functions can be assigned as shortcuts to predetermined keys on the keyboard. For example, the left arrow key of the keyboard is assigned a function to go back one frame, the right arrow key is assigned a function to go forward one frame, the up arrow key is assigned a function to add one tail to the adjustment number, and the down arrow key is assigned a function to subtract one tail from the adjustment number.

[0072] Furthermore, the following additional functions can be assigned to specific keys on the keyboard. For example, the function to play and stop recorded video can be assigned to the keyboard's space bar. The function to go back one frame in frame advance can be assigned to the keyboard's "A" key, and the function to go forward one frame in frame advance can be assigned to the keyboard's "D" key. The function to skip to the previous (past) automatic count position can be assigned to the keyboard's "Z" key, and the function to skip to the next automatic count position can be assigned to the keyboard's "C" key. The function of counting correction of +1 can be assigned to the keyboard's "W" key, and the function of counting correction of -1 can be assigned to the keyboard's "S" key. However, the above functions of the keyboard's "W" and "S" keys are only valid when the correction mode is Normal mode.

[0073] The fish count change display area R4 displays the current time P, as shown in FIG. 5 , as well as a correction time Q indicating the time when the correction value was input by the input unit 2 and a marking S. The marking S indicates the time when the processing control unit 44 of the processing device 4 detected a disturbance based on the image captured by the imaging unit 1. Here, a disturbance refers to an external factor that affects the fish analysis and fish counting by the counting unit 41. For example, one such disturbance is "reflection of direct sunlight." When direct sunlight is reflected in the image, the entire image becomes bright, making it difficult for the counting unit 41 to accurately recognize and count the fish F. In such a case, the processing control unit 44 outputs a control signal to the display device 3 to prompt the user to change the shooting direction of the imaging unit 1. The display device 3 can prompt the user to change the shooting direction by displaying a message instructing the user to change the shooting direction of the imaging unit 1 based on the control signal.

[0074] The processing control unit 44 can detect "reflection of direct sunlight" as a disturbance, for example, as follows: The processing control unit 44 divides one frame of image included in the video captured by the imaging unit 1 into n x n blocks (for example, n = 13). If there is a block where the average brightness is equal to or greater than a certain value, the average hue is equal to or less than a certain value, and the average values ​​of R (red), G (green), and B (blue) are equal to or greater than thresholds, the processing control unit 44 determines that there is reflection of direct sunlight (detects the presence of a disturbance). Information indicating the detected disturbance is displayed in the disturbance information display area R20.

[0075] The progress display area R10 displays, for example, the following information: The automatic count result, that is, the cumulative number of determined tails (cumulative count up to the current frame), is displayed as "FCS Count." The progress (playback status) of the captured video is displayed as "Progress." Here, "Progress" displays the current frame number / total frame number and the current frame playback time / total frame playback time in two rows.

[0076] Area R101 of the progress display area R10 displays a selectable display format for the automatically counted fish F in the video display area R2. There are three display format options: displaying a rectangle inscribed with the fish F (Rect), displaying the center point of the fish F (Center Point), and displaying nothing (None), and any of these three options can be selected using the input unit 2. For example, each time area R101 is clicked with the mouse, the selection can be switched between Rect, Center Point, and None in that order.

[0077] Area R102 of progress display area R10 displays selectable counting correction modes. There are two correction mode options: Normal and in video. These two correction modes can be selected by clicking the mouse in area R102. When Normal mode is selected, a correction value can be input by keyboard entry or mouse click. When in video mode is set, a correction value can be input by mouse click within the captured video.

[0078] Area R103 of progress display area R10 displays a selectable playback speed for playing back video without skipping frames. Playback speed options include 1x (x1), 2x (x2), and 4x (x4). Each time area R103 is clicked with the mouse, the speed can be switched between 1x (x1), 2x (x2), and 4x (x4).

[0079] [4. About the Program] The fish counting system 10 of this embodiment shown in FIG. 1 can be configured with a computer on which an operating program (application software) is installed. The computer (e.g., the processing control unit 44 as a CPU) reads and executes the program, thereby executing each process of the method for playing back captured video. Such a program may be obtained, for example, by downloading it from an external source via a network, or by reading the program from a computer-readable recording medium such as a CD-ROM (Compact Disk-Read Only Memory) on which the program is recorded.

[0080] [5. Supplementary Information] The playback wait time for each frame may be a moving average of the playback wait times for several frames before and after the specified frame. In this case, for example, the playback of a frame will stop gradually (slowly) instead of suddenly stopping, making it easier for the worker to watch the captured video. In other words, the worker will be less likely to notice sudden frame changes, and will be able to watch a captured video in which each frame changes smoothly.

[0081] The configuration may be such that a modified playback mode (time-saving mode) for executing the modified playback described in this embodiment and a normal playback mode for executing normal playback can be selectively set, and when the modified playback mode is set, the modified playback of this embodiment, i.e., the playback speed can be adjusted.

[0082] [6. Notes] The fish counting system, the method for playing back captured video, and the program described in this embodiment can also be expressed as described in the following appendix.

[0083] The fish counting system in Appendix (1) is A fish counting system including a counting unit that counts the number of fish based on a video of a fluid containing fish, a display unit for playing back the captured video; an adjustment unit that adjusts the playback speed of the captured video, When the display unit plays back the captured video, the adjustment unit changes the playback speed of the captured video in accordance with the number of fish counted by the counting unit.

[0084] The fish counting system of appendix (2) is the fish counting system described in appendix (1), The adjustment unit increases the playback speed above a reference speed when the number of fish counted by the counting unit is zero.

[0085] The fish counting system of appendix (3) is the fish counting system described in appendix (2), The adjustment unit increases the playback speed to a maximum speed when the number of fish counted by the counting unit is zero.

[0086] The fish counting system of appendix (4) is the fish counting system described in appendix (2) or (3), The adjustment unit reduces the playback speed below the reference speed when the number of fish counted by the counting unit is greater than zero.

[0087] The fish counting system of appendix (5) is the fish counting system described in appendix (4), When the number of fish counted by the counting unit is greater than zero, the adjustment unit stops playback of the captured images for a predetermined time.

[0088] The fish counting system of appendix (6) is a fish counting system according to any one of appendices (1) to (5), The adjustment unit changes the playback speed by adjusting a playback wait time, which indicates a playback stop time for each frame included in the captured video, in accordance with the number of fish.

[0089] The fish counting system of appendix (7) is the fish counting system described in appendix (6), an input unit that accepts a designation of a playback mode for the captured video and an instruction to resume playback of the captured video after it has been stopped; The playback mode is an automatic restart mode in which the playback of the next frame is automatically started after the playback waiting time for the predetermined frame has elapsed; and a manual restart mode in which, after the playback wait time for the predetermined frame has elapsed, playback of the next frame is started based on a restart instruction received by the input unit.

[0090] In the fish counting system of appendix (8), the method for playing back the captured video comprises: A method for playing back a video of a fish counting system including a counting unit that counts the number of fish based on the video of a fluid containing fish, the method comprising: adjusting a playback speed when the captured video is displayed on a display unit and played back; and playing back the captured video at the adjusted playback speed; Adjusting the playback speed includes changing the playback speed of the captured video in accordance with the number of fish counted by the counting unit.

[0091] The program in Appendix (9) is A program for causing a computer to execute the playback method described in appendix (8).

[0092] Although the embodiments of the present invention have been described above, the scope of the present invention is not limited to these, and the invention can be expanded or modified without departing from the spirit of the invention. [Industrial Applicability]

[0093] The present invention can be used in a fish counting system. [Explanation of symbols]

[0094] 2 Input section 10 Fish counting system 31 Display section 41 Counting Department 45 Adjustment part F-fish TA playback waiting time

Claims

1. A fish counting system including a counting unit that counts the number of fish based on a video of a fluid containing fish, a display unit for playing back the captured video; an adjustment unit that adjusts the playback speed of the captured video, A fish counting system in which the adjustment unit changes the playback speed of the captured video according to the number of fish counted by the counting unit when the display unit plays back the captured video.

2. The fish counting system according to claim 1 , wherein the adjustment unit increases the playback speed above a reference speed when the number of fish counted by the counting unit is zero.

3. The fish counting system according to claim 2 , wherein the adjustment unit increases the playback speed to a maximum speed when the number of fish counted by the counting unit is zero.

4. The fish counting system according to claim 2 , wherein the adjustment unit reduces the playback speed below the reference speed when the number of fish counted by the counting unit is greater than zero.

5. The fish counting system according to claim 4 , wherein the adjustment unit stops playback of the captured images for a predetermined time when the number of fish counted by the counting unit is greater than zero.

6. The fish counting system of any one of claims 1 to 5, wherein the adjustment unit changes the playback speed by adjusting a playback wait time indicating the playback stop time of each frame included in the captured video according to the number of fish.

7. an input unit that accepts a designation of a playback mode for the captured video and an instruction to resume playback of the captured video after it has been stopped; The playback mode is an automatic restart mode in which the playback of the next frame is automatically started after the playback waiting time for the predetermined frame has elapsed; The fish counting system of claim 6, further comprising a manual restart mode in which, after the playback wait time for the specified frame has elapsed, playback of the next frame is transitioned to based on a restart instruction received by the input unit.

8. A method for playing back a video of a fish counting system including a counting unit that counts the number of fish based on the video of a fluid containing fish, the method comprising: adjusting a playback speed when the captured video is displayed on a display unit and played back; and playing back the captured video at the adjusted playback speed; A method for playing back a filmed video in a fish counting system, wherein adjusting the playback speed includes changing the playback speed of the filmed video according to the number of fish counted by the counting unit.

9. A program for causing a computer to execute the reproducing method according to claim 8.

Images