A video data acquisition method and system for fish sorting
Through the video data frame image difference screening and corner point importance correction, the identification errors caused by motion blur in fish sorting are solved, and the sorting accuracy is improved.
Patent Information
- Application Number
- CN202411331414.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-24
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-09-24
AI Technical Summary
In traditional fish sorting methods, due to the movement of fish, the image frames are blurred due to the movement of fish, and the matching effect of the template matching algorithm is poor, resulting in fish species identification errors and reducing the sorting accuracy.
By obtaining the difference in frame images of video data, filtering the images to be matched, analyzing the stability and variation amplitude of the background cluster of corners, correcting the importance of corners, filtering out the preferred corners for template matching, and improving identification accuracy.
It improves the sorting accuracy of automatic fish sorting recognition, reduces fish species recognition errors, and enhances the stability of the sorting system.
Smart Images

Figure CN118864800B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of television system data communication, and in particular to a video data acquisition method and system for fish sorting. Background Art
[0002] Fish sorting is an important part of the dam water conservancy fish passage plan. The barrier of the dam fragments the river habitat and hinders the genetic exchange between fish populations upstream and downstream of the dam. The implementation of the fish passage plan can protect and restore the ecological connectivity of fish populations. The implementation of the fish passage plan mainly relies on the work of the fish lift system. The fish lift system is divided into seven steps: fish luring → fish gathering → lifting → transfer → sorting → transportation → release. The sorting process is an important process among them. It mainly obtains video frames of fish, obtains image frames from the video frames of fish, and then uses AI algorithms to identify the types of fish in the image frames. The fish sorting path is selected according to the identification results, so as to ensure that fish of the same type are on the same sorting path and avoid the struggle of different types of fish in the subsequent transportation process.
[0003] Traditional methods often use template matching algorithms to identify the species of fish in image frames. However, since fish may move, there will be motion blur in the image frames extracted from the fish. If the corner points used for template matching fall in the background area or the motion blur area, the matching effect of the template matching algorithm will be poor, which will lead to the misidentification of fish species, thereby reducing the sorting accuracy of automatic fish sorting. Summary of the invention
[0004] The invention provides a video data acquisition method and system for fish sorting to solve the existing problems.
[0005] The video data acquisition method and system for fish sorting of the present invention adopts the following technical solutions:
[0006] The present invention proposes a video data acquisition method for fish sorting, which comprises the following steps:
[0007] Obtain video data of the fish to be detected; decompose the video data into a plurality of frames of images of the fish to be detected;
[0008] Obtain all reference images of each frame of the fish image to be detected; obtain the degree of difference between adjacent frames of each frame of the fish image to be detected according to the grayscale difference between each frame of the fish image to be detected and the reference image; screen all frames of the fish image to be detected based on the degree of difference between adjacent frames to obtain images to be matched;
[0009] Obtain all corner points in the image to be matched; obtain the background cluster stability of each corner point by analyzing the changes in the clustering clusters to which each corner point belongs in the clustering results after multiple clustering of the image to be matched; obtain the corner point change amplitude of each corner point according to the distance difference between the image to be matched and the reference image; obtain the corner point importance of each corner point according to the background cluster stability of each corner point and the difference in the corner point change amplitude between each corner point and other corner points; correct the corner point importance according to the grayscale distribution of the pixels around each corner point to obtain the matching preference of each corner point;
[0010] All corner points are screened based on the matching preference degree to obtain matching corner points for fish sorting; template matching is performed through the matching corner points to obtain template matching results; and the fish to be detected are sorted according to the template matching results.
[0011] Preferably, the specific method for obtaining all reference images of each frame of the fish image to be detected is:
[0012] The first The adjacent frames of fish images to be detected before and after the frame of fish images to be detected are both recorded as The reference image of the fish image to be detected.
[0013] Preferably, the specific method for obtaining the degree of difference between adjacent frames of each frame of the fish image to be detected according to the grayscale difference between each frame of the fish image to be detected and the adjacent frames of the fish image to be detected is:
[0014] The first The adjacent frames of fish images to be detected before and after the frame of fish images to be detected are both recorded as A reference image of the fish image to be detected;
[0015] The first The first frame of the fish image to be detected Pixels and The first frame of the fish image to be detected The reference image The absolute value of the difference in grayscale values between pixels is recorded as The fish image to be detected is The reference image The grayscale difference value of the pixel points; The fish image to be detected is The mean of the grayscale difference values of all pixels between the reference images is recorded as The reference difference value of the reference image; The mean of the reference difference values of all reference images of the fish image to be detected in the frame is taken as the The difference degree between adjacent frames of the fish image to be detected.
[0016] Preferably, the specific method of screening all frames of fish images to be detected based on the difference between adjacent frame images to obtain the images to be matched is:
[0017] Among all the frames of fish images to be detected, the fish image to be detected with the smallest difference between adjacent frame images is used as the image to be matched.
[0018] Preferably, the specific method for obtaining the stability of the background cluster of each corner point by analyzing the change of the clustering cluster to which each corner point belongs in the clustering results after multiple clustering of the to-be-matched image is:
[0019] Perform Kmeans clustering with different K values on the matching images to obtain several clustering results;
[0020] The first The corner point is The number of all pixels in the cluster in the clustering result is recorded as The corner point is The background cluster area under the clustering result; The corner point is The background cluster area under the clustering result is The mean of the difference between the mean values of the background cluster areas of the corner points under all clustering results is recorded as The corner point is The change factor under the clustering result; The normalized value of the reciprocal of the mean of the variation factor of the corner point under all clustering results is taken as the The stability of the background cluster at each corner point.
[0021] Preferably, the specific method for obtaining the variation amplitude of each corner point according to the distance difference between each corner point in the image to be matched and the image of the fish to be detected in the adjacent frame is:
[0022] The first The corner point is The corresponding pixel point on the vertical projection of the reference image is recorded as The corner point is The target position on the reference image;
[0023] For Any corner point in the reference image is compared with the The corner point is The Euclidean distance between the target positions on the first reference image is used as the change distance of the corner point; Among all the corner points in the reference image, the corner point with the smallest change distance is taken as the first The corner point is target corner points on the reference image;
[0024] The first The average of the change distances of the target corner points on all reference images is taken as the The range of change of the corner points.
[0025] Preferably, the specific method for obtaining the importance of each corner point according to the stability of the background cluster of each corner point and the difference in the variation range of the corner point between each corner point and other corner points is:
[0026] For the images to be matched except the Any corner point other than the first corner point, The absolute value of the difference between the corner points' change amplitudes is recorded as the change difference value of the corner points; The mean of the change difference values of all corner points except the corner points is recorded as The overall change difference value of the corner points in the image to be matched;
[0027] In the Among the reference images, The corner point is Any corner point other than the target corner point on the reference image is compared with the The corner point is The absolute value of the difference between the corner point change amplitudes of the target corner points on the reference images is recorded as the target change difference value of the corner point; The corner point is The mean of the target change difference values of all corner points other than the target corner points on the reference image is recorded as The corner point is The overall change difference of the target in the reference image;
[0028] The first The average of the target overall change difference values of the corner points in all reference images is The absolute value of the difference between the overall change difference values of the corner points in the image to be matched is taken as the first The final change difference value of the corner points;
[0029] The first The final change difference between the corner points and The normalized value of the inverse of the product of the stability of the background clusters of the corner points is taken as the first The importance of each corner point.
[0030] Preferably, the specific method of correcting the importance of the corner points according to the grayscale distribution of the pixels around each corner point and obtaining the matching preference degree of each corner point is:
[0031] Preset a neighborhood parameter , with the The corner points are used as the window center, and the window size is obtained as and record this window as The neighborhood range of the corner points;
[0032] The first The mean of the grayscale values of all pixels in the neighborhood of the corner point is recorded as the first mean; The corner point in the neighborhood of The absolute value of the difference between the pixel point and the first mean is recorded as The first difference of the pixels; The reciprocal of the mean of the first differences of all pixels in the neighborhood of the corner point is recorded as the confidence factor;
[0033] The first The product of the importance of the corner point and the confidence factor is taken as the The matching preference of each corner point.
[0034] Preferably, the specific method of screening all corner points based on the matching preference degree to obtain matching corner points for fish sorting is:
[0035] All corner points in the image to be matched are sorted from large to small according to the matching preference degree to obtain a sorted angle sequence; the first 4 angles in the sorted angle sequence are recorded as matching corner points for fish sorting.
[0036] The present invention also proposes a video data acquisition system for fish sorting, comprising a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the steps of the above-mentioned video data acquisition method for fish sorting.
[0037] The beneficial effects of the technical solution of the present invention are as follows: the present invention obtains the corner point importance of each corner point according to the stability of the background cluster of each corner point and the difference in the corner point change amplitude between other corner points; corrects the corner point importance by the grayscale distribution of the pixels around each corner point to obtain the matching preference of each corner point; screens all corner points based on the matching preference to obtain matching corner points for fish sorting; performs template matching through the matching corner points to obtain template matching results; sorts the fish to be detected by species according to the template matching results; thereby improving the sorting accuracy of automatic fish sorting recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0039] Figure 1 The present invention is a flowchart of the steps of a video data acquisition method for fish sorting.
[0040] Figure 2 The present invention is a flow chart showing the characteristic relationship of a video data acquisition method for fish sorting. DETAILED DESCRIPTION
[0041] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of a video data acquisition method and system for fish sorting proposed by the present invention, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0042] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0043] The specific scheme of the video data acquisition method and system for fish sorting provided by the present invention is described in detail below with reference to the accompanying drawings.
[0044] See also Figure 1 , which shows a flow chart of the steps of a video data acquisition method for fish sorting provided by an embodiment of the present invention, the method comprising the following steps:
[0045] Step S001: Acquire video data of a fish to be detected; decompose the video data into a plurality of frames of images of the fish to be detected.
[0046] It should be noted that the dam water conservancy fish passage scheme can protect and restore the ecological connectivity of the dam's fish population; and the fish sorting process of the fish passage scheme uses a template matching algorithm to identify the species of fish in the collected video, and then realizes fish sorting; among them, the sorting pool sorts out a single fish to be tested from the fish collecting bucket, first shoots a video of the fish to be tested, and then extracts image frames from the video, and uses a template matching algorithm to identify the type of fish on the image frames, and then opens the corresponding fish sorting door according to the type of fish to realize fish sorting.
[0047] In a specific implementation of the embodiment of the present invention, video data of the fish to be detected is obtained; and a specific method of decomposing the video data into a plurality of frames of images of the fish to be detected is:
[0048] In the fish lift system, the water flow fish attracting method is used to attract fish into the fish collecting corridor; the number of fish in the fish collecting channel is monitored by the fish monitoring system, and when the number of fish reaches a certain number, the fish driving fence is used to drive the fish into the fish collecting bucket; when the fish in the fish collecting bucket accumulates to a certain number, the fish driving operation is suspended, and the fish collecting bucket is lifted to the sorting pool in the fish sorting and temporary rearing system by the fish sorting and temporary rearing system's opening and closing trolley; the high-definition camera of the fish sorting and temporary rearing system is used to shoot video data of the fish to be detected in the sorting pool, and the video acquisition time is selected as 10s to obtain the video data of the fish to be detected; the video data is decomposed into a plurality of frames of images of the fish to be detected.
[0049] So far, several frames of images of fish to be detected in the video data of the fish to be detected are obtained through the above method.
[0050] Step S002: Obtain all reference images of each frame of the fish image to be detected; obtain the degree of difference between adjacent frame images of each frame of the fish image to be detected according to the grayscale difference between each frame of the fish image to be detected and the reference image; screen all frames of the fish image to be detected based on the degree of difference between adjacent frame images to obtain images to be matched.
[0051] It should be noted that, due to the differences in the degree of movement of fish at different times, there will be motion blur in the video frame images of the fish. Obviously, in the images to be matched when matching fish recognition templates, the smaller the motion blur of the fish, the higher the sorting accuracy of the fish will be. For any frame of fish image to be detected, the greater the difference in performance between the frame of fish image to be detected and the adjacent frames of fish image to be detected, the greater the possibility that the movement amplitude of the fish in the frame of fish image to be detected is large, and the lower the preference of the frame of fish image to be detected as the image to be matched for fish recognition template matching. Therefore, the degree of difference between adjacent frame images is used to screen all frames of fish images to be detected to obtain images to be matched.
[0052] Preferably, in some implementations of the embodiments of the present invention, the specific method for obtaining all reference images of each frame of the fish image to be detected is:
[0053] The first The adjacent frames of fish images to be detected before and after the frame of fish images to be detected are both recorded as The reference image of the fish image to be detected.
[0054] Preferably, in some implementations of the embodiments of the present invention, according to the grayscale difference between each frame of the fish image to be detected and the reference image, the specific method for obtaining the difference degree between adjacent frames of each frame of the fish image to be detected is:
[0055] The first The adjacent frames of fish images to be detected before and after the frame of fish images to be detected are both recorded as A reference image of the fish image to be detected;
[0056] The first The first frame of the fish image to be detected Pixels and The first frame of the fish image to be detected The reference image The absolute value of the difference in grayscale values between pixels is recorded as The fish image to be detected is The reference image The grayscale difference value of the pixel points; The fish image to be detected is The mean of the grayscale difference values of all pixels between the reference images is recorded as The reference difference value of the reference image; The mean of the reference difference values of all reference images of the fish image to be detected in the frame is taken as the The difference degree between adjacent frames of the fish image to be detected.
[0057] The specific formula is:
[0058]
[0059] In the formula, Indicates The difference degree between adjacent frames of the image of the fish to be detected; Indicates The number of all reference images of the fish image to be detected; Indicates The number of pixels of all pixels in the frame of the fish image to be detected; Indicates The first frame of the fish image to be detected The gray value of each pixel; Indicates The first frame of the fish image to be detected The reference image The gray value of each pixel; Indicates taking the absolute value.
[0060] It should be noted that The greater the difference between adjacent frames of the fish image to be detected, the The larger the movement amplitude of the fish in the frame to be detected, the The frame of the fish to be detected has a lower preference as the image to be matched for the fish recognition template matching.
[0061] Preferably, in some implementations of the embodiments of the present invention, all frames of fish images to be detected are screened based on the degree of difference between adjacent frame images, and the specific method for obtaining the image to be matched is:
[0062] Among all the frames of fish images to be detected, the fish image to be detected with the smallest difference between adjacent frame images is used as the image to be matched.
[0063] So far, the image to be matched is obtained through the above method.
[0064] Step S003: Obtain all corner points in the image to be matched; obtain the background cluster stability of each corner point by analyzing the changes in the clustering clusters to which each corner point belongs in the clustering results after multiple clustering of the image to be matched; obtain the corner point variation range of each corner point according to the distance difference between the image to be matched and the reference image; obtain the corner point importance of each corner point according to the background cluster stability of each corner point and the difference in the corner point variation range with other corner points; correct the corner point importance according to the grayscale distribution of the pixels around each corner point to obtain the matching preference of each corner point.
[0065] It should be noted that since the corner points may fall in the non-fish area of the image to be matched, and the corner points used in the fish sorting and identification process should first ensure that the corner points fall in the fish area, it is necessary to analyze and judge the probability that the background where each corner point in the image to be matched is a fish, and obtain the matching preference degree of the corner point through the stability of the background cluster of the corner point, the variation amplitude of the corner point and the importance of the corner point.
[0066] Preferably, in some implementations of the embodiments of the present invention, a SIFT corner point detection algorithm is used to obtain all corner points in the image to be matched.
[0067] The SIFT corner point detection algorithm is an existing algorithm, and will not be described in detail in this embodiment.
[0068] 1. Obtain the background cluster stability of each corner point.
[0069] It should be noted that since fish are constantly moving, even fish in a relatively resting state will have slight movements due to breathing movements, while non-fish areas are still, that is, relatively stable; therefore, Kmeans clustering with different K values is performed on the matching images. If the cluster where a corner point is located does not change with the change of K value and is relatively stable, it means that the cluster where the corner point is located is likely to be a non-fish area; otherwise, it means that the cluster of the corner point is likely to be a fish area. Therefore, the stability of the background cluster of the corner point is constructed according to the above logic.
[0070] Preferably, in some implementations of the embodiments of the present invention, by analyzing the changes in the clusters to which each corner point belongs in the clustering results after multiple clustering of the image to be matched, the specific method for obtaining the stability of the background cluster of each corner point is:
[0071] Preset two clustering parameters , wherein this embodiment is based on This example is described as an example, and this embodiment is not specifically limited. Depends on the specific implementation situation;
[0072] by As the starting point, the step length is 1 and it is iterated until the value after iteration is , stop the iteration, and get the values after all iterations; record the numerical sequence composed of the values after all iterations as the clustering parameter sequence; use each clustering parameter in the clustering parameter sequence as the K value of Kmeans clustering, and perform Kmeans clustering with different K values on the matching images to obtain several clustering results;
[0073] The first The corner point is The number of all pixels in the cluster in the clustering result is recorded as The corner point is The background cluster area under the clustering result; The corner point is The background cluster area under the clustering result is The mean of the difference between the mean values of the background cluster areas of the corner points under all clustering results is recorded as The corner point is The change factor under the clustering result; The normalized value of the reciprocal of the mean of the variation factor of the corner point under all clustering results is taken as the The stability of the background cluster at each corner point.
[0074]
[0075] In the formula, Indicates The stability of the background cluster at each corner point; Represents the number of all clustering results of the image to be matched; Indicates The corner point is The number of all pixels in the clustering cluster in the clustering result; Indicates The mean of the background cluster area of each corner point under all clustering results; Indicates taking the absolute value; represents the linear normalization function.
[0076] It should be noted that the Kmeans clustering algorithm is an existing algorithm and will not be described in detail in this embodiment.
[0077] At this point, the background cluster stability of each corner point is obtained.
[0078] 2. Get the corner point change amplitude of each corner point.
[0079] It should be noted that, due to the difference in the movement amplitude of different parts of the fish during its movement, the smaller the movement amplitude, the lower the degree of motion blur. Therefore, the matching corner points used for fish sorting should be located in the area with smaller movement amplitude on the fish body. Therefore, the matching corner points are reflected according to the changes of a corner point in the image to be matched in its adjacent image frames.
[0080] Preferably, in some implementations of the embodiments of the present invention, the specific method for obtaining the corner point variation amplitude of each corner point according to the distance difference between the image to be matched and the reference image is:
[0081] The first The corner point is The corresponding pixel point on the vertical projection of the reference image is recorded as The corner point is The target position on the reference image;
[0082] For Any corner point in the reference image is compared with the The corner point is The Euclidean distance between the target positions on the first reference image is used as the change distance of the corner point; Among all the corner points in the reference image, the corner point with the smallest change distance is taken as the first The corner point is target corner points on the reference image;
[0083] The first The average of the change distances of the target corner points on all reference images is taken as the The range of change of the corner points.
[0084] The specific formula is:
[0085]
[0086] In the formula, Indicates The variation range of the corner points; Represents all reference images of the image to be matched; Indicates The corner point is The change distance of the target corner points on the reference image.
[0087] It should be noted that the target position is defined as: completely overlapping the image to be matched with the reference image, and projecting all the coordinate information in the image to be matched onto the reference image below. At this time, the coordinate information that coincides with the corner point is the target position of the corner point in the image to be matched. The greater the change distance of the corner point from the target corner point on the reference image, the greater the change amplitude of the corner point in the image to be matched, and the lower the preference of the corner point as a corner point for template matching.
[0088] At this point, the corner point variation range of each corner point is obtained.
[0089] 3. Get the corner importance of each corner point.
[0090] It should be noted that the matching corner points used for fish sorting should satisfy the requirement that the corner points can only be located in the area where the fish's activity range is smaller, so that the sorting accuracy of fish sorting is higher at this time; for any corner point in the image to be matched, if the corner point is located in the area where the fish's activity range is larger, then in the image to be matched and the reference image, the stable relationship between the corner point and other corner points will be affected to a greater extent, and the greater the corner point's importance.
[0091] Preferably, in some implementations of the embodiments of the present invention, according to the stability of the background cluster of each corner point and the difference in the variation range of the corner point between each corner point and other corner points, the specific method for obtaining the corner point importance of each corner point is:
[0092] For the images to be matched except the Any corner point other than the first corner point, The absolute value of the difference between the corner points' change amplitudes is recorded as the change difference value of the corner points; The mean of the change difference values of all corner points except the corner points is recorded as The overall change difference value of the corner points in the image to be matched;
[0093] In the Among the reference images, The corner point is Any corner point other than the target corner point on the reference image is compared with the The corner point is The absolute value of the difference between the corner point change amplitudes of the target corner points on the reference images is recorded as the target change difference value of the corner point; The corner point is The mean of the target change difference values of all corner points other than the target corner points on the reference image is recorded as The corner point is The overall change difference of the target in the reference image;
[0094] The first The average of the target overall change difference values of the corner points in all reference images is The absolute value of the difference between the overall change difference values of the corner points in the image to be matched is taken as the first The final change difference value of the corner points;
[0095] The first The final change difference between the corner points and The normalized value of the inverse of the product of the stability of the background clusters of the corner points is taken as the first The importance of each corner point.
[0096] The specific formula is:
[0097]
[0098] In the formula, Indicates The importance of each corner point; Indicates The stability of the background cluster at each corner point; Indicates that the image to be matched is The mean of the change difference values of all corner points except the corner points; Represents all reference images of the image to be matched; Indicates that except The corner point is The mean of the target change difference values of all corner points other than the target corner points on the reference image; Indicates taking the absolute value; represents the linear normalization function.
[0099] It should be noted that The average of the target overall change difference values of the corner points in all reference images is The greater the absolute value of the difference between the overall change difference values of the corner points in the image to be matched, the greater the The more likely the corner point is to be located in an area with a larger fish activity range, the The first corner point is less preferred as a template match. The more corner points there are, the less important they are.
[0100] At this point, the corner point importance of each corner point is obtained.
[0101] 4. Obtain the matching preference of each corner point.
[0102] It should be noted that in the process of fish sorting, the closer the corner point is to the brightly colored area on the fish body, the more likely the area where the corner point is located is to be a normal area on the fish body, otherwise it is more likely to be a motion blurred area. Therefore, the corner point used for fish sorting should be as close to the normal area of the fish as possible. Therefore, the matching preference degree of each corner point in the image to be matched is constructed according to the above logic.
[0103] Preferably, in some implementations of the embodiments of the present invention, the importance of the corner points is corrected by the grayscale distribution of the pixels around each corner point, and a specific method for obtaining the matching preference degree of each corner point is as follows:
[0104] Preset a neighborhood parameter , wherein this embodiment is based on This example is described as an example, and this embodiment is not specifically limited. Depends on the specific implementation situation;
[0105] First The corner points are used as the window center, and the window size is obtained as and record this window as The neighborhood range of the corner points;
[0106] The first The mean of the grayscale values of all pixels in the neighborhood of the corner point is recorded as the first mean; The corner point in the neighborhood of The absolute value of the difference between the pixel point and the first mean is recorded as The first difference of the pixels; The reciprocal of the mean of the first differences of all pixels in the neighborhood of the corner point is recorded as the confidence factor;
[0107] The first The product of the importance of the corner point and the confidence factor is taken as the The matching preference of each corner point.
[0108] The specific formula is:
[0109]
[0110] In the formula, Indicates The matching preference degree of each corner point; Indicates The number of all pixels in the neighborhood of a corner point; Indicates The corner point in the neighborhood of The gray value of each pixel; Indicates The mean gray value of all pixels in the neighborhood of a corner point; Indicates The importance of each corner point; Indicates taking the absolute value.
[0111] So far, the matching preference degree of each corner point is obtained through the above method.
[0112] Step S004: Screen all corner points based on the matching preference degree to obtain matching corner points for fish sorting; perform template matching through the matching corner points to obtain template matching results; and sort the fish to be detected according to the template matching results.
[0113] Preferably, in some implementations of the embodiments of the present invention, all corner points are screened based on the matching preference degree, and the specific method for obtaining the matching corner points for fish sorting is:
[0114] All corner points in the image to be matched are sorted from large to small according to the matching preference degree to obtain a sorted angle sequence; the first 4 angles in the sorted angle sequence are recorded as matching corner points for fish sorting.
[0115] Preferably, in some implementations of the embodiments of the present invention, the matching corner points are input into a template matching algorithm for template matching to obtain a template matching result; the fish to be detected are sorted by species according to the template matching result to obtain all species of fish to be detected; and corresponding sorting gates are opened for the species of fish to be detected to implement fish sorting.
[0116] The template matching algorithm is an existing algorithm, and will not be described in detail in this embodiment.
[0117] See also Figure 2 , which shows a characteristic relationship flow chart of a video data acquisition method for fish sorting.
[0118] Through the above steps, a video data acquisition method for fish sorting is completed.
[0119] Another embodiment of the present invention provides a video data acquisition system for fish sorting, the system comprising a memory and a processor, and when the processor executes the computer program stored in the memory, the above method steps S001 to S004 are performed.
[0120] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A video data acquisition method for fish sorting, characterized in that: The method comprises the following steps: Obtain video data of the fish to be detected; decompose the video data into a plurality of frames of images of the fish to be detected; Obtain all reference images of each frame of the fish image to be detected; obtain the degree of difference between adjacent frames of each frame of the fish image to be detected according to the grayscale difference between each frame of the fish image to be detected and the reference image; screen all frames of the fish image to be detected based on the degree of difference between adjacent frames to obtain images to be matched; Obtain all corner points in the image to be matched; obtain the background cluster stability of each corner point by analyzing the changes in the clustering clusters to which each corner point belongs in the clustering results after multiple clustering of the image to be matched; obtain the corner point change amplitude of each corner point according to the distance difference between the image to be matched and the reference image; obtain the corner point importance of each corner point according to the background cluster stability of each corner point and the difference in the corner point change amplitude between each corner point and other corner points; correct the corner point importance according to the grayscale distribution of the pixels around each corner point to obtain the matching preference of each corner point; Screening all corner points based on the matching preference degree to obtain matching corner points for fish sorting; performing template matching through the matching corner points to obtain template matching results; and sorting the fish to be detected according to the template matching results; The specific method for obtaining all reference images of each frame of the fish image to be detected is: The first The adjacent frames of fish images to be detected before and after the frame of fish images to be detected are both recorded as A reference image of the fish image to be detected; Based on the difference between adjacent frames, all frames of fish images to be detected are screened, and the specific method for obtaining the images to be matched is as follows: Among all the frames of fish images to be detected, the fish image to be detected with the smallest difference between adjacent frames is used as the image to be matched; According to the grayscale difference between each frame of the fish image to be detected and the adjacent frame of the fish image to be detected, the specific method for obtaining the difference degree between the adjacent frames of each frame of the fish image to be detected is: The first The adjacent frames of fish images to be detected before and after the frame of fish images to be detected are both recorded as A reference image of the fish image to be detected; The first The first frame of the fish image to be detected Pixels and The first frame of the fish image to be detected The reference image The absolute value of the difference in grayscale values between pixels is recorded as The fish image to be detected is The reference image The grayscale difference value of the pixel points; The fish image to be detected is The mean of the grayscale difference values of all pixels between the reference images is recorded as The reference difference value of the reference image; The mean of the reference difference values of all reference images of the fish image to be detected in the frame is taken as the The difference degree between adjacent frames of the image of the fish to be detected; By analyzing the changes in the clustering results of each corner point after multiple clustering of the image to be matched, the specific method for obtaining the stability of the background cluster of each corner point is as follows: Perform Kmeans clustering with different K values on the matching images to obtain several clustering results; The first The corner point is The number of all pixels in the cluster in the clustering result is recorded as The corner point is The background cluster area under the clustering result; The corner point is The background cluster area under the clustering result is The mean of the difference between the mean values of the background cluster areas of the corner points under all clustering results is recorded as The corner point is The change factor under the clustering result; The normalized value of the reciprocal of the mean of the variation factor of the corner point under all clustering results is taken as the The stability of the background cluster at each corner point; According to the distance difference between each corner point in the image to be matched and the image of the fish to be detected in the adjacent frame, the specific method for obtaining the change amplitude of each corner point is as follows: The first The corner point is The corresponding pixel point on the vertical projection of the reference image is recorded as The corner point is The target position on the reference image; For Any corner point in the reference image is compared with the The corner point is The Euclidean distance between the target positions on the first reference image is used as the change distance of the corner point; Among all the corner points in the reference image, the corner point with the smallest change distance is taken as the first The corner point is target corner points on the reference image; The first The average of the change distances of the target corner points on all reference images is taken as the The variation range of the corner points; According to the stability of the background cluster of each corner point and the difference in the change range of the corner point with other corner points, the specific method for obtaining the importance of each corner point is as follows: For the images to be matched except the Any corner point other than the first corner point, The absolute value of the difference between the corner points' change amplitudes is recorded as the change difference value of the corner points; The mean of the change difference values of all corner points except the corner points is recorded as The overall change difference value of the corner points in the image to be matched; In the Among the reference images, The corner point is Any corner point other than the target corner point on the reference image is compared with the The corner point is The absolute value of the difference between the corner point change amplitudes of the target corner points on the reference images is recorded as the target change difference value of the corner point; The corner point is The mean of the target change difference values of all corner points other than the target corner points on the reference image is recorded as The corner point is The overall change difference of the target in the reference image; The first The average of the target overall change difference values of the corner points in all reference images is The absolute value of the difference between the overall change difference values of the corner points in the image to be matched is taken as the first The final change difference value of the corner points; The first The final change difference between the corner points and The normalized value of the inverse of the product of the stability of the background clusters of the corner points is taken as the first The importance of each corner point; The importance of each corner point is corrected by the grayscale distribution of the pixels around each corner point. The specific method for obtaining the matching preference of each corner point is as follows: Preset a neighborhood parameter , with the The corner points are used as the window center, and the window size is obtained as and record this window as The neighborhood range of the corner points; The first The mean of the grayscale values of all pixels in the neighborhood of the corner point is recorded as the first mean; The corner point in the neighborhood of The absolute value of the difference between the pixel point and the first mean is recorded as The first difference of the pixels; The reciprocal of the mean of the first differences of all pixels in the neighborhood of the corner point is recorded as the confidence factor; The first The product of the importance of the corner point and the confidence factor is taken as the The matching preference degree of each corner point; The specific method of screening all corner points based on the matching preference degree and obtaining the matching corner points for fish sorting is as follows: All corner points in the image to be matched are sorted from large to small according to the matching preference degree to obtain a sorted angle sequence; the first 4 angles in the sorted angle sequence are recorded as matching corner points for fish sorting.
2. A video data acquisition system for fish sorting, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by a processor, the steps of a video data acquisition method for fish sorting as claimed in claim 1 are implemented.
Citation Information
Patent Citations
Fish behavior video identification method based on optical flow effect
CN111401308A
Intelligent supervision system for fish video recognition
CN112287913A
Garbage throwing behavior detection method based on video image feature analysis
CN117456428A
Three-axis turntable state monitoring method based on video monitoring
CN118334560A
Analysis and sorting in aquaculture
WO2021242368A1
