Intelligent lossless fish fry sorting equipment and method based on machine vision

Abstract

According to the intelligent fry lossless sorting equipment and method based on machine vision, fry in a fish entering bin can be recognized through a first binocular camera, the average length of the fry is obtained, and according to the average length of the fry, a driving motor is controlled through a control console to enable a hexagonal sorting barrel to rotate to a proper sorting face; the fry with the length smaller than the average value are discharged from the sorting holes of the sorting face, meanwhile, the sorting condition of the fry in the hexagonal sorting barrel is continuously monitored through the second binocular camera, complete sorting of the fry is ensured, and therefore accurate sorting automation of the fry is achieved, the sorting efficiency is improved, and the labor intensity is reduced; secondly, in the sorting process, water is continuously sprayed through a first water flow spray head and a second water flow spray head, the stability of fish fries entering the fish bin, the fish fry channel, the hexagonal sorting barrel, the outlet bin and the fish discharging channel is improved, the situation that the fish fries are damaged due to stress reaction caused by water shortage is prevented, the damage rate of living fish fries is reduced, and the breeding efficiency and the product quality are effectively improved.

Description

Technical Field

[0001] The invention relates to the technical field of aquatic fish fry sorting, and in particular to intelligent fish fry non-destructive sorting equipment and method based on machine vision. Background Art

[0002] In recent years, my country's aquaculture industry has developed rapidly, and the output of various types of aquatic seedlings has continued to rank among the top in the country, becoming an important distribution center for the production and sale of aquatic products and aquatic seedlings in the country. However, during the artificial seedling stage of highly economic carnivorous fish, cannibalism is common among the same species, which is one of the main factors restricting the survival rate of seedlings in the seedling stage. Therefore, before the fry of highly economic carnivorous fish are cultured, they need to be sorted to separate fry of different sizes. First, it is to reduce the damage to the fry during artificial sorting, and to sort out excellent and fresh fry for breeding to improve the breeding quality; second, it is to more accurately sort out fry of different sizes to avoid the situation where large fish cannibalize small fish and compete for food due to incomplete sorting, so as to improve the yield of seedling breeding.

[0003] At present, the fry sorting mainly relies on manual means. Taking sea bass as an example, the fry breeding process is divided into three stages. After the acclimatization is completed, the water splash reaches 6 days, and the fry body length reaches about 2-2.5 cm. At this time, the scales of the fry have grown relatively complete, and the big ones eat the small ones seriously, so they need to be screened in time. However, most of the traditional fish sorting devices are used to sort large fish that have grown to a certain size. The fry sorting is still in the stage of manual screening through the net. Due to the small size and fast swimming speed of the fry, it is difficult to achieve accurate and non-destructive fry sorting. Summary of the invention

[0004] The purpose of the present invention is to provide an intelligent non-destructive fish fry sorting device and method based on machine vision, which can not only realize the accurate and automated sorting of fish fry, improve the sorting efficiency and reduce the labor intensity, but also reduce the damage rate of live fish fry, and effectively improve the breeding efficiency and product quality.

[0005] In order to achieve the above object, the present invention adopts the following scheme:

[0006] An intelligent non-destructive fry sorting device based on machine vision comprises a frame, a fish entry bin, a sorting bin, an outlet bin and a control console; the fish entry bin is arranged on one side of the frame, the sorting bin and the outlet bin are both arranged on the other side of the frame, and the sorting bin is located above the outlet bin;

[0007] An electric gate and a fry passage connected to the sorting bin are provided at the outlet end of the fish entry bin, a first water flow nozzle is provided at one end of the fish entry bin away from the fry passage, and a first binocular camera is mounted above the fish entry bin;

[0008] The sorting bin comprises a hexagonal sorting barrel and a sorting barrel driving mechanism for driving the hexagonal sorting barrel to rotate forward or backward. The hexagonal sorting barrel is rotatably arranged on the outlet bin. The hexagonal sorting barrel has six sorting surfaces. The sorting holes on the six sorting surfaces have different aperture sizes. A second water flow nozzle and a second binocular camera are installed inside the hexagonal sorting barrel.

[0009] A fish discharge channel is provided at the bottom of the outlet bin;

[0010] The first water flow nozzle and the second water flow nozzle are respectively connected to a water supply control system;

[0011] The console is installed on the frame, and is electrically connected to the first binocular camera, the second binocular camera, the electric gate, the driving motor in the sorting barrel driving mechanism, and the valves and water pumps in the water supply control system.

[0012] As a preferred embodiment of the above-mentioned intelligent lossless fish fry sorting equipment, the six sorting surfaces are respectively the first sorting surface, the second sorting surface, the third sorting surface, the fourth sorting surface, the fifth sorting surface and the sixth sorting surface which are connected end to end in sequence, and the aperture diameters of the sorting holes of the first sorting surface, the aperture diameters of the sorting holes of the second sorting surface, the aperture diameters of the sorting holes of the third sorting surface, the aperture diameters of the sorting holes of the fourth sorting surface, the aperture diameters of the sorting holes of the fifth sorting surface and the aperture diameters of the sorting holes of the sixth sorting surface increase in sequence.

[0013] As a preferred solution of the above-mentioned intelligent lossless fish fry sorting equipment, the sorting holes on the first sorting surface, the sorting holes on the second sorting surface, the sorting holes on the third sorting surface, the sorting holes on the fourth sorting surface, the sorting holes on the fifth sorting surface and the sorting holes on the sixth sorting surface are all long holes.

[0014] As a preferred solution of the above-mentioned intelligent non-destructive fish fry sorting equipment, the width of the sorting holes on the first sorting surface is 3mm and the length is 20mm; the width of the sorting holes on the second sorting surface is 4mm and the length is 30mm; the width of the sorting holes on the third sorting surface is 5mm and the length is 40mm; the width of the sorting holes on the fourth sorting surface is 6mm and the length is 50mm; the width of the sorting holes on the fifth sorting surface is 8mm and the length is 60mm; the width of the sorting holes on the sixth sorting surface is 50mm and the length is 300mm.

[0015] As a preferred solution of the above-mentioned intelligent lossless fish fry sorting equipment, each sorting surface of the hexagonal sorting barrel is provided with a non-porous area with a width of 1 cm to 3 cm near the connection between the surfaces.

[0016] As a preferred solution of the above-mentioned intelligent lossless fry sorting equipment, the installation height of the fish inlet bin is greater than the installation height of the outlet bin; the fry channel is arranged inclined, the high end of the fry channel is connected to the outlet end of the fish inlet bin, and the low end of the fry channel extends into the hexagonal sorting barrel.

[0017] As a preferred solution of the above-mentioned intelligent lossless fish fry sorting equipment, the sorting barrel driving mechanism includes a driving motor, a driving gear and a driven gear ring, the driven gear ring is fixedly connected to the outside of the hexagonal sorting barrel, the driving motor is fixed on the frame, and the power output end of the driving motor is connected to the driving gear, and the driving gear is meshed with the driven gear ring.

[0018] As a preferred solution of the above-mentioned intelligent lossless fry sorting equipment, a driven bearing group is provided on the outer side of the hexagonal sorting barrel, and a driven track rollingly matched with the driven bearing group is provided on the side wall of the outlet bin.

[0019] As a preferred solution of the above-mentioned intelligent fish fry lossless sorting equipment, the sorting chamber is provided with a chamber cover, and the chamber cover is arranged on the outside of the hexagonal sorting barrel.

[0020] In addition, the present invention also provides a sorting method using the intelligent non-destructive fry sorting device based on machine vision described in the above contents, which comprises the following steps:

[0021] S1, open the first water flow nozzle, and pour all the fry to be sorted into the fish storage;

[0022] S2, using the first binocular camera to identify the average length of fry;

[0023] S21, the fry in the fish tank are photographed by the first binocular camera, and the photographed images are corrected for tangential and radial distortions by the formula:

[0024]

[0025]

[0026] Among them, X1, X2, Y1, and Y2 are the corrected coordinates, (x, y) are the original coordinates, and r x 、r y Indicates the focal length in each direction, A is the precision coefficient of radial distortion, K1 and K2 are the tangential correction coefficients;

[0027] S22, the YOLOv5 model is used for fish fry target detection and key point detection on the corrected image, and the fish mouth, gills, tail root and tail are used as key feature points. The adaptive Census transform is used to extract the information of the edge area of ​​the image, and the cost aggregation is performed through the minimum spanning tree MST aggregation model. The MST aggregation model includes bottom-up and top-down aggregation. The two aggregation formulas are as follows:

[0028]

[0029] in, are the aggregation results from leaf nodes to root nodes and from root nodes to leaf nodes, S(p,q) is the similarity between two nodes, and the set C h (p) contains all the children of pixel p. The q in formula (4) represents the child node of node p. It is represented as the aggregation result of the previous child node q. Pr(p) in formula (5) is the parent node of node p;

[0030] Finally, the winner-takes-all algorithm is used to determine the optimal disparity value of each pixel by the cost matrix after cost aggregation. The disparity map composed of the optimal disparity values ​​is the optimal disparity map.

[0031] S23, using the camera parameters and disparity map, calculates the three-dimensional coordinates of the feature points according to the triangulation principle, and calculates the real lengths in the three-dimensional space between AC, CD and DB according to the Euclidean distance formula (6):

[0032]

[0033] Where BL is the length between two key points of the fry, S and P represent two key points, respectively, and x, y, and z represent the coordinate positions of the key points in three-dimensional space;

[0034] S24, adding the three obtained lengths to obtain the accurate real length of the fry; after identifying all the lengths of the fry, calculating the average value, and the fry whose length is less than the average value are the fry that need to be sorted out;

[0035] S3, according to the average length of the fry obtained in step S2, controlling the driving motor to rotate the hexagonal sorting barrel to a suitable sorting surface; wherein the hexagonal sorting barrel has six sorting surfaces, and the aperture sizes of the sorting holes on the six sorting surfaces are different;

[0036] S4, controlling the electric gate to open the outlet of the fish inlet bin, so that the fry enter the sorting barrel through the fry channel;

[0037] S5, controlling the driving motor to make the hexagonal sorting barrel vibrate in the forward and reverse directions, and the vibration amplitude should ensure that the fry cannot leak out from other sorting surfaces; at the same time, according to the water flow rate and intensity corresponding to different sorting surfaces set in advance in the control console, the second water flow nozzle is controlled to appropriately increase or decrease the water volume;

[0038] S6, using the second binocular camera to continuously monitor the fry sorting in the hexagonal sorting bucket, and judging whether the length of the fry in the hexagonal sorting bucket is greater than or equal to the average length of the fry according to the average length of the fry obtained in step S2, if yes, the fry sorting needs to be completed, if not, the sorting continues;

[0039] S7, when the fry sorting is completed, the driving motor is controlled to rotate the hexagonal sorting barrel to the sorting surface with the largest sorting hole diameter, and the fish are discharged, and the fry or large fish with a length greater than or equal to the average length of the fry are discharged from the outlet bin.

[0040] The intelligent non-destructive sorting device and method of fish fry based on machine vision provided by the present invention has the following beneficial effects compared with the prior art:

[0041] The intelligent non-destructive fry sorting device and method based on machine vision of the present invention can identify the fry entering the fish bin through the first binocular camera and obtain the average length of the fry. According to the average length of the fry, the control console controls the driving motor in the sorting barrel driving mechanism to make the hexagonal sorting barrel turn to the appropriate sorting surface, so that the fry with a length less than the average value is discharged from the sorting holes of the sorting surface. At the same time, the second binocular camera is used to continuously monitor the sorting of the fry in the hexagonal sorting barrel to ensure that the fry are sorted completely, thereby realizing the accurate and automated sorting of the fry, improving the sorting efficiency and reducing the labor intensity. Secondly, in the sorting process, the first water flow The nozzle and the second water flow nozzle spray water continuously to improve the stability of the fry entering the fish warehouse, the fry channel, the hexagonal sorting barrel, the exit warehouse and the fish discharge channel, and prevent the fry from being damaged due to stress reaction due to lack of water, thereby reducing the damage rate of live fry and effectively improving the breeding efficiency and product quality; furthermore, the water volume control of the first water flow nozzle and the second water flow nozzle is realized according to the water flow rate and intensity corresponding to the different sorting surfaces set in advance in the control console, so that the water volume can be adaptively increased or decreased according to the length of the fry to be sorted, so as to realize water volume regulation and save water while ensuring that the damage caused by the stress reaction of the fry due to lack of water is effectively reduced.

[0042] In summary, the intelligent non-destructive fry sorting equipment and method based on machine vision implemented in the present invention can simulate the artificial fry sorting process, reduce the damage to the fry during manual sorting, solve the problem of incomplete fry sorting, and realize fully automated intelligent non-destructive sorting of highly economical carnivorous fish fry, thereby increasing the fry breeding yield, increasing farmers' income, and bringing better economic benefits to farmers. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a structural schematic diagram of an intelligent non-destructive fish fry sorting device based on machine vision provided by an embodiment of the present invention;

[0044] Figure 2 1 is another structural schematic diagram of the intelligent fish fry lossless sorting device based on machine vision provided by an embodiment of the present invention;

[0045] Figure 3 1 is a structural diagram of the connection between the hexagonal sorting barrel and the sorting barrel driving mechanism in an embodiment of the present invention;

[0046] Figure 4 It is a schematic diagram of the arrangement of the sorting surfaces of the hexagonal sorting barrel according to the size of the sorting hole in the embodiment of the present invention;

[0047] Figure 5 It is a structural schematic diagram of the fish inlet and outlet bins in an embodiment of the present invention.

[0048] Reference numerals:

[0049] Frame 1; fish inlet bin 2; sorting bin 3; outlet bin 4; control console 5; electric gate 6; fry channel 7; first water flow nozzle 8; first binocular camera 9; second water flow nozzle 10; second binocular camera 11; mounting bracket 12; fish discharge channel 13; driven bearing group 14; driven track 15;

[0050] Hexagonal sorting barrel 31; sorting hole 311; non-hole area 312; sorting barrel driving mechanism 32; driving motor 321; driving gear 322; driven gear ring 323; machine compartment cover 33; first sorting surface A; second sorting surface B; third sorting surface C; fourth sorting surface D; fifth sorting surface E; sixth sorting surface F. DETAILED DESCRIPTION

[0051] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0052] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0053] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0054] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0055] like Figures 1 to 5 As shown, an intelligent fish fry lossless sorting device based on machine vision provided by an embodiment of the present invention comprises a frame 1, a fish entry bin 2, a sorting bin 3, an outlet bin 4 and a control console 5; the fish entry bin 2 is arranged on one side of the frame 1, the sorting bin 3 and the outlet bin 4 are both arranged on the other side of the frame 1, and the sorting bin 3 is located above the outlet bin 4; an electric gate 6 and a fish fry passage 7 connected to the sorting bin 3 are provided at the outlet end of the fish entry bin 2, and a fish fry passage 7 connected to the sorting bin 3 is provided at the end of the fish entry bin 2 away from the fish fry passage 7. The first water flow nozzle 8 is provided above the fish inlet bin 2. The first binocular camera 9 is mounted. The sorting bin 3 includes a hexagonal sorting barrel 31 and a sorting barrel driving mechanism 32 for driving the hexagonal sorting barrel 31 to rotate forward or reversely. The hexagonal sorting barrel 31 is rotatably arranged on the outlet bin 4. The hexagonal sorting barrel 31 has 6 sorting surfaces (A, B, C, D, E and F shown in the figure). The aperture sizes of the sorting holes 311 on the 6 sorting surfaces are different. The interior of the hexagonal sorting barrel 31 is provided with a second water flow nozzle 10 and The second binocular camera 11, the second water flow nozzle 10 and the second binocular camera 11 are all fixed on the frame 1 through the mounting bracket 12, and will not rotate with the rotation of the hexagonal sorting barrel 31; the bottom of the outlet bin 4 is provided with a fish discharge channel 13; the first water flow nozzle 8 and the second water flow nozzle 10 are respectively connected to the water supply control system; the console 5 is installed on the frame 1, and the console 5 is respectively electrically connected to the first binocular camera 9, the second binocular camera 11, the electric gate 6, the drive motor 321 in the sorting barrel drive mechanism 32 and the valves and water pumps in the water supply control system, and the console 5 is used to set the water flow velocity corresponding to each sorting surface of the hexagonal sorting barrel 31, the drum rotation rate (i.e., the shaking rate) and other parameters, and receive the image data fed back by the first binocular camera 9 and the second binocular camera 11 to perform fry length recognition processing, and at the same time, it can control the opening and closing state of the electric gate 6, the rotation rate of the drive motor 321 and the output water volume of the water flow nozzles 8 and 10 according to these parameters and the fry length.

[0056] Therefore, according to the intelligent non-destructive fry sorting equipment based on machine vision provided by the present invention, the fry entering the fish bin 2 can be identified by the first binocular camera 9, and the average length of the fry can be obtained. According to the average length of the fry, the control console 5 controls the driving motor in the sorting barrel driving mechanism 32 to make the hexagonal sorting barrel 31 rotate to the appropriate sorting surface, so that the fry with a length less than the average value can be discharged from the sorting holes 311 of the sorting surface. At the same time, the second binocular camera 11 continuously monitors the sorting of the fry in the hexagonal sorting barrel 31 to ensure that the fry are sorted completely, thereby realizing the accurate and automated sorting of the fry, improving the sorting efficiency and reducing the labor intensity. Secondly, during the sorting process, The first water flow nozzle 8 and the second water flow nozzle 10 continuously spray water to improve the stability of the fry entering the fish warehouse 2, the fry channel 7, the hexagonal sorting barrel 31, the outlet warehouse 4 and the fish discharge channel 13, and prevent the fry from being damaged by stress reaction due to lack of water, thereby reducing the damage rate of live fry and effectively improving the breeding efficiency and product quality; furthermore, the water volume control of the first water flow nozzle 8 and the second water flow nozzle 10 is realized according to the water flow rate and intensity corresponding to the different sorting surfaces set in advance in the control console 5, so that the water volume can be adaptively increased or decreased according to the length of the fry to be sorted, and the water volume can be controlled while ensuring that the damage caused by the stress reaction of the fry due to lack of water is effectively reduced, thereby saving water.

[0057] For example, Figure 4 As shown, the six sorting surfaces are respectively the first sorting surface A, the second sorting surface B, the third sorting surface C, the fourth sorting surface D, the fifth sorting surface E and the sixth sorting surface F which are connected end to end in sequence, and the apertures of the sorting holes 311 of the first sorting surface A, the apertures of the sorting holes 311 of the second sorting surface B, the apertures of the sorting holes 311 of the third sorting surface C, the apertures of the sorting holes 311 of the fourth sorting surface D, the apertures of the sorting holes 311 of the fifth sorting surface E and the apertures of the sorting holes 311 of the sixth sorting surface F increase in sequence. Such a design can be conducive to the orderly sorting control of the hexagonal sorting barrel 31. When it is necessary to perform multi-stage sorting of fry, the fry can be sorted step by step from small to large according to the apertures of the sorting holes 311 of the sorting surface.

[0058] For example, Figure 3 and Figure 4 As shown, the sorting holes 311 on the first sorting surface A, the sorting holes 311 on the second sorting surface B, the sorting holes 311 on the third sorting surface C, the sorting holes 311 on the fourth sorting surface D, the sorting holes 311 on the fifth sorting surface E and the sorting holes 311 on the sixth sorting surface F are all long holes to adapt to the formation of fry growth.

[0059] For example, the sorting holes 311 on the first sorting surface A have a width of 3 mm and a length of 20 mm, and are used to sort fry of 6-day-old, hatched for 20-30 days, 2.0-2.5 cm in length, and 0.2-0.3 g in weight; the sorting holes 311 on the second sorting surface B have a width of 4 mm and a length of 30 mm, and are used to sort fry of 7-day-old, hatched for 30-40 days, 2.5-3.0 cm in length, and 0.4-0.5 g in weight; the sorting holes 311 on the third sorting surface C have a width of 5 mm and a length of The width of the sorting holes 311 on the fourth sorting surface D is 6mm, and the length is 50mm, which is used to sort the fry of 8 days old, hatched for 40-50 days, with a length of 3-4cm and a weight of 0.8-1g; the width of the sorting holes 311 on the fourth sorting surface D is 6mm, and the length is 50mm, which is used to sort the fry of 9 days old, hatched for 50-60 days, with a length of 4-5cm and a weight of 2-3.3g; the width of the sorting holes 311 on the fifth sorting surface E is 8mm, and the length is 60mm; the width of the sorting holes 311 on the sixth sorting surface F is 50mm, and the length is 300mm, which is used for draining fish.

[0060] For example, Figure 3 and Figure 4 As shown, each sorting surface of the hexagonal sorting barrel 31 is provided with a non-porous area 312 with a width of 1 cm to 3 cm near the connection between the surfaces, so as to prevent uneven sorting of the fry due to some fry not being on the specified sorting surface during the sorting process.

[0061] For example, Figure 1 , Figure 2 and Figure 5 As shown, the installation height of the fish inlet bin 2 is greater than the installation height of the outlet bin 4; the fry channel 7 is arranged obliquely, the high end of the fry channel 7 is connected to the outlet end of the fish inlet bin 2, and the low end of the fry channel 7 extends into the hexagonal sorting barrel 31. Such a design enables the fry to flow naturally into the hexagonal sorting barrel 31 under gravity.

[0062] For example, Figure 3 As shown, the sorting barrel driving mechanism 32 includes a driving motor 321, a driving gear 322 and a driven gear ring 323, wherein the driven gear ring 323 is fixedly connected to the outer side of the hexagonal sorting barrel 31, the driving motor 321 is fixed to the frame 1, and the power output end of the driving motor 321 is connected to the driving gear 322, and the driving gear 322 is meshed with the driven gear ring 323. When working, the driving motor 321 outputs torque, which is sequentially transmitted to the hexagonal sorting barrel 31 through the driving gear 322 and the driven gear ring 323, thereby driving the hexagonal sorting barrel 31 to rotate to the desired sorting surface or realize the forward and reverse shaking operation.

[0063] For example, Figure 3 and Figure 5 As shown, a driven bearing group 14 is provided on the outer side of the hexagonal sorting barrel 31, and a driven track 15 rollingly matched with the driven bearing group 14 is provided on the side wall of the outlet bin 4. Therefore, under the cooperation of the driven bearing group 14 and the driven track 15, the rolling support of the hexagonal sorting barrel 31 can be achieved, and the hexagonal sorting barrel 31 can be effectively prevented from axial movement, ensuring that the hexagonal sorting barrel 31 can rotate stably on the outlet bin 4.

[0064] For example, Figure 1 and Figure 2 As shown, the sorting bin 3 is provided with a bin cover 33, and the bin cover 33 is arranged on the outside of the hexagonal sorting barrel 31 to prevent the water flow from being thrown out of the equipment when the hexagonal sorting barrel 31 rotates.

[0065] Exemplarily, the first binocular camera 9 is provided with two Sony CCD sensors, and the binocular distance is 155 mm, forming an optical stereoscopic vision system.

[0066] In addition, the embodiment of the present invention also provides a sorting method using the intelligent non-destructive fish fry sorting device based on machine vision described in the above contents, which includes the following steps:

[0067] S1, open the first water flow nozzle 8, and pour all the fry to be sorted into the fish storage 2;

[0068] S2, using the first binocular camera 9 to identify the average length of the fry;

[0069] S21, the fry in the fish bin 2 are photographed by the first binocular camera 9, and the photographed image is corrected for tangential and radial distortion by the formula:

[0070]

[0071] Among them, X1, X2, Y1, and Y2 are the corrected coordinates, (x, y) are the original coordinates, and r x 、r y Indicates the focal length in each direction, A is the precision coefficient of radial distortion, K1 and K2 are the tangential correction coefficients;

[0072] S22, the YOLOv5 model is used for fish fry target detection and key point detection on the corrected image, and the fish mouth, gills, tail root and tail are used as key feature points. The adaptive Census transform is used to extract the information of the edge area of ​​the image, and the cost aggregation is performed through the minimum spanning tree MST aggregation model. The MST aggregation model includes bottom-up and top-down aggregation. The two aggregation formulas are as follows:

[0073]

[0074] in, are the aggregation results from leaf nodes to root nodes and from root nodes to leaf nodes, S(p,q) is the similarity between two nodes, and the set C h (p) contains all the children of pixel p. The q in formula (4) represents the child node of node p. It is represented as the aggregation result of the previous child node q. Pr(p) in formula (5) is the parent node of node p;

[0075] Finally, the winner-takes-all algorithm is used to determine the optimal disparity value of each pixel by the cost matrix after cost aggregation. The disparity map composed of the optimal disparity values ​​is the optimal disparity map.

[0076] S23, using the camera parameters and disparity map, calculates the three-dimensional coordinates of the feature points according to the triangulation principle, and calculates the real lengths in the three-dimensional space between AC, CD and DB according to the Euclidean distance formula (6):

[0077]

[0078] Where BL is the length between two key points of the fry, S and P represent two key points, respectively, and x, y, and z represent the coordinate positions of the key points in three-dimensional space;

[0079] S24, adding the three obtained lengths to obtain the accurate real length of the fry; after identifying all the lengths of the fry, calculating the average value, and the fry whose length is less than the average value are the fry that need to be sorted out;

[0080] S3, according to the length of the fry to be sorted, control the driving motor 321 to rotate the hexagonal sorting barrel 31 to a suitable sorting surface; wherein the hexagonal sorting barrel 31 has 6 sorting surfaces, and the aperture sizes of the sorting holes 311 on the 6 sorting surfaces are different;

[0081] S4, control the electric gate 6 to open the outlet end of the fish inlet bin 2, so that the fry enter the sorting barrel through the fry channel 7;

[0082] S5, control the driving motor 321 to make the hexagonal sorting barrel 31 vibrate in the forward and reverse directions, and the vibration amplitude should ensure that the fry cannot leak out from other sorting surfaces; at the same time, according to the water flow rate and intensity corresponding to different sorting surfaces set in advance in the control console 5, control the second water flow nozzle 10 to appropriately increase or decrease the water volume;

[0083] S6, using the second binocular camera 11 to continuously monitor the fry sorting situation in the hexagonal sorting barrel 31, and judging whether the length of the fry in the hexagonal sorting barrel 31 is greater than or equal to the average length of the fry according to the average length of the fry obtained in step S2, if yes, the fry sorting needs to be completed, if not, the sorting continues;

[0084] S7, when the fry sorting is completed, the driving motor 321 is controlled to rotate the hexagonal sorting barrel 31 to the sorting surface (i.e., the sixth sorting surface F) with the largest aperture of the sorting hole 311, and the fish are discharged, and the fry or large fish with a length greater than or equal to the average length of the fry flow out from the outlet bin 4.

[0085] Since the intelligent lossless fish fry sorting method of the embodiment of the present invention applies the intelligent lossless fish fry sorting equipment based on machine vision described in the above contents, it has all the beneficial effects of the above-mentioned intelligent lossless fish fry sorting equipment, which will not be repeated here.

[0086] It should also be noted that the average length identification method of fry adopted by the intelligent non-destructive fry sorting method of the embodiment of the present invention can effectively improve the accuracy of fry sorting and reduce the error of fry sorting. At the same time, the sorting method based on image recognition technology can avoid the damage to fry caused by traditional manual sorting methods.

[0087] To sum up, the intelligent lossless fry sorting equipment and method based on machine vision implemented in the embodiments of the present invention can simulate the artificial fry sorting process, reduce the damage to the fry during manual sorting, solve the problem of incomplete fry sorting, and realize fully automated intelligent lossless sorting of highly economical carnivorous fish fry, thereby improving the fry breeding yield, increasing farmers' income, and bringing better economic benefits to farmers.

[0088] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0089] Although embodiments of the present invention have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. An intelligent non-destructive fish fry sorting device based on machine vision, characterized in that: It comprises a frame, a fish inlet, a sorting bin, an outlet bin and a control console; the fish inlet is arranged on one side of the frame, the sorting bin and the outlet bin are both arranged on the other side of the frame, and the sorting bin is located above the outlet bin; An electric gate and a fry passage connected to the sorting bin are provided at the outlet end of the fish entry bin, a first water flow nozzle is provided at one end of the fish entry bin away from the fry passage, and a first binocular camera is mounted above the fish entry bin; The sorting bin comprises a hexagonal sorting barrel and a sorting barrel driving mechanism for driving the hexagonal sorting barrel to rotate forward or backward. The hexagonal sorting barrel is rotatably arranged on the outlet bin. The hexagonal sorting barrel has six sorting surfaces. The sorting holes on the six sorting surfaces have different aperture sizes. A second water flow nozzle and a second binocular camera are installed inside the hexagonal sorting barrel. A fish discharge channel is provided at the bottom of the outlet bin; The first water flow nozzle and the second water flow nozzle are respectively connected to a water supply control system; The console is installed on the frame, and is electrically connected to the first binocular camera, the second binocular camera, the electric gate, the driving motor in the sorting barrel driving mechanism, and the valves and water pumps in the water supply control system.

2. According to claim 1, the intelligent non-destructive fish fry sorting equipment based on machine vision is characterized in that: The six sorting surfaces are respectively the first sorting surface, the second sorting surface, the third sorting surface, the fourth sorting surface, the fifth sorting surface and the sixth sorting surface which are connected in sequence end to end, and the aperture diameters of the sorting holes of the first sorting surface, the aperture diameters of the sorting holes of the second sorting surface, the aperture diameters of the sorting holes of the third sorting surface, the aperture diameters of the sorting holes of the fourth sorting surface, the aperture diameters of the sorting holes of the fifth sorting surface and the aperture diameters of the sorting holes of the sixth sorting surface increase in sequence.

3. The intelligent non-destructive fry sorting device based on machine vision according to claim 2 is characterized in that: The sorting holes on the first sorting surface, the sorting holes on the second sorting surface, the sorting holes on the third sorting surface, the sorting holes on the fourth sorting surface, the sorting holes on the fifth sorting surface and the sorting holes on the sixth sorting surface are all long holes.

4. The intelligent non-destructive fry sorting device based on machine vision according to claim 3 is characterized in that: The width of the sorting holes on the first sorting surface is 3 mm and the length is 20 mm; the width of the sorting holes on the second sorting surface is 4 mm and the length is 30 mm; the width of the sorting holes on the third sorting surface is 5 mm and the length is 40 mm; the width of the sorting holes on the fourth sorting surface is 6 mm and the length is 50 mm; the width of the sorting holes on the fifth sorting surface is 8 mm and the length is 60 mm; the width of the sorting holes on the sixth sorting surface is 50 mm and the length is 300 mm.

5. The intelligent non-destructive fish fry sorting device based on machine vision according to claim 1, characterized in that: Each sorting surface of the hexagonal sorting barrel is provided with a non-porous area with a width of 1 cm to 3 cm near the connection between the surfaces.

6. The intelligent non-destructive fish fry sorting device based on machine vision according to claim 1, characterized in that: The installation height of the fish inlet bin is greater than the installation height of the outlet bin; the fry channel is arranged obliquely, the high end of the fry channel is connected to the outlet end of the fish inlet bin, and the low end of the fry channel extends into the hexagonal sorting barrel.

7. The intelligent non-destructive fish fry sorting device based on machine vision according to claim 1, characterized in that: The sorting barrel driving mechanism includes a driving motor, a driving gear and a driven gear ring. The driven gear ring is fixedly connected to the outer side of the hexagonal sorting barrel. The driving motor is fixed on the frame, and the power output end of the driving motor is connected to the driving gear, and the driving gear is meshed with the driven gear ring.

8. The intelligent non-destructive fish fry sorting device based on machine vision according to claim 1, characterized in that: A driven bearing group is arranged on the outer side of the hexagonal sorting barrel, and a driven track rollingly matched with the driven bearing group is arranged on the side wall of the outlet bin.

9. The intelligent non-destructive fish fry sorting device based on machine vision according to claim 1, characterized in that: The sorting bin is provided with a bin cover, and the bin cover is arranged on the outer side of the hexagonal sorting barrel.

10. A sorting method using the intelligent non-destructive fish fry sorting device based on machine vision according to any one of claims 1 to 9, characterized in that: The following steps are involved: S1, open the first water flow nozzle, and pour all the fry to be sorted into the fish storage; S2, using the first binocular camera to identify the average length of fry; S21, the fry in the fish tank are photographed by the first binocular camera, and the photographed images are corrected for tangential and radial distortions by the formula: Among them, X1, X2, Y1, and Y2 are the corrected coordinates, (x, y) are the original coordinates, and r x 、r y Indicates the focal length in each direction, A is the precision coefficient of radial distortion, K1 and K2 are the tangential correction coefficients; S22, the YOLOv5 model is used for fish fry target detection and key point detection on the corrected image, and the fish mouth, gills, tail root and tail are used as key feature points. The adaptive Census transform is used to extract the information of the edge area of ​​the image, and the cost aggregation is performed through the minimum spanning tree MST aggregation model. The MST aggregation model includes bottom-up and top-down aggregation. The two aggregation formulas are as follows: in, are the aggregation results from leaf nodes to root nodes and from root nodes to leaf nodes, S(p,q) is the similarity between two nodes, and the set C h (p) contains all the children of pixel p. The q in formula (4) represents the child node of node p. It is represented as the aggregation result of the previous child node q. Pr(p) in formula (5) is the parent node of node p; Finally, the winner-takes-all algorithm is used to determine the optimal disparity value of each pixel on the cost matrix after cost aggregation. The disparity map composed of the optimal disparity values ​​is the optimal disparity map. S23, using the camera parameters and disparity map, calculates the three-dimensional coordinates of the feature points according to the triangulation principle, and calculates the real lengths in the three-dimensional space between AC, CD and DB according to the Euclidean distance formula (6): Where BL is the length between two key points of the fry, S and P represent two key points, respectively, and x, y, and z represent the coordinate positions of the key points in three-dimensional space; S24, adding the three obtained lengths to obtain the accurate real length of the fry; after identifying all the lengths of the fry, calculating the average value, and the fry whose length is less than the average value are the fry that need to be sorted out; S3, according to the average length of the fry obtained in step S2, controlling the driving motor to rotate the hexagonal sorting barrel to a suitable sorting surface; wherein the hexagonal sorting barrel has six sorting surfaces, and the aperture sizes of the sorting holes on the six sorting surfaces are different; S4, control the electric gate to open the outlet end of the fish inlet bin, so that the fry enter the sorting barrel through the fry channel; S5, controlling the driving motor to make the hexagonal sorting barrel vibrate in the forward and reverse directions, and the vibration amplitude should ensure that the fry cannot leak out from other sorting surfaces; at the same time, according to the water flow rate and intensity corresponding to different sorting surfaces set in advance in the control console, the second water flow nozzle is controlled to appropriately increase or decrease the water volume; S6, using the second binocular camera to continuously monitor the fry sorting in the hexagonal sorting bucket, and judging whether the length of the fry in the hexagonal sorting bucket is greater than or equal to the average length of the fry according to the average length of the fry obtained in step S2, if yes, the fry sorting needs to be completed, if not, the sorting continues; S7, when the fry sorting is completed, the driving motor is controlled to rotate the hexagonal sorting barrel to the sorting surface with the largest sorting hole diameter, and the fish are discharged, and the fry or large fish with a length greater than or equal to the average length of the fry are discharged from the outlet bin.

Images