Tilapia fillet trimming track generation method based on image recognition

Abstract

The invention relates to a tilapia fillet trimming track generation method based on image recognition, and relates to the technical field of aquatic product processing, and the tilapia fillet trimming track generation method comprises the following steps: image data pre-processing, fish back burr trimming track, fish tail trimming track, fish belly trimming track and fish head side part trimming and spine removal V-opening track. According to the method, each independent fish fillet has different shapes and contours, and an exclusive trimming track is planned and customized through visual sampling.

Description

Technical Field

[0001] The present application relates to the technical field of aquatic product processing, and in particular to a method for generating a trimming trajectory for tilapia fillets based on image recognition. Background Art

[0002] Tilapia is a tropical small to medium-sized fish native to Africa, belonging to the class of bony fish, order of Perciformes, suborder of Perciformes, family of Osteichthyes. Since its introduction to China, its breeding and processing industries have developed rapidly. In addition to being eaten fresh, tilapia is mostly processed into fish fillets for sale. It has the advantages of few bones, no fishy smell, pure taste, and is suitable for different cooking methods. It is favored by high-value fish consumer groups in Europe and the United States.

[0003] Tilapia fillet processing technology includes: slaughtering, bleeding, filleting, skinning, trimming, sterilization and freezing. Among them, the slaughtering, filleting and skinning processes can be processed by machines automatically. However, the fillet trimming process is still completely dependent on manual processing.

[0004] Fish fillet trimming requires separate processing of multiple residues on both sides, including removing the remaining scraps of meat on the dorsal side of the fillet and the remnants of the fins on the ventral side, the remaining skin tissue on the body surface, the inner membrane of the abdominal cavity, blood spots and residual organs, and other excess residues that affect the appearance, as well as the fish meat with bone spurs attached to the spinal line in the head position and the remaining tail in the tail position.

[0005] With regard to the above-mentioned related technologies, the inventors believe that in the process of removing the residues, since various residues exist in different parts of the fish fillets and different fish fillets have different shapes, they need to be removed or trimmed in a targeted manner. The process is complicated, the operation is cumbersome, and the workers' experience is required to be high. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a method for generating a trimming trajectory for tilapia fillets based on image recognition, which solves the technical problem that in the process of removing residues, various residues exist in different parts of the fillets, and different fillets have different shapes, so directional removal or trimming is required, the process is complicated, the operation is cumbersome, and high requirements on workers' experience are required.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A method for generating a tilapia fillet trimming trajectory based on image recognition comprises the following steps:

[0009] S1: Process the captured fish fillet image to obtain the outer contour of the fish fillet and establish a camera geometric coordinate system model;

[0010] S2: Set the maximum x-coordinate value of the fish body to A and the minimum x-coordinate to B, extract all points between point A and point B in the outer contour, and the y-coordinate is less than the larger one of point A and point B, and obtain the fish back burr trimming trajectory;

[0011] S3: Calculate the minimum circumscribed rectangle of the fish body through the outer contour of the fish body, set the two vertices with smaller x-axis coordinates among the four vertices of the minimum circumscribed rectangle as bb1 and bb2 respectively, calculate the slope of the straight line passing through bb1 and bb2, record it as k, and calculate the cutting trajectory at the fish tail through the coordinates of point B and the k value;

[0012] S4: Identify the fish fins and fish belly in the image, extract all points of the fish belly outer contour, set the point of the fish belly at the minimum X coordinate as D, the point of the maximum X coordinate as F, set the point of the fish fin at the minimum X coordinate as C, and connect C, D, and F to form a fish belly trimming trajectory;

[0013] S5: Identify the spine mark in the fish fillet image, designate the point with the maximum value of the spine in the x-coordinate as N, and the intersection of the twelfth diaphragm stripe and the spine mark as M; set the spine V-mouth trimming factor as e w %, and according to the V-mouth trimming factor, the trimming points N1 and N2 on both sides of the spine are obtained. F, N1, M, N2, and A are connected to form the V-mouth trajectory for trimming the fish head side and removing the spine.

[0014] Through the above technical solution, each independent fish fillet has a different appearance contour. Through visual sampling, a customized exclusive trimming trajectory is planned, which simplifies the fish fillet repair process, reduces the operation difficulty, and reduces the experience requirements for workers.

[0015] Furthermore, in step S2, the fish back rough edge trimming trajectory is set as f1, the extracted coordinate points are converted into the camera geometric coordinate system, and the fish back curve is fitted, and the equation f1=a0+a1*x+a2*x is set 2 +a3*x 3 , and use the least squares method to find the coefficients.

[0016] Through the above technical solution, a prepared fitting curve is set to make the fish back trimmed more smooth and improve the meat yield of the fish fillet.

[0017] Further, in step S3, the trimming factor of the fishtail back is set as e1, and a point on the f1 curve is set as B ′ , B ′ The x coordinate of A = the x coordinate of B + e1.

[0018] Through the above technical solution, the trimming factor is set. According to the trimming process requirements, by assigning values ​​to the trimming factors of each part, precise control of different trimming requirements can be achieved.

[0019] Furthermore, in step S2, the back trimming factor is set to e b , set the cutting path of the fish back to f2, and set the equation f2 = f1 + e b , the cutting starting point of f2 is A ′ , set equation A ′ =A+e b , the cutting end point is B″, the x coordinate of B″ = B ′ x coordinate of +e b .

[0020] Further, in step S3, according to the coordinates of B″ and the k value, the cutting path f3 of the fishtail is calculated, the intersection point of the cutting path f3 of the fishtail and the outer contour of the fish body is C, the cutting end point of the cutting path f3 is set to C', and the trimming factor of the fishtail is set to e t , the y coordinate of C' = the y coordinate of C - e t .

[0021] Furthermore, in step S4, the cutting path of the first section of the fish belly is set to f4, and the point with the minimum y value in the outer contour of the fish fin is set to E, C, E, and D as the cutting path of f4.

[0022] Further, in step S4, the cutting path of the second section of the fish belly is set to f5, and all points between points D and F in the outer contour are extracted, that is, the x coordinate is between points D and F, and the y coordinate is less than the larger one of points D and F, and the extracted points are fitted to the fish belly curve, and the equation f5=g0+g1*x+g2*x is set. 2 +g3*x 3 , the coefficients are obtained by the least square method. f5 is the second cutting curve of the fish belly, with the starting point at point D and the end point at point F.

[0023] Further, in step S5, the fish head cutting path is set to

[0024] The first segment starts at point F and ends at point N1;

[0025] The second segment starts at point N1 and ends at point M;

[0026] The starting point of the third segment is point M and the end point is point N2;

[0027] The fourth segment starts at point N2 and ends at point A. ′ point;

[0028] The cutting path of the whole fish body is: A ′ -B″-C'-DEF-N1-M-N2-A ′ .

[0029] Furthermore, in step S2, there is a gap at the fish back position, and the deepest point of the gap is set as Q b , set Q b Points Q1 and Q2 on both sides, Q1 and Q2 are on the cutting path f2, Q1, Q b , Q2 three-point fitting curve f q =q0+q1*x+q2*x 2 +q3*x 3 , the trimming path is A ′ -Q1, Q1-Q2, Q2-B″.

[0030] Through the above technical solution, the trimming parameters can be automatically matched according to the actual defective part, and the surrounding curves can be fitted to perform the most economical trimming.

[0031] In summary, the present application includes at least one of the following beneficial technical effects of a method for generating a trimming trajectory for tilapia fillets based on image recognition:

[0032] 1. Each individual fish fillet has a different shape and contour. Through visual sampling, a customized trimming trajectory is planned;

[0033] 2. Set the trimming factor in the trajectory equation of each part of the fish fillet. According to the trimming process requirements, by assigning values ​​to the trimming factors of each part, accurate control of different trimming requirements can be achieved;

[0034] 3. In case of severely damaged fish fillets, the trimming parameters can be automatically matched according to the actual missing defective parts, and the surrounding curves can be fitted to perform the most economical trimming. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a schematic diagram of the fish fillet cutting trajectory provided by this embodiment;

[0036] Figure 2 It is a schematic diagram of the cutting trajectory of the fish fillet with a notch provided in this embodiment. DETAILED DESCRIPTION

[0037] In order to make the technical means, creative features, objectives and effects of the present invention easier to understand, the present invention is further described below in conjunction with specific embodiments.

[0038] The following is combined with Figure 1-2 This application is described in further detail.

[0039] The embodiment of the present application discloses a method for generating a trimming trajectory for tilapia fillets based on image recognition.

[0040] Reference Figure 1A method for generating a trimming trajectory for tilapia fillets based on image recognition comprises the following steps: image data pre-processing, a fish back burr trimming trajectory, a fish tail trimming trajectory, a fish belly trimming trajectory, and a fish head side trimming and spine removal V-mouth trajectory.

[0041] Image data pre-processing:

[0042] To process the captured fish fillet image, we first perform preprocessing operations such as denoising and smoothing, then extract the outer contour of the fish fillet using the Canny operator, and then process the image again using the open and close morphological operations to finally obtain the outer contour of the fish fillet and the circumscribed rectangle of the fish fillet, as shown in the figure below: Figure 1 The horizontal rectangle of the dotted line in the horizontal α divides the α circumscribed rectangle into two equal parts, the upper and the lower parts.

[0043] Establish the camera geometric coordinate system model, that is, the relationship between the image pixel coordinates and the world coordinate system.

[0044] Fish back rough edge trimming track:

[0045] Set the point with the largest x coordinate in the outer contour of the fish body to be A;

[0046] Set the point with the smallest x coordinate in the outer contour of the fish as B.

[0047] Extract all points between point A and point B in the outer contour, that is, points located in the lower part of the fish body, with x coordinates between point A and point B, and y coordinates less than the larger one of point A and point B, and set the equation y≤max(A,B). Convert the coordinates of the extracted points to the world coordinate system and fit the fish back curve. The fish back curve is f1, and set the equation f1=a0+a1*x+a2*x 2 +a3*x 3 , and use the least squares method to find the coefficients.

[0048] The cutting path of the fish back is f2, and the trimming factor of the fish back is e b , set the equation f2 = f1 + e b , e b As the trimming factor of the fish back, different values ​​can be assigned according to the trimming requirements. In the actual operation process, when the weight of the fish fillet is 100g-200g, the fine trimming is 4-6mm, the ordinary trimming is 2-3mm, and 2.0mm≤e b When the weight of the fillet <4.0 mm is 200-300 g, the fine trim is 6-8 mm, and the rough trim is 3-5 mm. Preferably, when the weight of the fillet is 100 g-200 g, the fine trim is 5 mm, and the rough trim is 3 mm, and when the weight of the fillet is 200-300 g, the fine trim is 8 mm, and the rough trim is 5 mm.

[0049] Set point B on the f1 curve ′, and the trimming factor of the fishtail back is e1, B ′ The x coordinate of is the x coordinate of B + e1.

[0050] The cutting path of the fish back is f2, and the cutting starting point of f2 is A ′ , A ′ The x-axis coordinate is the same as A, and the y-axis coordinate is A+e b The cutting end point is B″, and the x coordinate of B″ is the same as B ′ Same, the y-axis coordinate is B ′ +e b .

[0051] Therefore, the trimming trajectory of the fish back edge is from A ′ -B″ cuts along the f2 curve.

[0052] Fishtail trim track:

[0053] The minimum circumscribed rectangle of the fish body is calculated by the outer contour of the fish body, as shown in the β rectangular box in the figure. The two vertices with smaller x-axis coordinates among the four vertices of the minimum circumscribed rectangle are set as bb1 and bb2, and the two larger fixed points are set as bb3 and bb4. The slope of the straight line passing through bb1 and bb2 is calculated and recorded as k.

[0054] According to the coordinates of B″ and the value of k, the cutting path f3 at the fish tail and the straight line passing through B″ with a slope of k are calculated. The intersection of f3 and the outer contour of the fish body is recorded as C, and the trimming factor of the fish tail is set to e t , the y coordinate of C - e t The cutting end point of f3 is C'. Since C' is on the cutting path f3 of the fishtail, the x coordinate of C' can be obtained after the y coordinate of C' is obtained.

[0055] Therefore, the fish back burr trimming trajectory is cut from B″-C' along the f3 curve.

[0056] Fish belly trimming track:

[0057] In the prior art, after tilapia is slaughtered and sliced, the fins of the tilapia are light black and the belly is white. The light black of the fins, the white of the belly and the light red of the fish meat form an obvious color difference. When in use, the fins and belly in the image are visually identified by the recognition device, and after the image is converted to the HSV color space, the colors of the fins and belly are separated, and only the pixels located in the upper half of the rectangular frame outside the fish body are retained to prevent other debris in the background from interfering with the recognition result.

[0058] Extract all points of the outer contour of the fish belly and get the point with the smallest x coordinate value, which is recorded as D.

[0059] Set the first cutting path of the fish belly as f4, and determine whether the outer contour points of all the fins are located on f4. If so, f4 is the first cutting path of the fish belly, and the cutting start point of f4 is C' and the cutting end point is D. If not, the point with the largest y value in the outer contour of the fin is recorded as E, and the first cutting path is C'-ED.

[0060] Set the point with the largest x-coordinate value in the outer contour of the fish belly as F.

[0061] Set the second cutting path of the fish belly to f5, the cutting starting point of f5 to point D, and the cutting end point to point F.

[0062] Extract all points between points D and F in the outer contour, that is, the x-coordinate axis is between points D and F, and the y-coordinate is smaller than the larger one of points D and F. Fit the extracted points to the fish belly curve, and set the equation f5 = g0 + g1*x + g2*x 2 +g3*x 3 , and use the least squares method to find the coefficients.

[0063] Therefore, the belly trimming trajectory is from C'-EDF, and trimming is performed along the f4 and f5 cutting curves.

[0064] Trimming the side of the fish head and removing the spine V-shaped track:

[0065] The traditional method is difficult to identify the myofascial stripes and vertebral imprints in fish meat, and the accuracy is low, so deep learning method is used for identification. First, a data set is established, and the myofascial stripes and vertebral imprints are manually annotated in the collected fish fillet images, and an annotation file is generated. The data set is divided into training set, validation set and test set in a ratio of 7:2:1. The training set and validation set are put into the deep learning model for training, and the weight file obtained by training is verified on the test set. When the accuracy rate on the training set reaches more than 95%, the model is considered to be effective. Otherwise, adjust the parameters and retrain.

[0066] The trained deep learning model is used to identify the myofascial stripes and vertebral imprints of the fish fillet, and the identification results are processed to obtain the point with the largest x-coordinate in the vertebral imprint, denoted as N; the intersection of the 12th myofascial stripe and the vertebral imprint is denoted as M.

[0067] Calculate the distance from point N to point M. If the distance is less than 40% of the total length of the fillet, the coordinates of the two points remain unchanged. If the distance is greater than 40% of the total length of the fillet, point M moves closer to point N along the spine until the distance between the two points is less than or equal to 40% of the total length of the fillet. The total length of the fillet is equal to the long side of the minimum circumscribed rectangle of the fillet, i.e. Figure 1 The Euclidean distance between bb2 and bb3 in the β rectangular box.

[0068] Calculate the equation of the straight line passing through points F and N, record it as f6. Set the spine V-mouth trimming factor to e w %, based on the V-shaped trimming factor e of the spine w %, and obtain the trimming points N1 and N2 on both sides of the spine.

[0069] N1 is on the curve of f6, and the distance from N1 to N is e of the total width of the fish body. w %. The total width of the fillet is equal to the short side of the minimum circumscribed rectangle of the fillet, that is, Figure 1 The Euclidean distance between bb1 and bb2 in the β rectangular box. w % is the V-mouth trimming factor, which can be assigned different values ​​according to actual trimming requirements. w The trimming range is 8%-15%. In order to ensure the removal of intermuscular thorns, a V-end trimming factor of 15% is given, and in order to increase the meat yield, a V-end trimming factor of 8% is given.

[0070] Calculate the equation of the line passing through points N1 and M, denoted as f7;

[0071] Calculate A ′ The equation of the straight line between point A and point N is denoted as f8.

[0072] N2 is on the curve of f8, and the distance from N2 to N is calculated as e of the total width of the fish body. w %. Calculate the straight line equation between N2 and M and record it as f9.

[0073] During the trimming process, the fish body has uneven parts that are not trimmed properly. Therefore, the head trimming factor is set to e h mm, move point N1 along f7 to point M by e h mm. Similarly, N2 moves along f9 to point M by e h mm. Calculate the equation of the straight line connecting point F and N1, record it as f6 ′ , and connect A ′ The equation of the line between point and N2 is denoted as f8 ′ .

[0074] Therefore, the cutting path at the fish head is the first segment along f6 ′ The curve starts at point F and ends at point N1;

[0075] The second segment follows the f7 curve, starting at point N1 and ending at point M;

[0076] The third segment follows the f9 curve, with the starting point being point M and the end point being point N2;

[0077] The fourth section is along f8 ′ The curve starts at point N2 and ends at point A. ′ point;

[0078] The cutting path of the whole fish body is: A ′ -B″-C'-DEF-N1-M-N2-A ′ .

[0079] like Figure 2 In actual operation, there are cases where the fish back and belly are missing, which affects the trimming of the fish body. When the gap is at the back of the fish, set the deepest point of the gap as Q b The original route was A ′ f2 curve to B″. Now calculate the x-axis coordinate as Q b and A ′ The midpoint of the f2 curve is Q1, and its x-axis coordinate is Q b The point on the f2 curve that is the midpoint of Q1 and B″ is denoted as Q2. b , Q2 three-point fitting curve f q =q0+q1*x+q2*x 2 +q3*x 3 The trimming path is A ′ From point Q1 to point Q1, the trimming path is f2; from point Q1 to point Q2, the trimming path is f q , and through Q b Point. From point Q2 to point B″, the path is f2.

[0080] When the notch is located at other positions, the same trimming method is used, taking the trimming path from C' to D at the fish belly as an example. If there is a notch at the fish belly, the deepest point is recorded as Q g The x-axis coordinate is Q g The point on the C'D straight line is Q3. The x-axis coordinate is Q g The point on the C'D straight line that is the midpoint of D is denoted as Q4. g , Q4 three-point fitting curve f g =g0+g1*x+g2*x 2 +g3*x 3 . The trimming path is C ′ From point Q3 to point Q4, the trimming path is f g , and through Q g point, Q4 to point D.

[0081]

[0082] W is the weight of the fish fillet.

[0083] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for generating a trimming trajectory of tilapia fillets based on image recognition, characterized in that: The following steps are involved: S1: Process the captured fish fillet image to obtain the outer contour of the fish fillet and establish a camera geometric coordinate system model; S2: Set the maximum x-coordinate value of the fish body to A and the minimum x-coordinate to B, extract all points between point A and point B in the outer contour, and the y-coordinate is less than the larger one of point A and point B, and obtain the fish back burr trimming trajectory; S3: Calculate the minimum circumscribed rectangle of the fish body through the outer contour of the fish body, set the two vertices with smaller x-axis coordinates among the four vertices of the minimum circumscribed rectangle as bb1 and bb2 respectively, calculate the slope of the straight line passing through bb1 and bb2, record it as k, and calculate the cutting trajectory at the fish tail through the coordinates of point B and the k value; S4: Identify the fish fins and fish belly in the image, extract all points of the fish belly outer contour, set the point of the fish belly at the minimum X coordinate as D, the point of the maximum X coordinate as F, set the point of the fish fin at the minimum X coordinate as C, and connect C, D, and F to form a fish belly trimming trajectory; S5: Identify the vertebral imprint in the fish fillet image, designate the point with the maximum value of the vertebral bone in the x-coordinate as N, and the intersection of the twelfth diaphragm stripe and the vertebral imprint as M; Set the spine V-mouth trim factor to e w %, and according to the V-mouth trimming factor, the trimming points N1 and N2 on both sides of the spine are obtained. F, N1, M, N2, and A are connected to form the V-mouth trajectory for trimming the fish head side and removing the spine.

2. A method for generating a tilapia fillet trimming trajectory based on image recognition according to claim 1, characterized in that: In step S2, the fish back rough edge trimming trajectory is set to f1, the extracted coordinate points are converted into the camera geometric coordinate system, and the fish back curve is fitted, and the equation f1 = a0 + a1*x + a2*x is set 2 +a3*x 3 , and use the least squares method to find the coefficients.

3. A method for generating a tilapia fillet trimming trajectory based on image recognition according to claim 1, characterized in that: In step S3, the trimming factor of the fishtail back is set as e1, and a point on the f1 curve is set as B ′ , B ′ The x coordinate of A = the x coordinate of B + e1.

4. A method for generating a tilapia fillet trimming trajectory based on image recognition according to claim 1, characterized in that: In step S2, the back trimming factor is set to e b , set the cutting path of the fish back to f2, and set the equation f2 = f1 + e b , the cutting starting point of f2 is A ′ , set equation A ′ =A+e b , the cutting end point is B″, the x coordinate of B″ = B ′ x coordinate of +e b .

5. A method for generating a tilapia fillet trimming trajectory based on image recognition according to claim 1, characterized in that: In step S3, the cutting path f3 of the fishtail is calculated according to the coordinates of B″ and the k value. The intersection point of the cutting path f3 of the fishtail and the outer contour of the fish is C. The cutting end point of the cutting path f3 is set to C', and the trimming factor of the fishtail is set to e t , C' y coordinate = C y coordinate - e t .

6. A method for generating a tilapia fillet trimming trajectory based on image recognition according to claim 1, characterized in that: In step S4, the cutting path of the first section of the fish belly is set to f4, and the point with the minimum y value in the outer contour of the fish fin is set to E, C, E, and D as the cutting path of f4.

7. A method for generating a tilapia fillet trimming trajectory based on image recognition according to claim 1, characterized in that: In step S4, the cutting path of the second section of the fish belly is set to f5, and all points between points D and F in the outer contour are extracted, that is, the x coordinate is between points D and F, and the y coordinate is smaller than the larger one of points D and F. The extracted points are fitted to the fish belly curve, and the equation f5=g0+g1*x+g2*x is set. 2 +g3*x 3 , the coefficients are obtained by the least square method. f5 is the second cutting curve of the fish belly, with the starting point at point D and the end point at point F.

8. A method for generating a tilapia fillet trimming trajectory based on image recognition according to claim 1, characterized in that: In step S5, the fish head cutting path is set to The first segment starts at point F and ends at point N1; The second segment starts at point N1 and ends at point M; The starting point of the third segment is point M and the end point is point N2; The fourth segment starts at point N2 and ends at point A. ′ point; The cutting path of the whole fish body is: A ′ -B″-C'-DEF-N1-M-N2-A ′ .

9. A method for generating a tilapia fillet trimming trajectory based on image recognition according to claim 1, characterized in that: In step S2, there is a gap at the fish back, and the deepest point of the gap is set as Q b , set Q b Points Q1 and Q2 on both sides, Q1 and Q2 are on the cutting path f2, Q1, Q b , Q2 three-point fitting curve f q =q0+q1*x+q2*x 2 +q3*x 3 , the trimming path is A ′ -Q1, Q1-Q2, Q2-B″.

Images