A method for identifying the placement position of tilapia fillets based on image recognition

Abstract

The present invention relates to a method for identifying the placement position of tilapia fillets based on image recognition, and relates to the field of intelligent aquatic product processing. The present invention uses image recognition to rapidly identify information such as the thickness distribution, width distribution, and meat color distribution of tilapia fillets, thereby matching the placement position of the tilapia. This application improves the efficiency of fillet processing and can quickly identify the placement position of the fillets.

Description

Technical Field

[0001] The present invention relates to the field of intelligent processing of aquatic products, and in particular to a method for identifying the placement position of tilapia fillets based on image recognition. Background Art

[0002] Tilapia is a small to medium-sized tropical fish native to Africa. Since its introduction to China, its aquaculture and processing industries have developed rapidly. Besides being eaten fresh, tilapia is often processed and sold as fillets.

[0003] In the existing technology, the tilapia fillet processing process includes slaughtering, bleeding, filleting, skinning, trimming, sterilization and freezing. During the filleting process, a whole tilapia is generally removed from its head, tail, fins, scales and other landmarks, and the tilapia is cut into two mirror-image fillets. When the fillets need to be trimmed, workers are generally required to first visually determine the position of the tail, head and belly on the fillets, and then select different trimming solutions based on the different parts.

[0004] It takes a long time to manually visually identify the placement of fish fillets. When the number of fish fillets is large, it is easy to lead to a decrease in the processing efficiency of the fish fillets. Summary of the Invention

[0005] In order to improve the efficiency of fish fillet processing and quickly identify the placement position of fish fillets, the present invention provides a method for identifying the placement position of tilapia fillets based on image recognition.

[0006] A method for identifying the placement position of tilapia fillets based on image recognition, characterized by comprising:

[0007] Get images of tilapia fillets;

[0008] Matching a pre-processed image from a preset image processing method according to the fish fillet image;

[0009] Match the transformed image from the preset HSV image transformation method according to the preprocessed image;

[0010] Extracting background points using a preset background point extraction method according to the transformed image;

[0011] Matching a background threshold value using a preset background threshold matching method according to the background points;

[0012] Matching a binary image from a preset binarization processing method according to the transformed image and the background threshold;

[0013] Matching a binary mask from a preset binary mask processing method according to the binary image;

[0014] Matching a separation image from a preset image separation method according to the binary mask and the fish fillet image;

[0015] Match the changed image from the preset HSV image conversion method according to the separated image;

[0016] Separate the green channel based on the change image;

[0017] Match the number of edge pixels using the preset edge detection method based on the green channel;

[0018] Calculating the number of extracted pixels from a preset extracted pixel identification method according to the binary mask;

[0019] The quotient of the number of edge pixels and the number of extracted pixels was calculated and defined as the fillet pixel ratio;

[0020] When the pixel ratio of the fish fillet exceeds a preset inner pixel interval, controlling a preset flipping device to flip the fish fillet so that the inner side of the fish fillet faces upward;

[0021] Obtain point cloud data of fish fillet outline;

[0022] According to the point cloud data, the center of mass and main direction of the fish fillet are matched using the preset principal component analysis method;

[0023] Match the upper point cloud and the lower point cloud from the preset ventral and dorsal point cloud segmentation method based on the fish fillet centroid, point cloud data and the main direction of the fish fillet;

[0024] Calculating the upper thickness using a preset average thickness calculation method based on the upper point cloud, and calculating the lower thickness using a preset average thickness calculation method based on the lower point cloud;

[0025] When the thickness of the upper portion is greater than the thickness of the lower portion, the fish belly feature is identified using a preset fish belly feature recognition method based on the lower point cloud and the fish fillet image;

[0026] When the thickness of the lower portion is greater than the thickness of the upper portion, the fish belly feature is identified using a preset fish belly feature recognition method based on the upper point cloud and the fish fillet image;

[0027] The fish belly contour is matched according to the fish belly features using a preset fish belly contour matching method.

[0028] By adopting the above technical solution, the background color in the fish fillet image is extracted to form a background threshold, so that the fish fillet is separated from the background according to the background threshold, and the canny operator is used to judge the placement state of the fish fillet based on the difference in fish meat color inside and outside the fish fillet, thereby keeping the inside of the fish fillet facing up, improving the convenience of subsequent processing of the fish fillet, and analyzing the head and tail orientation and center of mass position of the fish fillet from the point cloud data, so that the fish fillet is divided into two parts according to the head and tail orientation of the fish fillet, and the part where the abdomen is located is judged by comparing the average thickness of the two parts of the fish fillet, so as to identify the abdominal area on the fish fillet, and then judge the placement state of the fish fillet according to the abdominal area, thereby improving the efficiency of fish fillet processing.

[0029] Optionally, the average thickness calculation method includes:

[0030] Where D is the required average thickness value;

[0031] n is the total number of points contained in the point cloud;

[0032] Z is the Z-axis coordinate value of each point in the point cloud data.

[0033] Optionally, also include:

[0034] Calculating a thickness difference according to the upper thickness and the lower thickness using a preset difference calculation method;

[0035] When the thickness difference is not greater than the preset ventral-dorsal difference, the upper standard deviation is calculated from the upper point cloud using the preset thickness standard deviation calculation method;

[0036] Calculate the lower standard deviation from the preset thickness standard deviation calculation method based on the lower point cloud;

[0037] When the upper standard deviation is greater than the lower standard deviation, the fish belly feature is identified from a preset fish belly feature recognition method based on the upper point cloud and the fish fillet image;

[0038] When the lower standard deviation is greater than the upper standard deviation, the fish belly feature is identified from a preset fish belly feature recognition method based on the lower point cloud and the fish fillet image.

[0039] By adopting the above technical solution, when the average thickness difference between the two parts is not large, it is not easy to directly judge the part where the abdomen is located by the thickness. At this time, the thickness difference in the part is displayed by the standard deviation of the two parts, and the part with a larger difference is judged as the abdomen, so that the placement position of the fish fillet can be judged according to the abdominal area.

[0040] Optionally, the thickness standard deviation calculation method includes:

[0041] Where σ is the required thickness standard deviation;

[0042] n is the total number of points contained in the point cloud;

[0043] Z is the Z-axis coordinate value of each point in the point cloud data;

[0044] D is the average thickness value corresponding to the point cloud.

[0045] Optionally, a method for identifying the head and tail of a fish fillet is further included, and the method for identifying the head and tail of a fish fillet comprises:

[0046] Match the left point cloud and the right point cloud from the preset head and tail point cloud segmentation method based on the fish fillet centroid, point cloud data and main direction of the fish fillet;

[0047] Calculate the left width using the preset average width calculation method based on the main direction of the fish fillet and the left point cloud;

[0048] The right width is calculated using a preset average width calculation method according to the main direction of the fillet and the right point cloud;

[0049] When the left width is greater than the right width, the fish head feature is identified using a preset fish head feature recognition method based on the left point cloud and the fish fillet image;

[0050] When the right width is greater than the left width, the fish head feature is identified from a preset fish head feature recognition method according to the right point cloud and the fish fillet image.

[0051] By adopting the above technical solution, the fish fillet is divided into two parts according to the center of mass and the direction perpendicular to the head and tail of the fish fillet, and the average width of the two parts of the fish fillet is calculated. The wider part is judged to be the part where the head of the fish fillet is located based on the average width, and then the position of the fish head is identified, and the placement state of the fish fillet is judged based on the position of the fish head.

[0052] Optionally, the fish fillet head and tail identification method further includes:

[0053] Matching the fillet length using a preset fillet length recognition method based on the fillet main direction and point cloud data;

[0054] When the length of the fish fillet exceeds the preset length calculation interval, the left length is matched from the preset fish fillet length recognition method according to the main direction of the fish fillet and the left point cloud;

[0055] According to the main direction of the fish fillet and the right point cloud, the right length is matched from the preset fish fillet length recognition method;

[0056] Calculate the length difference according to the left length and the right length using a preset difference calculation method;

[0057] Match the left cut point cloud from the preset point cloud cut method according to the length difference, the main direction of the fish fillet and the left point cloud;

[0058] The left width is calculated using the preset average width calculation method according to the main direction of the fish fillet and the left interception point cloud;

[0059] Match the right cut point cloud from the preset point cloud cut method according to the length difference, the main direction of the fish fillet and the right point cloud;

[0060] The left width is calculated from the preset average width calculation method according to the main direction of the fish fillet and the right intercept point cloud.

[0061] By adopting the above technical solution, when the fish fillet is long, the amount of calculation required to calculate the average width of the fish fillet is large. At this time, a segment of equal relative length is cut from each of the two parts of the fish fillet, and the average width of the segment is used as the average width of the fish fillet part, thereby reducing the amount of calculation required to calculate the average width.

[0062] Optionally, the fish fillet head and tail identification method further includes:

[0063] Calculating a width ratio using a preset width ratio calculation method according to the left width and the right width;

[0064] When the width ratio is less than the preset head-to-tail ratio, the upper left point cloud and the lower left point cloud are matched from the preset ventral and dorsal point cloud segmentation method based on the centroid of the fish fillet, the left point cloud and the main direction of the fish fillet;

[0065] Calculate the upper left thickness using a preset average thickness calculation method based on the upper left point cloud, and calculate the lower left thickness using a preset average thickness calculation method based on the lower left point cloud;

[0066] Based on the centroid of the fish fillet, the right point cloud and the main direction of the fish fillet, the upper right point cloud and the lower right point cloud are matched from the preset ventral and dorsal point cloud segmentation method;

[0067] The upper right thickness is calculated from the preset average thickness calculation method based on the upper right point cloud, and the lower right thickness is calculated from the preset average thickness calculation method based on the lower right point cloud;

[0068] Matching the minimum thickness from a preset minimum thickness matching method according to the upper left thickness, the lower left thickness, the upper right thickness, and the lower right thickness;

[0069] Match the fish head point cloud from the preset point cloud database based on the minimum thickness;

[0070] The fish head features are identified from the preset fish head feature recognition method based on the fish head point cloud and the fish fillet image.

[0071] By adopting the above technical solution, when the widths of the two parts are similar, it is not easy to determine the position of the fish head. At this time, the fish fillet particle is used as the origin, and the fish fillet is divided into four parts along the main direction of the fish fillet and two cutting lines perpendicular to the main direction of the fish fillet. The average thickness of the four parts is calculated respectively, so as to determine the position of the fish belly and then the position of the fish head.

[0072] Optionally, a method for adjusting the head and tail of a fish fillet is further included, and the method for adjusting the head and tail of a fish fillet comprises:

[0073] Matching the fish head orientation based on the fish head features using a preset fish head orientation matching method;

[0074] Matching an adjustment angle from a preset adjustment angle matching method according to the fish head orientation and the preset fish fillet processing direction;

[0075] Matching an adjustment stroke from a preset adjustment stroke matching method according to the centroid of the fish fillet and the adjustment angle;

[0076] According to the adjustment stroke, the preset adjustment device is controlled to turn the fish head toward the fish fillet processing direction.

[0077] By adopting the above technical solution, when the head direction of the fish fillet deviates from the processing direction of the fish fillet, the head direction of the fish fillet is adjusted by the adjustment device, so that the fish fillet is arranged in a body shape with the head direction consistent with the processing direction of the fish fillet, thereby improving the convenience of subsequent processing of the fish fillet.

[0078] In summary, this application includes at least one of the following beneficial technical effects:

[0079] 1. By analyzing the head-tail orientation and center of mass of the fish fillet from the point cloud data, the fillet is divided into two parts according to the head-tail orientation. The average thickness of the two parts is compared to determine the location of the belly, thereby identifying the belly area on the fillet. The placement of the fillet is then determined based on the belly area, improving the efficiency of fish fillet processing.

[0080] 2. When the average thickness difference between the two parts of a fish fillet is small, it is difficult to directly determine the belly portion based on the thickness. In this case, the standard deviation of the two parts is used to show the thickness difference between the parts. The part with the larger thickness difference is identified as the belly, and the placement of the fish fillet is determined based on the belly area.

[0081] 3. Divide the fish fillet into two parts according to the center of mass and the direction perpendicular to the head and tail of the fish fillet, and calculate the average width of the two parts. Based on the average width, the wider part is determined to be the part where the head of the fish fillet is located. Then, the position of the fish head is identified, and the placement status of the fish fillet is determined based on the position of the fish head. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 This is a process of identifying the placement status of tilapia fillets based on image recognition Figure 1 ;

[0083] Figure 2 This is a process of identifying the placement status of tilapia fillets based on image recognition Figure 2 ;

[0084] Figure 3 This is a process of identifying the placement status of tilapia fillets based on image recognition Figure 3 ;

[0085] Figure 4 This is a process of identifying the placement status of tilapia fillets based on image recognition Figure 4 ;

[0086] Figure 5 This is the process of identifying the head and tail of fish fillets Figure 1 ;

[0087] Figure 6 This is the process of identifying the head and tail of fish fillets Figure 2 ;

[0088] Figure 7 This is the process of identifying the head and tail of fish fillets Figure 3 ;

[0089] Figure 8 It is a flow chart of the method for adjusting the head and tail of fish fillet;

[0090] Figure 9 is a flow chart of the fish fillet separation method. DETAILED DESCRIPTION

[0091] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0092] An embodiment of the present invention discloses a method for identifying the placement position of tilapia fillets based on image recognition. The present invention uses image recognition to quickly identify information such as the thickness distribution, width distribution, and meat color distribution of tilapia fillets, thereby matching the placement position of the tilapia.

[0093] Reference Figure 1 A method for identifying the placement position of tilapia fillets based on image recognition comprises the following steps:

[0094] Step 100: Acquire a tilapia fillet image.

[0095] The fish fillet image refers to a color picture of tilapia. The fish fillet image can be obtained through a camera. The method for obtaining the fish fillet image is selected by the staff according to the actual situation and will not be described in detail here.

[0096] Step 101: matching a pre-processed image from a preset image processing method according to the fish fillet image.

[0097] The image processing method refers to a method of obtaining a pre-processed image by pre-processing the fish fillet image through Gaussian filtering and noise reduction. The image processing method is selected by the staff according to the actual situation and will not be described in detail here.

[0098] Step 102: Match a converted image using a preset HSV image conversion method according to the preprocessed image.

[0099] The HSV image conversion method refers to a method of converting a preprocessed image in RGB format into a converted image in HSV format. The HSV image conversion method is selected by the staff according to the actual situation and will not be described in detail here.

[0100] Step 103: extracting background points from the transformed image using a preset background point extraction method.

[0101] The background point extraction method refers to extracting small squares of 25*25 pixels in the upper left corner, lower left corner, upper right corner and lower right corner of the fish fillet image as background points. The background point extraction method is selected by the staff according to the actual situation and will not be described here.

[0102] Step 104: Matching a background threshold value according to the background points using a preset background threshold matching method.

[0103] The background threshold matching method can calculate the maximum and minimum values ​​of the three channels H, S and V in the four small squares respectively, and then calculate the average of the maximum and minimum values ​​of the three channels H, S and V in the four small squares, and use the average of the minimum values ​​of the three channels H, S and V as the lower limit of the background threshold, and use the average of the maximum values ​​of the three channels H, S and V as the upper limit of the background threshold to form the background threshold. The background threshold matching method is selected by the staff according to actual conditions and will not be elaborated here.

[0104] Step 105: A binary image is matched from a preset binarization processing method according to the converted image and the background threshold.

[0105] The binarization processing method can set the pixel values ​​in the converted image within the background threshold range to [255, 255, 255], and the pixel values ​​outside the background threshold range to [0, 0, 0] (that is, the background is black and the fish fillets are white) to form a binary image. The binarization processing method is selected by the staff according to the actual situation and will not be elaborated here.

[0106] Step 106: Match a binary mask according to the binary image using a preset binary mask processing method.

[0107] The binary mask processing method can obtain a binary mask after opening and closing the binary image and performing noise removal operations. The binary mask processing method is selected by the staff according to the actual situation and will not be described here.

[0108] Step 107: Matching a separated image using a preset image separation method according to the binary mask and the fish fillet image.

[0109] The image separation method can perform bitwise AND operation on the fish fillet image and the binary mask to obtain a separation image in which the fish fillet in the fish fillet image is separated from the background.

[0110] Reference Figure 2 A method for identifying the placement position of tilapia fillets based on image recognition further includes the following steps:

[0111] Step 108: Match a changed image according to the separated image using a preset HSV image conversion method.

[0112] The changed image is an HSV image obtained by converting the RGB separated image through the HSV image conversion method.

[0113] Step 109: Separate the green channel based on the changed image.

[0114] The green channel is an image formed by separating the green area from the HSV image. The outer side of the fish fillet contains dark meat, which is more obvious under the green channel, and more edges can be detected. The green channel separation method is selected by the staff based on the actual situation and is not detailed here.

[0115] Step 110: Match the number of edge pixels using a preset edge detection method according to the green channel.

[0116] The edge detection method can count the number of detected edge pixels after the change image under the green channel is detected by the Canny operator. The edge detection method is selected by the staff according to the actual situation and will not be described here.

[0117] Step 111: Calculate the number of extracted pixels using a preset extraction pixel identification method according to the binary mask.

[0118] The extracted pixel recognition method can match the total number of pixels in the white part (i.e., the fish fillet part) in the binary mask. The number of extracted pixels is the result obtained by matching through the extracted pixel recognition method. The extracted pixel recognition method is selected by the staff according to the actual situation and will not be described here.

[0119] Step 112: Calculate the quotient of the number of edge pixels and the number of extracted pixels, and define it as the fillet pixel ratio.

[0120] The fillet pixel ratio is obtained by scaling the number of edge pixels by dividing the number of extracted pixels.

[0121] Step 113: When the pixel ratio of the fish fillet exceeds the preset inner pixel interval, controlling a preset flipping device to flip the fish fillet so that the inner side of the fish fillet faces upward.

[0122] The inner pixel range refers to the preset maximum pixel ratio within the fish fillet. This range is selected by staff based on actual conditions and is not detailed here. The flipping device is used to flip the fish fillet from the outside to the inside. If the pixel ratio of the fish fillet exceeds the inner pixel range, it indicates that the fish fillet in the fish fillet image is on the outside. In this case, the flipping device adjusts the fish fillet's position to facilitate subsequent processing.

[0123] Reference Figure 3 A method for identifying the placement position of tilapia fillets based on image recognition further includes the following steps:

[0124] Step 114: Obtain point cloud data of the fish fillet outline.

[0125] Point cloud data refers to the spatial position data set of the points of the outline of the fish fillet. The point cloud data can be obtained by a laser scanner. The method of obtaining the point cloud data is selected by the staff according to the actual situation and will not be described here.

[0126] Step 115: Match the fish fillet's centroid and main direction using a preset principal component analysis method based on the point cloud data.

[0127] Principal component analysis is a statistical method that can find the main direction of data change in point cloud data, that is, the main direction of the fish fillet, and identify the center of mass in the point cloud data as the center of mass of the fish fillet. After matching the center of mass of the fish fillet, the center of mass of the fish fillet is used as the origin, and the direction of the fish fillet away from the background is used as the positive direction of the z-axis, where the background refers to the platform on which the fish fillet is placed. The background is generally a conveyor belt. The fish fillet processing direction is used as the positive direction of the y-axis, and the scanning direction of the laser scanner is used as the positive direction of the x-axis to establish a coordinate system. The point cloud data is substituted into the coordinate system to obtain the coordinate position of the point of the fish fillet's outline in the coordinate system, and the point cloud data is updated to the coordinate position. The fish fillet processing direction refers to the direction in which the fish fillet moves on the processing line. The fish fillet processing direction is selected by the staff according to actual conditions and will not be elaborated here.

[0128] Step 116: Match the upper point cloud and the lower point cloud from the preset ventral and dorsal point cloud segmentation method based on the centroid of the fish fillet, the point cloud data, and the main direction of the fish fillet.

[0129] The ventral and dorsal point cloud segmentation method can separate the point cloud data according to the dividing line. The upper point cloud and the lower point cloud are the two sets of coordinate point data obtained by dividing the point cloud data with a dividing line passing through the centroid of the fish fillet and facing the main direction of the fish fillet.

[0130] Step 117: Calculate the upper thickness using a preset average thickness calculation method based on the upper point cloud, and calculate the lower thickness using a preset average thickness calculation method based on the lower point cloud.

[0131] The average thickness calculation method can calculate the average thickness of the point cloud data based on the z-axis coordinate value of the coordinates in the point cloud data. The upper thickness is the thickness result calculated by the average thickness calculation method for the upper point cloud, and the lower thickness is the thickness result calculated by the average thickness calculation method for the lower point cloud.

[0132] The average thickness is calculated using the formula Where D is the required average thickness value, n is the total number of points contained in the point cloud, and Z is the Z-axis coordinate value of each point in the point cloud data.

[0133] Step 118: When the upper thickness is greater than the lower thickness, the fish belly feature is identified using a preset fish belly feature recognition method based on the lower point cloud and the fish fillet image.

[0134] The average thickness of a fish fillet's ventral side is generally thinner than its dorsal side. A greater thickness at the top indicates that the top point cloud corresponds to the dorsal side of the fillet, while the bottom point cloud corresponds to the ventral side. The belly feature recognition method can identify belly features based on color difference within the region where the bottom point cloud is located, matching the region on the fillet image.

[0135] Step 119: When the thickness of the lower portion is greater than the thickness of the upper portion, the fish belly feature is identified using a preset fish belly feature recognition method based on the upper point cloud and the fish fillet image.

[0136] The thickness of the lower part is greater than that of the upper part, indicating that the lower point cloud corresponds to the dorsal side of the fish fillet, while the upper point cloud corresponds to the ventral side. The belly feature recognition method is used to match the upper point cloud to the area in the fish fillet image, and then the belly features are identified based on color difference in the area where the upper point cloud is located.

[0137] Step 120: Match the fish belly contour according to the fish belly features using a preset fish belly contour matching method.

[0138] The fish belly contour matching method refers to using the color of the fish belly feature as an indicator to identify the range of the fish belly contour according to the color difference.

[0139] Reference Figure 4 A method for identifying the placement position of tilapia fillets based on image recognition further includes the following steps:

[0140] Step 121: Calculate the thickness difference according to the upper thickness and the lower thickness using a preset difference calculation method.

[0141] The difference calculation method is a method of calculating the thickness difference by subtracting the smaller thickness from the larger thickness of the upper portion and the lower portion.

[0142] Step 122: When the thickness difference is not greater than the preset ventral-dorsal difference, the upper standard deviation is calculated according to the upper point cloud using a preset thickness standard deviation calculation method.

[0143] The "ventral-dorsal difference" is a preset minimum thickness difference between the ventral and dorsal sides of a fillet. This value can be set to one-tenth of the larger of the upper or lower thicknesses. This value is determined by the staff based on actual conditions and is not detailed here. If the thickness difference is not greater than the "ventral-dorsal difference," it means that the ventral and dorsal sides of the fillet cannot be directly determined.

[0144] Thickness standard deviation calculation method refers to the calculation method used to calculate thickness standard deviation. The thickness standard deviation calculation method generally uses the formula Where σ is the required thickness standard deviation, n is the total number of points in the point cloud, Z is the Z-axis coordinate value of each point in the point cloud data, D is the average thickness value corresponding to the point cloud, and the upper standard deviation is the standard deviation result calculated by the thickness standard deviation calculation method of the upper point cloud.

[0145] Step 123: Calculate the lower standard deviation according to the lower point cloud using a preset thickness standard deviation calculation method.

[0146] The lower standard deviation is the standard deviation result calculated by the thickness standard deviation calculation method of the lower point cloud.

[0147] Step 124: When the upper standard deviation is greater than the lower standard deviation, the fish belly feature is identified using a preset fish belly feature recognition method based on the upper point cloud and the fish fillet image.

[0148] The thickness difference on the ventral side of the fish fillet is large. The upper standard deviation is greater than the lower standard deviation, which means that the thickness difference of the upper point cloud is large, that is, the upper point cloud corresponds to the ventral side of the fish fillet.

[0149] Step 125: When the lower standard deviation is greater than the upper standard deviation, the fish belly feature is identified using a preset fish belly feature recognition method based on the lower point cloud and the fish fillet image.

[0150] The lower standard deviation is greater than the upper standard deviation, which means that the thickness difference of the lower point cloud is larger, that is, the lower point cloud corresponds to the ventral side of the fish fillet.

[0151] Reference Figure 5 The method for identifying the head and tail of a fish fillet comprises the following steps:

[0152] Step 200: Match the left point cloud and the right point cloud from a preset head and tail point cloud segmentation method based on the fish fillet centroid, point cloud data and the main direction of the fish fillet.

[0153] The head and tail point cloud segmentation method can separate the point cloud data according to the dividing line. The left point cloud and the right point cloud are two sets of coordinate point data obtained by dividing the point cloud data with a dividing line passing through the centroid of the fish fillet and perpendicular to the main direction of the fish fillet.

[0154] Step 201: Calculate the left width using a preset average width calculation method according to the main direction of the fish fillet and the left point cloud.

[0155] The average width calculation method can calculate the average width of the point cloud data in the main direction of the fish fillet based on the coordinate values ​​in the point cloud data. The left width is the width result obtained by calculating the left point cloud using the average width calculation method. The average width calculation method is selected by the staff based on actual conditions and will not be elaborated here.

[0156] Step 202: Calculate the right width using a preset average width calculation method according to the main direction of the fish fillet and the right point cloud.

[0157] The right width is the width result calculated by the average width calculation method of the right point cloud.

[0158] Step 203: When the left width is greater than the right width, the fish head feature is identified using a preset fish head feature recognition method based on the left point cloud and the fish fillet image.

[0159] The head of a fish fillet is generally wider than the tail. A larger left-hand width than a right-hand width indicates that the left point cloud corresponds to the head of the fish fillet. The fish head feature recognition method matches the left point cloud to the area on the fish fillet image and then uses the outline of the area within the left point cloud as the fish head feature.

[0160] Step 204: When the right width is greater than the left width, the fish head feature is identified using a preset fish head feature recognition method based on the right point cloud and the fish fillet image.

[0161] The right width is greater than the left width, which means that the right point cloud corresponds to the head side of the fish fillet. The fish head feature is the prominent point identified in the corresponding area of ​​the right point cloud on the fish fillet image by the fish head feature recognition method.

[0162] Reference Figure 6 The fish fillet head and tail identification method further comprises the following steps:

[0163] Step 205: Match the length of the fish fillet using a preset fish fillet length recognition method according to the main direction of the fish fillet and the point cloud data.

[0164] The fish fillet length recognition method can calculate the maximum length of the point cloud data in the main direction of the fish fillet according to the coordinate values ​​of the point cloud data, and define the maximum length as the fish fillet length.

[0165] Step 206: When the length of the fish fillet exceeds the preset length calculation interval, the left length is matched from a preset fish fillet length recognition method according to the main direction of the fish fillet and the left point cloud.

[0166] The length calculation interval refers to the fillet length range where width calculations require less effort. Fillet lengths outside this range indicate that calculating the average width requires more effort and takes longer. The length calculation interval is selected by staff based on actual conditions and is not detailed here. The left length is the maximum length of the left point cloud in the main direction of the fillet, as found using the fillet length recognition method.

[0167] Step 207: Match the right length of the fish fillet using a preset fish fillet length recognition method according to the main direction of the fish fillet and the right point cloud.

[0168] The right length is the maximum length of the right point cloud matched by the fillet length recognition method in the main direction of the fillet.

[0169] Step 208: Calculate the length difference according to the left portion length and the right portion length using a preset difference calculation method.

[0170] The length difference is the difference between the larger of the left length and the right length calculated by the difference calculation method and the smaller of the larger of the left length and the right length.

[0171] Step 209: Match the left-cut point cloud from a preset point cloud interception method according to the length difference, the main direction of the fish fillet and the left point cloud.

[0172] The point cloud interception method can intercept the coordinate data of a part of the point cloud data. The main direction of the fish fillet is generally the same as the positive direction of the y-axis of the coordinate system. The length difference is defined as L, and the minimum y-axis value of the point in the left point cloud is defined as y - max , define the y-axis maximum value of the point in the right point cloud as y + max , L=y + max +y - max , generally the y-axis coordinates of the left point cloud are within the range (y - max +0.05L,y - max The coordinate point set within +0.15L) is intercepted as the left intercept point cloud.

[0173] Step 210: Calculate the left width using a preset average width calculation method according to the main direction of the fish fillet and the left cut point cloud.

[0174] The left width is the average width of the left-cut point cloud in the main direction of the fillet calculated by the average width calculation method.

[0175] Step 211: Match the right-cut point cloud from a preset point cloud interception method according to the length difference, the main direction of the fish fillet and the right point cloud.

[0176] The right intercept point cloud is the point cloud on the right intercepted by the point cloud interception method with the y-axis coordinate within the range (y + max -0.15L,y + max A set of coordinate points within -0.05L).

[0177] Step 212: Calculate the left width using a preset average width calculation method according to the main direction of the fish fillet and the right intercept point cloud.

[0178] The right width is the average width of the right-cut point cloud in the main direction of the fillet calculated by the average width calculation method.

[0179] Reference Figure 7 The fish fillet head and tail identification method further comprises the following steps:

[0180] Step 213: Calculate the width ratio according to the left width and the right width using a preset width ratio calculation method.

[0181] The width ratio calculation method refers to dividing the larger of the left width and the right width by the smaller one to obtain the width ratio. The width ratio calculation method is selected by the staff based on actual conditions and will not be elaborated here.

[0182] Step 214: When the width ratio is less than the preset head-to-tail ratio, the upper left point cloud and the lower left point cloud are matched from the preset ventral and dorsal point cloud segmentation method based on the centroid of the fish fillet, the left point cloud and the main direction of the fish fillet.

[0183] The head-to-tail ratio refers to the minimum width ratio between the head and dorsal sides of a fillet. A typical value of 1.3 is used. This value is determined by the staff based on actual circumstances and is not detailed here. The upper left and lower left point clouds are the two coordinate data sets obtained by segmenting the left point cloud using the ventral and dorsal point cloud segmentation method, along a line passing through the fillet's centroid and oriented in the main direction of the fillet.

[0184] Step 215: Calculate the upper left thickness using a preset average thickness calculation method based on the upper left point cloud, and calculate the lower left thickness using a preset average thickness calculation method based on the lower left point cloud.

[0185] The upper left thickness is the average thickness calculated from the upper left point cloud using the average thickness calculation method. The lower left thickness is the average thickness calculated from the lower left point cloud using the average thickness calculation method.

[0186] Step 216: Based on the centroid of the fillet, the right point cloud, and the main direction of the fillet, the upper right point cloud and the lower right point cloud are matched from the preset ventral and dorsal point cloud segmentation method.

[0187] The upper right point cloud and the lower right point cloud are two sets of coordinate data obtained by segmenting the right point cloud according to a segmentation line passing through the centroid of the fish fillet and toward the main direction of the fish fillet using the ventral and dorsal point cloud segmentation method.

[0188] Step 217: Calculate the upper right thickness using a preset average thickness calculation method based on the upper right point cloud, and calculate the lower right thickness using a preset average thickness calculation method based on the lower right point cloud.

[0189] The upper right thickness is the average thickness calculated from the upper right point cloud using the average thickness calculation method. The lower right thickness is the average thickness calculated from the lower right point cloud using the average thickness calculation method.

[0190] Step 218: Match the minimum thickness using a preset minimum thickness matching method according to the upper left thickness, the lower left thickness, the upper right thickness, and the lower right thickness.

[0191] The minimum thickness matching method refers to a method of selecting the smallest thickness value from the upper left thickness, lower left thickness, upper right thickness and lower right thickness. The minimum thickness matching method is selected by the staff according to actual conditions and will not be described here.

[0192] Step 219: Match the fish head point cloud from a preset point cloud database based on the minimum thickness.

[0193] The point cloud database is a pre-set database that records the upper left point cloud, lower left point cloud, upper right point cloud, and lower right point cloud, as well as their corresponding average thickness values. The fish head point cloud is the point cloud data matched from the point cloud database based on the minimum thickness.

[0194] Step 220: Identify the fish head features using a preset fish head feature recognition method based on the fish head point cloud and the fish fillet image.

[0195] The fish head feature is the feature point of the fish fillet head identified from the fish fillet image based on the fish head point cloud. When the fish head point cloud is the upper left point cloud, the upper left point cloud is judged to be the belly of the fish fillet, and the fish head feature is located in the left point cloud; when the fish head point cloud is the lower left point cloud, the lower left point cloud is judged to be the belly of the fish fillet, and the fish head feature is located in the left point cloud; when the fish head point cloud is the upper right point cloud, the upper right point cloud is judged to be the belly of the fish fillet, and the fish head feature is located in the right point cloud; when the fish head point cloud is the lower right point cloud, the lower right point cloud is judged to be the belly of the fish fillet, and the fish head feature is located in the right point cloud.

[0196] Reference Figure 8 , the methods for adjusting the head and tail of fish fillets include:

[0197] Step 300: Match the fish head orientation based on the fish head features using a preset fish head orientation matching method.

[0198] The fish head orientation matching method refers to a method of determining the orientation of the fish head based on the relative position of the fish head features and the center of the fish fillet. The fish head orientation matching method is selected by the staff based on actual conditions and will not be described in detail here.

[0199] Step 301: matching an adjustment angle from a preset adjustment angle matching method according to the fish head orientation and the preset fish fillet processing direction.

[0200] The adjustment angle matching method can match the adjustment angle that the fish fillet needs to rotate to adjust the fish fillet from the fish head direction to the fish fillet processing direction.

[0201] Step 302: Matching an adjustment stroke from a preset adjustment stroke matching method according to the centroid of the fish fillet and the adjustment angle.

[0202] The adjustment device is a device for adjusting the orientation of the fish fillet. The adjustment device is selected by the staff according to the actual situation and is not described in detail here. The adjustment stroke matching method can match the adjustment stroke of fixing the fish fillet at the center of mass of the fish fillet by the adjustment device and adjusting the orientation of the fish fillet according to the adjustment angle.

[0203] Step 303: Control the preset adjustment device according to the adjustment stroke to turn the fish head toward the fish fillet processing direction.

[0204] The placement of the fish fillet is adjusted by the adjustment device, thereby facilitating the subsequent positioning of various parts of the fish fillet, thereby improving the convenience of fish fillet processing.

[0205] Reference Figure 9 , the fish fillet separation method comprises the following steps:

[0206] Step 400: Determine whether there is any overlap of fish fillets using a preset overlap recognition method based on the fish fillet image.

[0207] The overlapping recognition method refers to a method for determining whether there is overlap of fish fillets based on the color distribution in the fish fillet image. The overlapping recognition method is selected by the staff according to the actual situation and will not be described in detail here.

[0208] Step 401: When there is a situation where fish fillets overlap, the boundary position is matched according to the fish fillet image using a preset overlapping boundary position matching method.

[0209] The overlapping boundary position matching method refers to a method of identifying the portion of the outline of the upper fish fillet within the outline of the lower fish fillet in two overlapping fish fillets based on color difference, that is, a method of identifying the boundary position between the two fish fillets. The overlapping boundary position matching method is selected by the staff based on actual conditions and will not be elaborated here.

[0210] Step 402: Match the grasping position using a preset grasping position matching method according to the boundary position and the fish fillet image.

[0211] The grasping position matching method can divide the fish fillet image into the upper fish fillet part and the lower fish fillet part according to the color difference and boundary position in the fish fillet image, and match the center point of the lower fish fillet part as the grasping position.

[0212] Step 403: Match a grabbing stroke from a preset grabbing stroke matching method according to the grabbing position.

[0213] The gripping device is used to grab fish fillets and change their placement. The gripping device is selected by the staff based on actual conditions and is not described in detail here. The gripping stroke matching method is a neural network method previously trained with samples. This method can match the gripping stroke that allows the gripping device to grasp and remove the fish fillet located below according to the gripping position.

[0214] Step 404: Control a preset grabbing device according to the grabbing stroke to grab the fish fillets and take them away to separate the overlapping fish fillets.

[0215] The fish fillets at the bottom are pulled away by a grabbing device to reduce the overlap of fish fillets.

[0216] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for identifying the placement position of tilapia fillets based on image recognition, characterized in that: include: Get images of tilapia fillets; Matching a pre-processed image from a preset image processing method according to the fish fillet image; Match the transformed image from the preset HSV image transformation method according to the preprocessed image; Extracting background points using a preset background point extraction method according to the transformed image; Matching a background threshold value using a preset background threshold matching method according to the background points; Matching a binary image from a preset binarization processing method according to the transformed image and the background threshold; Matching a binary mask from a preset binary mask processing method according to the binary image; Matching a separation image from a preset image separation method according to the binary mask and the fish fillet image; Match the changed image from the preset HSV image conversion method according to the separated image; Separate the green channel based on the change image; Match the number of edge pixels using the preset edge detection method based on the green channel; Calculating the number of extracted pixels from a preset extracted pixel identification method according to the binary mask; The quotient of the number of edge pixels and the number of extracted pixels was calculated and defined as the fillet pixel ratio; When the pixel ratio of the fish fillet exceeds a preset inner pixel interval, controlling a preset flipping device to flip the fish fillet so that the inner side of the fish fillet faces upward; Obtain point cloud data of fish fillet outline; According to the point cloud data, the center of mass and main direction of the fish fillet are matched using the preset principal component analysis method; Match the upper point cloud and the lower point cloud from the preset ventral and dorsal point cloud segmentation method based on the fish fillet centroid, point cloud data and the main direction of the fish fillet; Calculating the upper thickness using a preset average thickness calculation method based on the upper point cloud, and calculating the lower thickness using a preset average thickness calculation method based on the lower point cloud; When the thickness of the upper portion is greater than the thickness of the lower portion, the fish belly feature is identified using a preset fish belly feature recognition method based on the lower point cloud and the fish fillet image; When the thickness of the lower portion is greater than the thickness of the upper portion, the fish belly feature is identified using a preset fish belly feature recognition method based on the upper point cloud and the fish fillet image; The fish belly contour is matched according to the fish belly features using a preset fish belly contour matching method.

2. A method for identifying the placement position of tilapia fillets based on image recognition according to claim 1, characterized in that: The average thickness calculation method includes: Where D is the required average thickness value; n is the total number of points contained in the point cloud; Z is the Z-axis coordinate value of each point in the point cloud data.

3. A method for identifying the placement position of tilapia fillets based on image recognition according to claim 1, characterized in that: Also includes: Calculating a thickness difference according to the upper thickness and the lower thickness using a preset difference calculation method; When the thickness difference is not greater than the preset ventral-dorsal difference, the upper standard deviation is calculated from the upper point cloud using the preset thickness standard deviation calculation method; Calculate the lower standard deviation from the preset thickness standard deviation calculation method based on the lower point cloud; When the upper standard deviation is greater than the lower standard deviation, the fish belly feature is identified from a preset fish belly feature recognition method based on the upper point cloud and the fish fillet image; When the lower standard deviation is greater than the upper standard deviation, the fish belly feature is identified from a preset fish belly feature recognition method based on the lower point cloud and the fish fillet image.

4. A method for identifying the placement position of tilapia fillets based on image recognition according to claim 3, characterized in that, The thickness standard deviation calculation method includes: Where σ is the required thickness standard deviation; n is the total number of points contained in the point cloud; Z is the Z-axis coordinate value of each point in the point cloud data; D is the average thickness value corresponding to the point cloud.

5. A method for identifying the placement position of tilapia fillets based on image recognition according to claim 1, characterized in that, The invention also includes a method for identifying the head and tail of a fish fillet, wherein the method comprises: Match the left point cloud and the right point cloud from the preset head and tail point cloud segmentation method based on the fish fillet centroid, point cloud data and main direction of the fish fillet; Calculate the left width using the preset average width calculation method based on the main direction of the fish fillet and the left point cloud; The right width is calculated using a preset average width calculation method according to the main direction of the fillet and the right point cloud; When the left width is greater than the right width, the fish head feature is identified using a preset fish head feature recognition method based on the left point cloud and the fish fillet image; When the right width is greater than the left width, the fish head feature is identified from a preset fish head feature recognition method according to the right point cloud and the fish fillet image.

6. A method for identifying the placement position of tilapia fillets based on image recognition according to claim 5, characterized in that: The fish fillet head and tail identification method also includes: Matching the fillet length using a preset fillet length recognition method based on the fillet main direction and point cloud data; When the length of the fish fillet exceeds the preset length calculation interval, the left length is matched from the preset fish fillet length recognition method according to the main direction of the fish fillet and the left point cloud; According to the main direction of the fish fillet and the right point cloud, the right length is matched from the preset fish fillet length recognition method; Calculate the length difference according to the left length and the right length using a preset difference calculation method; Match the left cut point cloud from the preset point cloud cut method according to the length difference, the main direction of the fish fillet and the left point cloud; The left width is calculated using the preset average width calculation method according to the main direction of the fish fillet and the left interception point cloud; Match the right cut point cloud from the preset point cloud cut method according to the length difference, the main direction of the fish fillet and the right point cloud; The left width is calculated from the preset average width calculation method according to the main direction of the fish fillet and the right intercept point cloud.

7. A method for identifying the placement position of tilapia fillets based on image recognition according to claim 5, characterized in that: The fish fillet head and tail identification method also includes: Calculating a width ratio using a preset width ratio calculation method according to the left width and the right width; When the width ratio is less than the preset head-to-tail ratio, the upper left point cloud and the lower left point cloud are matched from the preset ventral and dorsal point cloud segmentation method based on the centroid of the fish fillet, the left point cloud and the main direction of the fish fillet; Calculate the upper left thickness using a preset average thickness calculation method based on the upper left point cloud, and calculate the lower left thickness using a preset average thickness calculation method based on the lower left point cloud; Based on the centroid of the fish fillet, the right point cloud and the main direction of the fish fillet, the upper right point cloud and the lower right point cloud are matched from the preset ventral and dorsal point cloud segmentation method; The upper right thickness is calculated from the preset average thickness calculation method based on the upper right point cloud, and the lower right thickness is calculated from the preset average thickness calculation method based on the lower right point cloud; Matching the minimum thickness from a preset minimum thickness matching method according to the upper left thickness, the lower left thickness, the upper right thickness, and the lower right thickness; Match the fish head point cloud from the preset point cloud database based on the minimum thickness; The fish head features are identified from the preset fish head feature recognition method based on the fish head point cloud and the fish fillet image.

8. A method for identifying the placement position of tilapia fillets based on image recognition according to claim 7, characterized in that: The invention also includes a method for adjusting the head and tail of a fish fillet, wherein the method comprises: Matching the fish head orientation based on the fish head features using a preset fish head orientation matching method; Matching an adjustment angle from a preset adjustment angle matching method according to the fish head orientation and the preset fish fillet processing direction; Matching an adjustment stroke from a preset adjustment stroke matching method according to the centroid of the fish fillet and the adjustment angle; According to the adjustment stroke, the preset adjustment device is controlled to turn the fish head toward the fish fillet processing direction.

Images