Method for recognizing tilapia fillet laying position state based on image recognition

Abstract

The invention relates to a tilapia fillet laying position state identification method based on image identification, relates to the field of intelligent processing of aquatic products, and aims to rapidly identify information such as thickness distribution, width distribution and meat color distribution of tilapia fillets through image identification so as to match the laying position state of tilapia. The method has the effects that the fillet processing efficiency is improved, and the laying position state of the fillets can be quickly recognized.

Description

Technical Field

[0001] The invention relates to the field of intelligent processing of aquatic products, and in particular to a method for identifying the placement position of tilapia fillets based on image recognition. Background Art

[0002] Tilapia is a small to medium-sized tropical fish native to Africa. Since its introduction into China, its breeding and processing industries have developed rapidly. In addition to being eaten fresh, tilapia is mostly processed into fish fillets for sale.

[0003] In the prior art, the tilapia fillet processing technology includes slaughtering, bleeding, filleting, skinning, trimming, sterilization and freezing. During the filleting process, generally, a whole tilapia is removed from its head, tail, fins, scales and other markers and the tilapia is cut into two mirror-image fillets. When the fillets need to be trimmed, workers are generally required to first visually identify the positions of the tail, head and belly on the fillets, and then select different trimming plans according to different parts.

[0004] It takes a long time to manually visually identify the placement of fish fillets. When the number of fish fillets is large, it is easy to lead to a decrease in the processing efficiency of the fish fillets. Summary of the invention

[0005] In order to improve the efficiency of fish fillet processing and quickly identify the placement position of fish fillets, the present invention provides a method for identifying the placement position of tilapia fillets based on image recognition.

[0006] A method for identifying the placement position of tilapia fillets based on image recognition, characterized by comprising:

[0007] Get an image of a tilapia fillet;

[0008] Matching a pre-processed image from a preset image processing method according to the fish fillet image;

[0009] Matching a transformed image from a preset HSV image transformation method according to the preprocessed image;

[0010] Extracting background points using a preset background point extraction method according to the transformed image;

[0011] Matching a background threshold value from a preset background threshold matching method according to the background points;

[0012] Matching a binary image from a preset binary processing method according to the transformed image and the background threshold;

[0013] Matching a binary mask from a preset binary mask processing method according to the binary image;

[0014] Matching a separation image from a preset image separation method according to the binary mask and the fish fillet image;

[0015] According to the separated image, the changed image is matched from the preset HSV image conversion method;

[0016] Separate the green channel from the change image;

[0017] Match the number of edge pixels from the preset edge detection method according to the green channel;

[0018] Calculating the number of extracted pixels from a preset extracted pixel identification method according to the binary mask;

[0019] The quotient of the number of edge pixels and the number of extracted pixels was calculated and defined as the fillet pixel ratio;

[0020] When the pixel ratio of the fish fillet exceeds a preset inner pixel interval, controlling a preset flipping device to flip the fish fillet so that the inner side of the fish fillet faces upward;

[0021] Obtain point cloud data of fish fillet contour;

[0022] According to the point cloud data, the centroid and main direction of the fish fillet are matched by the preset principal component analysis method;

[0023] Match the upper point cloud and the lower point cloud from the preset ventral and dorsal point cloud segmentation method based on the fish fillet centroid, point cloud data and the main direction of the fish fillet;

[0024] The upper thickness is calculated from a preset average thickness calculation method according to the upper point cloud, and the lower thickness is calculated from a preset average thickness calculation method according to the lower point cloud;

[0025] When the thickness of the upper portion is greater than the thickness of the lower portion, the fish belly feature is identified from a preset fish belly feature identification method according to the lower point cloud and the fish fillet image;

[0026] When the thickness of the lower part is greater than the thickness of the upper part, the fish belly feature is identified from a preset fish belly feature identification method according to the upper point cloud and the fish fillet image;

[0027] The fish belly contour is matched according to the fish belly features from a preset fish belly contour matching method.

[0028] By adopting the above technical scheme, the background color in the fish fillet image is extracted to form a background threshold, so that the fish fillet is separated from the background according to the background threshold, and the placement state of the fish fillet is judged by the difference in the color of the fish meat inside and outside the fish fillet through the canny operator, so as to keep the inside of the fish fillet facing up, thereby improving the convenience of subsequent processing of the fish fillet, and the head and tail direction and center of mass position of the fish fillet are analyzed from the point cloud data, so as to divide the fish fillet into two parts according to the head and tail direction of the fish fillet, and the part where the abdomen is located is judged by comparing the average thickness of the two parts of the fish fillet, so as to identify the abdominal area on the fish fillet, and then judge the placement position of the fish fillet according to the abdominal area, thereby improving the efficiency of fish fillet processing.

[0029] Optionally, the average thickness calculation method includes:

[0030] Where D is the required average thickness value;

[0031] n is the total number of point clouds;

[0032] Z is the Z-axis coordinate value of each point in the point data.

[0033] Optionally, also include:

[0034] Calculating a thickness difference according to the upper thickness and the lower thickness using a preset difference calculation method;

[0035] When the thickness difference is not greater than the preset ventral-dorsal difference, the upper standard deviation is calculated from the preset thickness standard deviation calculation method according to the upper point cloud;

[0036] Calculate the lower standard deviation from the preset thickness standard deviation calculation method according to the lower point cloud;

[0037] When the upper standard deviation is greater than the lower standard deviation, the fish belly feature is identified from a preset fish belly feature identification method according to the upper point cloud and the fish fillet image;

[0038] When the lower standard deviation is greater than the upper standard deviation, the fish belly feature is identified from a preset fish belly feature recognition method according to the lower point cloud and the fish fillet image.

[0039] By adopting the above technical solution, when the average thickness difference between the two parts is not large, it is not easy to directly judge the part where the abdomen is located by the thickness. At this time, the thickness difference in the part is displayed by the standard deviation of the two parts, and the part with a larger difference is judged as the abdomen, so as to judge the placement position of the fish fillet according to the abdominal area.

[0040] Optionally, the thickness standard deviation calculation method includes:

[0041] Where σ is the required thickness standard deviation;

[0042] n is the total number of points contained in the point cloud;

[0043] Z is the z-axis coordinate value of each point contained in the point cloud;

[0044] D is the average thickness corresponding to the point cloud.

[0045] Optionally, a method for identifying the head and tail of a fish fillet is also included, and the method for identifying the head and tail of a fish fillet comprises:

[0046] Match the left point cloud and the right point cloud from the preset head and tail point cloud segmentation method based on the fish fillet centroid, point cloud data and the main direction of the fish fillet;

[0047] The left width is calculated from the preset average width calculation method according to the main direction of the fish fillet and the left point cloud;

[0048] The right width is calculated from the preset average width calculation method according to the main direction of the fish fillet and the right point cloud;

[0049] When the left width is greater than the right width, the fish head feature is identified from a preset fish head feature identification method according to the left point cloud and the fish fillet image;

[0050] When the right width is greater than the left width, the fish head feature is identified from a preset fish head feature recognition method according to the right point cloud and the fish fillet image.

[0051] By adopting the above technical solution, the fish fillet is divided into two parts according to the center of mass and the direction perpendicular to the head and tail of the fish fillet, and the average width of the two parts of the fish fillet is calculated. The wider part is judged as the part where the head of the fish fillet is located based on the average width, and the position of the fish head is then identified, and the placement position of the fish fillet is judged based on the position of the fish head.

[0052] Optionally, the fish fillet head and tail identification method further includes:

[0053] According to the main direction of the fish fillet and the point cloud data, the length of the fish fillet is matched from a preset fish fillet length recognition method;

[0054] When the length of the fish fillet exceeds the preset length calculation interval, the left length is matched from a preset fish fillet length recognition method according to the main direction of the fish fillet and the left point cloud;

[0055] According to the main direction of the fish fillet and the right point cloud, the right length is matched from a preset fish fillet length recognition method;

[0056] Calculate the length difference according to the left length and the right length using a preset difference calculation method;

[0057] Match the left cut point cloud from the preset point cloud cut method according to the length difference, the main direction of the fish fillet and the left point cloud;

[0058] The left width is calculated from the preset average width calculation method according to the main direction of the fish fillet and the left interception point cloud;

[0059] According to the length difference, the main direction of the fish fillet and the right point cloud, the right point cloud is matched from the preset point cloud interception method;

[0060] The left width is calculated from the preset average width calculation method according to the main direction of the fish fillet and the right intercept point cloud.

[0061] By adopting the above technical solution, when the fish fillet is long, the amount of calculation for calculating the average width of the fish fillet is large. At this time, a segment of equal relative length is cut from each of the two parts of the fish fillet, and the average width of the segment is used as the average width of the fish fillet part, thereby reducing the amount of calculation for the average width.

[0062] Optionally, the fish fillet head and tail identification method further includes:

[0063] Calculating a width ratio from a preset width ratio calculation method according to the left width and the right width;

[0064] When the width ratio is less than the preset head-tail ratio, the upper left point cloud and the lower left point cloud are matched from the preset belly-back point cloud segmentation method based on the centroid of the fish fillet, the left point cloud and the main direction of the fish fillet;

[0065] The upper left thickness is calculated from a preset average thickness calculation method according to the upper left point cloud, and the lower left thickness is calculated from a preset average thickness calculation method according to the lower left point cloud;

[0066] Based on the centroid of the fish fillet, the right point cloud and the main direction of the fish fillet, the upper right point cloud and the lower right point cloud are matched from the preset abdominal and dorsal point cloud segmentation method;

[0067] The upper right thickness is calculated from a preset average thickness calculation method according to the upper right point cloud, and the lower right thickness is calculated from a preset average thickness calculation method according to the lower right point cloud;

[0068] Matching the minimum thickness from a preset minimum thickness matching method according to the upper left thickness, the lower left thickness, the upper right thickness and the lower right thickness;

[0069] Match the fish head point cloud from the preset point cloud database based on the minimum thickness;

[0070] The fish head features are identified from a preset fish head feature recognition method according to the fish head point cloud and the fish fillet image.

[0071] By adopting the above technical solution, when the widths of the two parts are similar, it is not easy to determine the position of the fish head. At this time, the fish fillet particle is taken as the origin, and the fish fillet is divided into four parts by the main direction of the fish fillet and two cutting lines perpendicular to the main direction of the fish fillet, and the average thickness of the four parts is calculated respectively, so as to determine the position of the fish belly and then the position of the fish head.

[0072] Optionally, a method for adjusting the head and tail of a fish fillet is also included, and the method for adjusting the head and tail of a fish fillet comprises:

[0073] Matching the fish head orientation from a preset fish head orientation matching method based on the fish head feature;

[0074] Matching an adjustment angle from a preset adjustment angle matching method according to the fish head orientation and the preset fish fillet processing direction;

[0075] Matching the adjustment stroke from a preset adjustment stroke matching method according to the fish fillet mass center and the adjustment angle;

[0076] The preset adjustment device is controlled according to the adjustment stroke to direct the fish head toward the fish fillet processing direction.

[0077] By adopting the above technical solution, when the head direction of the fish fillet deviates from the processing direction of the fish fillet, the head direction of the fish fillet is adjusted by the adjustment device, so that the fish fillet is arranged in a body shape with the head direction consistent with the processing direction of the fish fillet, thereby improving the convenience of subsequent processing of the fish fillet.

[0078] In summary, the present application includes at least one of the following beneficial technical effects:

[0079] 1. The head and tail orientation and center of mass of the fish fillet are analyzed from the point cloud data, and the fish fillet is divided into two parts according to the head and tail orientation. The belly part is determined by comparing the average thickness of the two parts of the fish fillet, and the belly area on the fish fillet is identified. The placement status of the fish fillet is determined based on the belly area, thereby improving the efficiency of fish fillet processing;

[0080] 2. When the average thickness difference between the two parts of the fish fillet is not large, it is not easy to directly determine the part where the abdomen is located by thickness. In this case, the thickness difference between the two parts is displayed by the standard deviation of the two parts, and the part with the larger thickness difference is determined as the abdomen, so as to determine the placement status of the fish fillet according to the abdomen area;

[0081] 3. Divide the fish fillet into two parts according to the center of mass and the direction perpendicular to the head and tail of the fish fillet, and calculate the average width of the two parts of the fish fillet, so as to judge that the wider part is the part where the head of the fish fillet is located based on the average width, and then identify the position of the fish head, and judge the placement status of the fish fillet based on the position of the fish head. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 The present invention is a process of identifying the placement status of tilapia fillets based on image recognition. Figure 1 ;

[0083] Figure 2 The present invention is a process of identifying the placement status of tilapia fillets based on image recognition. Figure 2 ;

[0084] Figure 3 The present invention is a process of identifying the placement status of tilapia fillets based on image recognition. Figure 3 ;

[0085] Figure 4 The present invention is a process of identifying the placement status of tilapia fillets based on image recognition. Figure 4 ;

[0086] Figure 5 The process of identifying the head and tail of fish fillet Figure 1 ;

[0087] Figure 6 The process of identifying the head and tail of fish fillet Figure 2 ;

[0088] Figure 7 The process of identifying the head and tail of fish fillet Figure 3 ;

[0089] Figure 8 It is a flow chart of the method of adjusting the head and tail of a fish fillet;

[0090] Fig. 9 is a flow chart of the fish fillet separation method. DETAILED DESCRIPTION

[0091] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0092] The embodiment of the present invention discloses a method for identifying the placement position of tilapia fillets based on image recognition. The present invention quickly identifies information such as thickness distribution, width distribution and meat color distribution of tilapia fillets through image recognition, thereby matching the placement position of the tilapia.

[0093] Reference Figure 1 A method for identifying the placement status of tilapia fillets based on image recognition comprises the following steps:

[0094] Step 100: Acquire a tilapia fillet image.

[0095] The fish fillet image refers to a color picture of tilapia. The fish fillet image can be obtained through a camera. The method for obtaining the fish fillet image is selected by the staff according to the actual situation and will not be described in detail here.

[0096] Step 101: matching a pre-processed image from a preset image processing method according to the fish fillet image.

[0097] The image processing method refers to a method of obtaining a preprocessed image by preprocessing the fish fillet image through Gaussian filtering and noise reduction. The image processing method is selected by the staff according to the actual situation and will not be described here.

[0098] Step 102: Match a converted image from a preset HSV image conversion method according to the preprocessed image.

[0099] The HSV image conversion method refers to a method of converting a preprocessed image in RGB format into a converted image in HSV format. The HSV image conversion method is selected by the staff according to the actual situation and will not be described here.

[0100] Step 103: extracting background points from the converted image using a preset background point extraction method.

[0101] The background point extraction method refers to extracting small squares of 25*25 pixels in the upper left corner, lower left corner, upper right corner and lower right corner of the fish fillet image as background points. The background point extraction method is selected by the staff according to the actual situation and will not be described here.

[0102] Step 104: Matching a background threshold according to the background points using a preset background threshold matching method.

[0103] The background threshold matching method can calculate the maximum and minimum values ​​of the three channels H, S and V in the four small squares respectively, and then calculate the average value of the maximum and minimum values ​​of the three channels H, S and V of the four small squares, and use the average value of the minimum values ​​of the three channels H, S and V as the lower limit of the background threshold, and use the average value of the maximum values ​​of the three channels H, S and V as the upper limit of the background threshold to form the background threshold. The background threshold matching method is selected by the staff according to the actual situation and will not be elaborated here.

[0104] Step 105: A binary image is matched from a preset binary processing method according to the converted image and the background threshold.

[0105] The binarization processing method can set the pixel values ​​in the converted image within the background threshold range to [255, 255, 255], and the pixel values ​​outside the background threshold range to [0, 0, 0] (i.e., the background is black and the fish fillet is white) to form a binary image. The binarization processing method is selected by the staff according to the actual situation and will not be elaborated here.

[0106] Step 106: Match a binary mask from a preset binary mask processing method according to the binary image.

[0107] The binary mask processing method can obtain a binary mask after opening and closing the binary image and performing noise removal operations. The binary mask processing method is selected by the staff according to the actual situation and will not be described in detail here.

[0108] Step 107: Matching a separation image from a preset image separation method according to the binary mask and the fish fillet image.

[0109] The image separation method can perform bitwise AND of the fish fillet image and the binary mask to obtain a separation image in which the fish fillet in the fish fillet image is separated from the background.

[0110] Reference Figure 2 A method for identifying the placement position of tilapia fillets based on image recognition also includes the following steps:

[0111] Step 108: Match the changed image from the preset HSV image conversion method according to the separated image.

[0112] The changed image is an HSV image obtained by converting the RGB separated image through the HSV image conversion method.

[0113] Step 109: Separate the green channel according to the changed image.

[0114] The green channel refers to an image formed by separating the green area from the HSV-format changing image. The outer side of the fish fillet contains dark meat, which is more obvious under the green channel, and more edges can be detected. The green channel separation method is selected by the staff according to the actual situation and will not be elaborated here.

[0115] Step 110: Match the number of edge pixels using a preset edge detection method according to the green channel.

[0116] The edge detection method can detect the change image under the green channel through the canny operator and then count the number of detected edge pixels. The edge detection method is selected by the staff according to the actual situation and will not be described here.

[0117] Step 111: Calculate the number of extracted pixels from a preset extracted pixel identification method according to the binary mask.

[0118] The extracted pixel recognition method can match the total number of pixels of the white part (i.e., the fish fillet part) in the binary mask. The number of extracted pixels is the result obtained by matching the extracted pixel recognition method. The extracted pixel recognition method is selected by the staff according to the actual situation and will not be elaborated here.

[0119] Step 112: Calculate the quotient of the number of edge pixels and the number of extracted pixels, and define it as the fillet pixel ratio.

[0120] The fillet pixel ratio is obtained by scaling the edge pixel number by dividing it by the number of extracted pixels.

[0121] Step 113: When the fish fillet pixel ratio exceeds the preset inner pixel interval, control a preset flipping device to flip the fish fillet so that the inner side of the fish fillet faces upward.

[0122] The inner pixel interval refers to the maximum value of the pixel ratio in the fish fillet set in advance. The inner pixel interval is selected by the staff according to the actual situation and will not be described in detail here. The flipping device is used to flip the fish fillet so that the fish fillet is flipped from the outside to the inside facing up. If the fish fillet pixel ratio exceeds the inner pixel interval, it means that the fish fillet in the fish fillet image is on the outside at this time. At this time, the flipping device is used to adjust the placement of the fish fillet to facilitate the subsequent processing of the fish fillet.

[0123] Reference Figure 3 A method for identifying the placement position of tilapia fillets based on image recognition also includes the following steps:

[0124] Step 114: Obtain point cloud data of the fish fillet contour.

[0125] Point cloud data refers to the spatial position data set of points of the outline of the fish fillet. The point cloud data can be obtained by a laser scanner. The method for obtaining the point cloud data is selected by the staff according to the actual situation and will not be elaborated here.

[0126] Step 115: Match the fish fillet mass center and the fish fillet main direction using a preset principal component analysis method according to the point cloud data.

[0127] Principal component analysis is a statistical method. It can find the main direction of data change in point cloud data, that is, the main direction of fish fillets, and identify the center of mass in the point cloud data as the center of mass of fish fillets. After matching the center of mass of fish fillets, the center of mass of fish fillets is used as the origin, and the direction of the fish fillets away from the background is used as the positive direction of the z-axis, where the background refers to the platform on which the fish fillets are placed, and the background is generally a conveyor belt. The fish fillet processing direction is used as the positive direction of the y-axis, and the scanning direction of the laser scanner is used as the positive direction of the x-axis to establish a coordinate system, and the point cloud data is substituted into the coordinate system to obtain the coordinate position of the points of the fish fillet's outline in the coordinate system, and the point cloud data is updated to the coordinate position, where the fish fillet processing direction refers to the direction in which the fish fillet moves on the processing flow line. The fish fillet processing direction is selected by the staff according to actual conditions and will not be elaborated here.

[0128] Step 116: Match the upper point cloud and the lower point cloud from the preset ventral and dorsal point cloud segmentation method based on the centroid of the fish fillet, the point cloud data and the main direction of the fish fillet.

[0129] The ventral and dorsal point cloud segmentation method can separate the point cloud data according to the dividing line. The upper point cloud and the lower point cloud are two sets of coordinate point data obtained by dividing the point cloud data with a dividing line passing through the centroid of the fish fillet and facing the main direction of the fish fillet.

[0130] Step 117: Calculate the upper thickness according to the upper point cloud using a preset average thickness calculation method, and calculate the lower thickness according to the lower point cloud using a preset average thickness calculation method.

[0131] The average thickness calculation method can calculate the average thickness of the point cloud data according to the z-axis coordinate value of the coordinates in the point cloud data. The upper thickness is the thickness result calculated by the average thickness calculation method for the upper point cloud, and the lower thickness is the thickness result calculated by the average thickness calculation method for the lower point cloud.

[0132] The average thickness is generally calculated using the formula Where D is the required average thickness value, n is the total number of point clouds, and Z is the Z-axis coordinate value of each point in the point cloud data.

[0133] Step 118: When the upper thickness is greater than the lower thickness, the fish belly feature is identified from a preset fish belly feature recognition method according to the lower point cloud and the fish fillet image.

[0134] The average thickness of the ventral side of the fish fillet is generally smaller than the average thickness of the dorsal side. The upper thickness is greater than the lower thickness, which means that the upper point cloud corresponds to the dorsal side of the fish fillet, and the lower point cloud corresponds to the ventral side of the fish fillet. The fish belly feature recognition method can identify the fish belly features according to the color difference in the area where the lower point cloud is located based on the matching of the area of ​​the lower point cloud on the fish fillet image.

[0135] Step 119: When the thickness of the lower portion is greater than the thickness of the upper portion, the fish belly feature is identified using a preset fish belly feature recognition method based on the upper point cloud and the fish fillet image.

[0136] The lower thickness is greater than the upper thickness, which means that the lower point cloud corresponds to the dorsal side of the fish fillet, and the upper point cloud corresponds to the ventral side of the fish fillet. The fish belly feature recognition method is used to match the area of ​​the upper point cloud in the fish fillet image, and then the fish belly feature is recognized from the area where the upper point cloud is located according to the color difference.

[0137] Step 120: Match the fish belly contour according to the fish belly features using a preset fish belly contour matching method.

[0138] The fish belly contour matching method refers to using the color of the fish belly feature as an indicator to identify the range of the fish belly contour according to the color difference.

[0139] Reference Figure 4 A method for identifying the placement position of tilapia fillets based on image recognition also includes the following steps:

[0140] Step 121: Calculate the thickness difference according to the upper thickness and the lower thickness using a preset difference calculation method.

[0141] The difference calculation method refers to a calculation method of obtaining the thickness difference by subtracting the smaller thickness from the larger thickness of the upper portion and the lower portion.

[0142] Step 122: When the thickness difference is not greater than the preset ventral-dorsal difference, the upper standard deviation is calculated according to the upper point cloud using a preset thickness standard deviation calculation method.

[0143] The belly-back difference refers to the preset minimum thickness difference between the belly and back sides of the fish fillet. The belly-back difference can be one-tenth of the larger of the upper thickness and the lower thickness. The belly-back difference is selected by the staff according to the actual situation and will not be elaborated here. If the thickness difference is not greater than the belly-back difference, it means that the belly and back sides of the fish fillet cannot be directly judged at this time.

[0144] The thickness standard deviation calculation method refers to the calculation method used to calculate the thickness standard deviation. The thickness standard deviation calculation method generally uses the formula Where σ is the required thickness standard deviation, n is the total number of points in the point cloud, Z is the z-axis coordinate value of each point in the point cloud, D is the average thickness corresponding to the point cloud, and the upper standard deviation is the standard deviation result calculated by the thickness standard deviation calculation method for the upper point cloud.

[0145] Step 123: Calculate the lower standard deviation according to the lower point cloud using a preset thickness standard deviation calculation method.

[0146] The lower standard deviation is the standard deviation result calculated by the thickness standard deviation calculation method for the lower point cloud.

[0147] Step 124: When the upper standard deviation is greater than the lower standard deviation, the fish belly feature is identified from a preset fish belly feature recognition method based on the upper point cloud and the fish fillet image.

[0148] The thickness difference on the ventral side of the fish fillet is large. The upper standard deviation is greater than the lower standard deviation, which means that the thickness difference of the upper point cloud is large, that is, the upper point cloud corresponds to the ventral side of the fish fillet.

[0149] Step 125: When the lower standard deviation is greater than the upper standard deviation, the fish belly feature is identified from a preset fish belly feature recognition method according to the lower point cloud and the fish fillet image.

[0150] The lower standard deviation is greater than the upper standard deviation, which means that the thickness difference of the lower point cloud is larger, that is, the lower point cloud corresponds to the ventral side of the fish fillet.

[0151] Reference Figure 5 , the fish fillet head and tail identification method comprises the following steps:

[0152] Step 200: Match the left point cloud and the right point cloud from a preset head and tail point cloud segmentation method based on the fish fillet centroid, point cloud data and the main direction of the fish fillet.

[0153] The head and tail point cloud segmentation method can separate the point cloud data according to the segmentation line. The left point cloud and the right point cloud are two sets of coordinate point data obtained by segmenting the point cloud data with a segmentation line passing through the centroid of the fish fillet and perpendicular to the main direction of the fish fillet.

[0154] Step 201: Calculate the left width according to the main direction of the fish fillet and the left point cloud using a preset average width calculation method.

[0155] The average width calculation method can calculate the average width of the point cloud data in the main direction of the fish fillet according to the coordinate values ​​in the point cloud data. The left width is the width result obtained by calculating the left point cloud by the average width calculation method. The average width calculation method is selected by the staff according to the actual situation and will not be elaborated here.

[0156] Step 202: Calculate the right width according to the main direction of the fish fillet and the right point cloud using a preset average width calculation method.

[0157] The right width is the width result calculated by the average width calculation method of the right point cloud.

[0158] Step 203: When the left width is greater than the right width, the fish head feature is identified from a preset fish head feature recognition method according to the left point cloud and the fish fillet image.

[0159] The width of the head side of the fish fillet is generally greater than the width of the tail side. The left width is greater than the right width, which means that the left point cloud corresponds to the head side of the fish fillet. The fish head feature recognition method can match the area of ​​the left point cloud on the fish fillet image, and then highlight the contour points in the area where the left point cloud is located as the fish head feature.

[0160] Step 204: When the right width is greater than the left width, the fish head feature is identified from a preset fish head feature recognition method according to the right point cloud and the fish fillet image.

[0161] The right width is greater than the left width, which means that the right point cloud corresponds to the head side of the fish fillet. The fish head feature is a prominent point identified from the corresponding area of ​​the right point cloud on the fish fillet image by the fish head feature recognition method.

[0162] Reference Figure 6 The fish fillet head and tail identification method also includes the following steps:

[0163] Step 205: matching the length of the fish fillet from a preset fish fillet length recognition method according to the main direction of the fish fillet and the point cloud data.

[0164] The fish fillet length recognition method can calculate the maximum length of the point cloud data in the main direction of the fish fillet according to the coordinate values ​​of the point cloud data, and define the maximum length as the fish fillet length.

[0165] Step 206: When the length of the fish fillet exceeds the preset length calculation interval, the left length is matched from a preset fish fillet length recognition method according to the main direction of the fish fillet and the left point cloud.

[0166] The length calculation interval refers to the length interval of the fish fillet with less width calculation. The fish fillet length exceeding the preset length calculation interval means that the calculation amount of the average width calculation is large and the calculation time is long. The length calculation interval is selected by the staff according to the actual situation and will not be described here. The left length is the maximum length of the left point cloud matched by the fish fillet length recognition method in the main direction of the fish fillet.

[0167] Step 207: Match the right length from a preset fish fillet length recognition method according to the main direction of the fish fillet and the right point cloud.

[0168] The right length is the maximum length of the right point cloud matched by the fish fillet length recognition method in the main direction of the fish fillet.

[0169] Step 208: Calculate the length difference according to the left length and the right length using a preset difference calculation method.

[0170] The length difference is the difference between the larger of the left length and the right length calculated by the difference calculation method and the smaller of the larger of the left length and the right length.

[0171] Step 209: Match the left cut point cloud from a preset point cloud cut method according to the length difference, the main direction of the fish fillet and the left point cloud.

[0172] The point cloud interception method can intercept the coordinate data of some points in the point cloud data. The main direction of the fish fillet is generally the same as the positive direction of the y-axis of the coordinate system. The length difference is defined as L, and the minimum y-axis value of the point in the left point cloud is defined as y - max , define the maximum value of the y-axis of the point in the right point cloud as y + max , L = y + max +y - max , generally the y-axis coordinates of the left point cloud are within the range (y - max +0.05L,y - max +0.15L) is taken as the left intercept point cloud.

[0173] Step 210: Calculate the left width using a preset average width calculation method according to the main direction of the fish fillet and the left interception point cloud.

[0174] The left width is the average width value of the left-cut point cloud in the main direction of the fish fillet calculated by the average width calculation method.

[0175] Step 211: Match the right cut point cloud from a preset point cloud cut method according to the length difference, the main direction of the fish fillet and the right point cloud.

[0176] The right intercept point cloud is the point cloud on the right intercepted by the point cloud interception method with the y-axis coordinate in the range (y + max -0.15L,y + max -0.05L) within the coordinate point set.

[0177] Step 212: Calculate the left width using a preset average width calculation method according to the main direction of the fish fillet and the right intercept point cloud.

[0178] The right width is the average width value of the right-cut point cloud in the main direction of the fish fillet calculated by the average width calculation method.

[0179] Reference Figure 7 The fish fillet head and tail identification method also includes the following steps:

[0180] Step 213: Calculate the width ratio according to the left width and the right width using a preset width ratio calculation method.

[0181] The width ratio calculation method refers to dividing the larger of the left width and the right width by the smaller one to obtain the width ratio. The width ratio calculation method is selected by the staff according to actual conditions and will not be elaborated here.

[0182] Step 214: When the width ratio is less than the preset head-to-tail ratio, the upper left point cloud and the lower left point cloud are matched from the preset belly and back point cloud segmentation method based on the centroid of the fish fillet, the left point cloud and the main direction of the fish fillet.

[0183] The head-to-tail ratio refers to the minimum width ratio between the head side and the back side of the fish fillet. The head-to-tail ratio is generally 1.3. The head-to-tail ratio is selected by the staff according to the actual situation and will not be described here. The upper left point cloud and the lower left point cloud are two sets of coordinate data obtained by segmenting the left point cloud according to the segmentation line passing through the centroid of the fish fillet and facing the main direction of the fish fillet through the ventral and dorsal point cloud segmentation method.

[0184] Step 215: Calculate the upper left thickness using a preset average thickness calculation method based on the upper left point cloud, and calculate the lower left thickness using a preset average thickness calculation method based on the lower left point cloud.

[0185] The upper left thickness is the average thickness result calculated from the upper left point cloud by the average thickness calculation method. The lower left thickness is the average thickness result calculated from the lower left point cloud by the average thickness calculation method.

[0186] Step 216: Match the upper right point cloud and the lower right point cloud from the preset belly and back point cloud segmentation method based on the fish fillet centroid, the right point cloud and the main direction of the fish fillet.

[0187] The upper right point cloud and the lower right point cloud are two sets of coordinate data obtained by segmenting the right point cloud according to a segmentation line passing through the centroid of the fish fillet and toward the main direction of the fish fillet using the ventral and dorsal point cloud segmentation method.

[0188] Step 217: Calculate the upper right thickness using a preset average thickness calculation method based on the upper right point cloud, and calculate the lower right thickness using a preset average thickness calculation method based on the lower right point cloud.

[0189] The upper right thickness is the average thickness result calculated from the upper right point cloud by the average thickness calculation method. The lower right thickness is the average thickness result calculated from the lower right point cloud by the average thickness calculation method.

[0190] Step 218: Match the minimum thickness from a preset minimum thickness matching method according to the upper left thickness, the lower left thickness, the upper right thickness and the lower right thickness.

[0191] The minimum thickness matching method refers to a method of selecting the smallest thickness value from the upper left thickness, lower left thickness, upper right thickness and lower right thickness. The minimum thickness matching method is selected by the staff according to actual conditions and will not be elaborated here.

[0192] Step 219: Match the fish head point cloud from a preset point cloud database based on the minimum thickness.

[0193] The point cloud database refers to a pre-set database that records the upper left point cloud, lower left point cloud, upper right point cloud and lower right point cloud and their corresponding average thickness values. The fish head point cloud is the point cloud data matched from the point cloud database according to the minimum thickness.

[0194] Step 220: Identify the fish head features using a preset fish head feature recognition method based on the fish head point cloud and the fish fillet image.

[0195] The fish head feature is the feature point of the fish fillet head identified from the fish fillet image based on the fish head point cloud, where when the fish head point cloud is the upper left point cloud, the upper left point cloud is judged to be the belly of the fish fillet, and the fish head feature is located in the left point cloud; when the fish head point cloud is the lower left point cloud, the lower left point cloud is judged to be the belly of the fish fillet, and the fish head feature is located in the left point cloud; when the fish head point cloud is the upper right point cloud, the upper right point cloud is judged to be the belly of the fish fillet, and the fish head feature is located in the right point cloud; when the fish head point cloud is the lower right point cloud, the lower right point cloud is judged to be the belly of the fish fillet, and the fish head feature is located in the right point cloud.

[0196] Reference Figure 8 , the methods for adjusting the head and tail of fish fillets include:

[0197] Step 300: Match the fish head orientation based on the fish head feature from a preset fish head orientation matching method.

[0198] The fish head orientation matching method refers to a method for determining the orientation of the fish head based on the relative position between the fish head features and the center of the fish fillet. The fish head orientation matching method is selected by the staff based on actual conditions and will not be elaborated here.

[0199] Step 301: matching an adjustment angle from a preset adjustment angle matching method according to the fish head orientation and a preset fish fillet processing direction.

[0200] The adjustment angle matching method can match the adjustment angle required for the fish fillet to rotate from the fish head direction to the fish fillet processing direction.

[0201] Step 302: matching an adjustment stroke from a preset adjustment stroke matching method according to the centroid of the fish fillet and the adjustment angle.

[0202] The adjustment device refers to a device for adjusting the direction of the fish fillet. The adjustment device is selected by the staff according to the actual situation and will not be described in detail here. The adjustment stroke matching method can match the adjustment stroke of fixing the fish fillet at the center of mass of the fish fillet through the adjustment device and adjusting the direction of the fish fillet according to the adjustment angle.

[0203] Step 303: Controlling a preset adjustment device according to the adjustment stroke to direct the fish head toward the fish fillet processing direction.

[0204] The placement of the fish fillet is adjusted by means of an adjustment device, thereby facilitating the subsequent positioning of various parts of the fish fillet, thereby improving the convenience of fish fillet processing.

[0205] Reference Fig. 9 , the fish fillet separation method comprises the following steps:

[0206] Step 400: judging whether there is any overlap of fish fillets according to the fish fillet image by using a preset overlap identification method.

[0207] The overlapping identification method refers to a method for determining whether there is overlap of fish fillets based on the color distribution in the fish fillet image. The overlapping identification method is selected by the staff according to the actual situation and will not be described in detail here.

[0208] Step 401: When there is overlap of fish fillets, the boundary position is matched according to the fish fillet image using a preset overlapping boundary position matching method.

[0209] The overlapping boundary position matching method refers to a method of identifying the portion of the contour line of the upper fish fillet within the contour range of the lower fish fillet among the two overlapping fish fillets based on color difference, that is, a method of identifying the boundary position between the two fish fillets. The overlapping boundary position matching method is selected by the staff according to actual conditions and will not be elaborated here.

[0210] Step 402: Match the grasping position according to the boundary position and the fish fillet image using a preset grasping position matching method.

[0211] The grasping position matching method can divide the fish fillet image into the upper fish fillet part and the lower fish fillet part according to the color difference and boundary position in the fish fillet image, and match the center point from the lower fish fillet part as the grasping position.

[0212] Step 403: Match the grabbing stroke from a preset grabbing stroke matching method according to the grabbing position.

[0213] The grabbing device refers to a device used to grab the fish fillet to change the placement of the fish fillet. The grabbing device is selected by the staff according to the actual situation and will not be described in detail here. The grabbing stroke matching method refers to a neural network method obtained through sample training in advance. The grabbing stroke matching method can match the grabbing stroke of grabbing the fish fillet located below according to the grabbing position through the grabbing device and extracting the fish fillet located below.

[0214] Step 404: Control a preset grabbing device according to the grabbing stroke to grab and take away the fish fillets so as to separate the overlapping fish fillets.

[0215] The fish fillets at the bottom are pulled away by a grabbing device to reduce the overlap of fish fillets.

[0216] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for identifying the placement position of tilapia fillets based on image recognition, characterized in that: include: Get an image of a tilapia fillet; Matching a pre-processed image from a preset image processing method according to the fish fillet image; Matching a transformed image from a preset HSV image transformation method according to the preprocessed image; Extracting background points using a preset background point extraction method according to the transformed image; Matching a background threshold value from a preset background threshold matching method according to the background points; Matching a binary image from a preset binary processing method according to the transformed image and the background threshold; Matching a binary mask from a preset binary mask processing method according to the binary image; Matching a separation image from a preset image separation method according to the binary mask and the fish fillet image; According to the separated image, the changed image is matched from the preset HSV image conversion method; Separate the green channel from the change image; Match the number of edge pixels from the preset edge detection method according to the green channel; Calculating the number of extracted pixels from a preset extracted pixel identification method according to the binary mask; The quotient of the number of edge pixels and the number of extracted pixels was calculated and defined as the fillet pixel ratio; When the pixel ratio of the fish fillet exceeds a preset inner pixel interval, controlling a preset flipping device to flip the fish fillet so that the inner side of the fish fillet faces upward; Obtain point cloud data of fish fillet contour; According to the point cloud data, the centroid and main direction of the fish fillet are matched by the preset principal component analysis method; Match the upper point cloud and the lower point cloud from the preset ventral and dorsal point cloud segmentation method based on the fish fillet centroid, point cloud data and the main direction of the fish fillet; The upper thickness is calculated from a preset average thickness calculation method according to the upper point cloud, and the lower thickness is calculated from a preset average thickness calculation method according to the lower point cloud; When the thickness of the upper portion is greater than the thickness of the lower portion, the fish belly feature is identified from a preset fish belly feature identification method according to the lower point cloud and the fish fillet image; When the thickness of the lower part is greater than the thickness of the upper part, the fish belly feature is identified from a preset fish belly feature identification method according to the upper point cloud and the fish fillet image; The fish belly contour is matched according to the fish belly features from a preset fish belly contour matching method.

2. A method for identifying the placement position of tilapia fillets based on image recognition according to claim 1, characterized in that: The average thickness calculation method comprises: Where D is the required average thickness value; n is the total number of point clouds; Z is the Z-axis coordinate value of each point in the point data.

3. A method for identifying the placement position of tilapia fillets based on image recognition according to claim 1, characterized in that: Also includes: Calculating a thickness difference according to the upper thickness and the lower thickness using a preset difference calculation method; When the thickness difference is not greater than the preset ventral-dorsal difference, the upper standard deviation is calculated from the preset thickness standard deviation calculation method according to the upper point cloud; Calculate the lower standard deviation from the preset thickness standard deviation calculation method according to the lower point cloud; When the upper standard deviation is greater than the lower standard deviation, the fish belly feature is identified from a preset fish belly feature identification method according to the upper point cloud and the fish fillet image; When the lower standard deviation is greater than the upper standard deviation, the fish belly feature is identified from a preset fish belly feature recognition method according to the lower point cloud and the fish fillet image.

4. A method for identifying the placement position of tilapia fillets based on image recognition according to claim 3, characterized in that: The thickness standard deviation calculation method includes: Where σ is the required thickness standard deviation; n is the total number of points contained in the point cloud; Z is the z-axis coordinate value of each point contained in the point cloud; D is the average thickness corresponding to the point cloud.

5. A method for identifying the placement position of tilapia fillets based on image recognition according to claim 1, characterized in that: The invention also includes a method for identifying the head and tail of a fish fillet, wherein the method comprises: Match the left point cloud and the right point cloud from the preset head and tail point cloud segmentation method based on the fish fillet centroid, point cloud data and the main direction of the fish fillet; The left width is calculated from the preset average width calculation method according to the main direction of the fish fillet and the left point cloud; The right width is calculated from the preset average width calculation method according to the main direction of the fish fillet and the right point cloud; When the left width is greater than the right width, the fish head feature is identified from a preset fish head feature identification method according to the left point cloud and the fish fillet image; When the right width is greater than the left width, the fish head feature is identified from a preset fish head feature recognition method according to the right point cloud and the fish fillet image.

6. A method for identifying the placement position of tilapia fillets based on image recognition according to claim 5, characterized in that: The fish fillet head and tail identification method also includes: According to the main direction of the fish fillet and the point cloud data, the length of the fish fillet is matched from a preset fish fillet length recognition method; When the length of the fish fillet exceeds the preset length calculation interval, the left length is matched from a preset fish fillet length recognition method according to the main direction of the fish fillet and the left point cloud; According to the main direction of the fish fillet and the right point cloud, the right length is matched from a preset fish fillet length recognition method; Calculate the length difference according to the left length and the right length using a preset difference calculation method; Match the left cut point cloud from the preset point cloud cut method according to the length difference, the main direction of the fish fillet and the left point cloud; The left width is calculated from the preset average width calculation method according to the main direction of the fish fillet and the left interception point cloud; According to the length difference, the main direction of the fish fillet and the right point cloud, the right point cloud is matched from the preset point cloud interception method; The left width is calculated from the preset average width calculation method according to the main direction of the fish fillet and the right intercept point cloud.

7. A method for identifying the placement position of tilapia fillets based on image recognition according to claim 5, characterized in that: The fish fillet head and tail identification method also includes: Calculating a width ratio from a preset width ratio calculation method according to the left width and the right width; When the width ratio is less than the preset head-tail ratio, the upper left point cloud and the lower left point cloud are matched from the preset belly-back point cloud segmentation method based on the centroid of the fish fillet, the left point cloud and the main direction of the fish fillet; The upper left thickness is calculated from a preset average thickness calculation method according to the upper left point cloud, and the lower left thickness is calculated from a preset average thickness calculation method according to the lower left point cloud; Based on the centroid of the fish fillet, the right point cloud and the main direction of the fish fillet, the upper right point cloud and the lower right point cloud are matched from the preset abdominal and dorsal point cloud segmentation method; The upper right thickness is calculated from a preset average thickness calculation method according to the upper right point cloud, and the lower right thickness is calculated from a preset average thickness calculation method according to the lower right point cloud; Matching the minimum thickness from a preset minimum thickness matching method according to the upper left thickness, the lower left thickness, the upper right thickness and the lower right thickness; Match the fish head point cloud from the preset point cloud database based on the minimum thickness; The fish head features are identified from a preset fish head feature recognition method according to the fish head point cloud and the fish fillet image.

8. A method for identifying the placement position of tilapia fillets based on image recognition according to claim 7, characterized in that: The invention also includes a method for adjusting the head and tail of a fish fillet, wherein the method comprises: Matching the fish head orientation from a preset fish head orientation matching method based on the fish head feature; Matching an adjustment angle from a preset adjustment angle matching method according to the fish head orientation and the preset fish fillet processing direction; Matching the adjustment stroke from a preset adjustment stroke matching method according to the fish fillet mass center and the adjustment angle; The preset adjustment device is controlled according to the adjustment stroke to direct the fish head toward the fish fillet processing direction.

Images