An intelligent segmentation method for salmon based on line laser scanning

Abstract

An intelligent salmon segmentation method based on line laser scanning, including a point cloud acquisition module, a point cloud segmentation module, and a segmentation planning module: The point cloud acquisition module is used to generate a complete point cloud model, including line laser scanning, point cloud data processing, and point cloud stitching; The point cloud segmentation module is used for rough segmentation of the point cloud model and generating labels, and its segmentation network is a deep learning network improved by combining PointNet++ and Transformer; The segmentation planning module is used for mapping labels and generating a plan, which optimizes the rough segmentation model by removing abnormal points and maps the cutting on the basis of three-dimensional reconstruction. The present invention nondestructively acquires the surface of raw materials through line laser scanning, combines point cloud stitching, point cloud processing, and deep learning for point cloud segmentation, introduces Transformer to improve features at the same time, and optimizes and visually reflects the cutting plan through abnormal point removal and three-dimensional reconstruction. This system accurately identifies salmon of different sizes and completes part segmentation.

Description

Technical Field

[0001] The present invention belongs to the field of aquatic product processing, and particularly relates to an intelligent segmentation method for salmon based on line laser scanning. Background Art

[0002] Salmon is a kind of cold-water marine fish, and the fish meat in different parts has different tastes, values and uses. The segmentation of salmon is of great significance for enhancing the value of salmon itself. At present, the segmentation mainly relies on manual work, resulting in low production efficiency and low meat yield. How to quickly, non-destructively and accurately segment salmon urgently needs to be solved. For similar segmentation problems, the patent "A method and system for planning cutting paths based on fish body characteristics" (CN 112669336A) proposes a method for planning cutting paths of fish bodies based on image data segmentation and deep learning. This method inputs the image data of the fish body to be segmented into a deep learning model to obtain the segmentation result and plan the cutting path. However, this method ignores the three-dimensional geometric characteristics of fish, and can only perform simple segmentation. Since the shapes of different fish bodies vary greatly, the segmentation results lacking three-dimensional geometric characteristics are not accurate. Especially for the cutting of fish meat, the boundaries of fish meat in each part are not obvious, and traditional image data cannot accurately distinguish them, and three-dimensional characteristics are needed.

[0003] At present, the line laser measurement technology is widely used in the assembly line. It can non-destructively collect the point cloud data of materials and provide three-dimensional information. With the development of artificial intelligence, the point cloud segmentation technology integrating deep learning has also begun to be combined with the field of aquatic product processing, but there are still the following disadvantages: (1) Due to its characteristics, the line laser can only collect surface information and cannot collect all the point clouds of the material; (2) The point cloud segmentation task requires all the data characteristics of the original point set, while general deep learning algorithms will lose the original characteristics due to multi-layer feature extraction, resulting in low segmentation accuracy. And due to the characteristics of permutation invariance and rotation invariance of point cloud data, general feature extraction algorithms cannot be applied; (3) There will still be a small number of abnormal points after point cloud segmentation, which will affect the segmentation result.

[0004] Therefore, this paper proposes an intelligent segmentation system for salmon based on line laser scanning. The raw material point cloud collected by the line laser is improved by using point cloud processing and point cloud stitching; for point cloud segmentation, a raw material point cloud segmentation network based on PointNet++ geometric deep learning and improved by introducing the Transformer model is designed to generate a rough point cloud segmentation model; a method for removing abnormal points based on point cloud label discrimination is designed to optimize the rough segmentation model and improve the segmentation accuracy, making the cutting plan more reliable. This system can effectively improve the accuracy and objectivity of salmon part segmentation. Summary of the Invention

[0005] In view of the deficiencies of the prior art, the present invention provides a salmon intelligent segmentation system based on line laser scanning, which includes a point cloud acquisition module, a point cloud segmentation module, and a segmentation planning module, realizing the rapid segmentation of different parts of salmon.

[0006] Technical solution of the present invention:

[0007] A salmon intelligent segmentation method based on line laser scanning, including a point cloud acquisition module, a point cloud segmentation module, and a segmentation planning module, the steps are as follows:

[0008] Step 1: Build a three-dimensional information acquisition platform for the raw material surface;

[0009] The three-dimensional information acquisition platform for the raw material surface mainly consists of an industrial computer, an encoder, a stepping motor, a conveyor belt, a dark box, a light source, and a line laser sensor; the dark box is installed on the conveyor belt, and a light source and a line laser sensor are arranged in the dark box; an encoder is installed on the conveyor belt, and the encoder is further connected to the stepping motor for driving the conveyor belt to run; the encoder and the line laser sensor are connected to the industrial computer;

[0010] Step 2: In the point cloud acquisition module, the line laser sensor is used to scan the salmon to be processed in real time to obtain the point cloud data of the salmon surface; based on the PCL (Point Cloud Library) data processing and ICP (Iterative Closest Point) point cloud stitching algorithm, a method for generating a salmon point cloud model is designed, and the conveyor belt no-load point cloud and the salmon running point cloud are processed and effectively stitched by ICP to generate a complete salmon point cloud model;

[0011] When the conveyor belt runs smoothly, the trigger mode is used to collect the salmon surface data. The line laser measurement of the line laser sensor adopts the laser triangulation reflection principle, and the measurement data is the point cloud data of the salmon in the Z-axis and X-axis directions. The data of the salmon in the Y-axis direction is supplemented according to the conveyor belt speed; the ICP point cloud stitching algorithm in the point cloud acquisition module is used to stitch the conveyor belt no-load point cloud and the salmon running point cloud;

[0012] Step 3: Design a salmon point cloud segmentation network model based on PointNet++ and combined with the Transformer model to optimize the feature extraction effect in the point cloud segmentation module, that is, introduce the Transformer model to extract features in the original network feature transfer layer SA (Set Abstraction layers), and use the Transformer to perform another feature extraction in the first feature extraction stage to strengthen the features. Input the salmon point cloud model generated in the point cloud acquisition module into the trained salmon point cloud segmentation network model to obtain the label classification of each point cloud in the salmon point cloud model, and generate a rough segmentation model;

[0013] The backbone network of the salmon point cloud segmentation network model is PointNet++, which includes hierarchical point set feature learning and feature propagation; in hierarchical point set feature learning, a Transformer model is introduced to optimize the extraction of local and global features of the point cloud. Hierarchical point set feature learning includes two sets of set abstraction layers SA. Each set abstraction layer SA includes a sampling layer, a grouping layer, and a feature layer. In set abstraction layer SA1, a Transformer model is introduced to optimize the original point set features of the salmon point cloud model. The steps are as follows:

[0014] Step 3.1.1 Set the sampling layer to use farthest point sampling (FPS) to select the center points of the point cloud clusters from the original point set of the salmon point cloud model;

[0015] Step 3.1.2 Set the grouping layer to select the neighborhood points of the sampling center points according to ball query, and form sub-point clouds from the selected neighborhood points and send them to the feature layer;

[0016] Step 3.1.3 Set the feature layer to use PointNet to extract the local feature information of each sub-point cloud, and PointNet extracts the features of the sampled center point set as the global feature of the point cloud, completing the downsampling and feature extraction of set abstraction layer SA1;

[0017] Step 3.1.4 Set the Transformer model to perform enhanced feature processing on the local features and the global feature of the point cloud in Step 3.1.3;

[0018] Set the sampling points in Step 3.1.1 as the original point set of set abstraction layer SA2, and repeat the above steps 3.1.2 and 3.1.3 once in sequence to complete the downsampling and feature extraction of set abstraction layers SA1 and SA2, obtaining two sets of downsampled point clouds containing local spatial features;

[0019] Feature propagation is used to transfer features back to the original point set. The specific steps are as follows:

[0020] Step 3.2.1 Generate interpolation features based on inverse distance weighted interpolation for the features obtained from set abstraction layer SA1 and set abstraction layer SA2, and use skip-link to splice and stack the interpolation features with the features of set abstraction layer SA;

[0021] Step 3.2.2 Set the stacked features to pass through unitPointNet (mlp) to perform feature extraction again;

[0022] Step 3.2.3 Repeat steps 3.2.1 - 3.2.2 three times to obtain the predicted scores of each point in 5 categories, take the maximum score as the category of the input point cloud, complete the segmentation of the salmon point cloud model, generate a rough segmentation model of the salmon, and generate labels; the point cloud label classification results in the rough segmentation model include the label of the upper fish back, the label of the fish belly, the label of the fish belly, the label of the rear section, and the label of the fish tail;

[0023] Step 4: In the segmentation planning module, use the ball pivoting algorithm to perform surface reconstruction on the salmon point cloud model generated in step 2, design an outlier removal method based on label discrimination, optimize the rough segmentation model obtained in step 3, and finally generate a segmentation plan;

[0024] The outlier removal method based on label discrimination in the segmentation planning module is as follows: The point labels in the rough segmentation model are set to 0, 1, 2, 3, 4 according to the classification results, corresponding to the fish back, fish belly, fish belly, rear section, and fish tail respectively. Designate any point as the center point, determine the radius K, compare the point labels within a distance of K with the center point. If the labels of other points are all different from it, then this point is identified as an outlier, otherwise it is a normal point. Perform this operation on all points until the rough segmentation model is optimized;

[0025] The surface reconstruction method is generated using the ball pivoting algorithm and includes: (1) Seed triangle selection, that is, determine a seed triangle according to the rules; (2) Triangular mesh expansion, define the side where the seed triangle is located as the expansion side, roll a ball with a radius of R along an expansion side until the ball touches a point and stops, connect this point to the expansion side to form a new triangular mesh, and define the newly added side on the mesh as the expansion side. Repeat the above until all expansion sides are completed to form a triangular mesh expansion, thereby generating a three-dimensional mesh model.

[0026] Among them, the method steps for generating a complete point cloud model designed based on PCL point cloud processing and ICP stitching algorithm are as follows:

[0027] Step 2.1 Write the salmon surface point cloud data collected by the line laser into a PCD point cloud model;

[0028] Step 2.2 Use PCL for data optimization and perform point cloud reduction using voxel filtering downsampling;

[0029] Step 2.3 Use the moving least squares method for point cloud smoothing to improve the point cloud accuracy;

[0030] Step 2.4 Use the ICP algorithm to stitch the point cloud of the conveyor belt running empty and the point cloud scanned during the running of the salmon;

[0031] Step 2.5 Use the threshold segmentation method to remove the interference data formed on the surface of the conveyor belts on both sides of the raw material, and form a complete salmon point cloud model.

[0032] Advantages of the present invention:

[0033] (1) To obtain the three-dimensional information of the raw material, in the point cloud acquisition module, the line laser scanning technology is used to obtain the point cloud information on the surface of the salmon without damage. At the same time, the PCL point cloud processing and ICP algorithm are used for splicing to improve the point cloud information of the raw material and remove the interference information, realizing the function of real-time generating a complete raw material point cloud model. Compared with the two-dimensional image, it can provide more accurate three-dimensional information of the material, improving the subsequent segmentation accuracy.

[0034] (2) The point cloud segmentation task requires regressing to the characteristics of the original point set. However, due to the multi-level downsampling of PointNet++, some features are inevitably lost. At the same time, due to the permutation invariance and rotation invariance of the point cloud data, general algorithms cannot extract its features. To solve such problems, in the raw material point cloud segmentation module, a point cloud segmentation network is designed. It is based on PointNet++ geometric deep learning, and the Transformer model is introduced in the first layer Set Abstraction of the original network framework to extract features. The attention mechanism of the Transformer model and its characteristics of being insensitive to data arrangement and quantity are used to optimize the feature extraction effect of the original SA. Compared with the original network, the segmentation accuracy is improved. At the same time, compared with the classic deep learning segmentation method combining images, the point cloud geometric deep learning combining three-dimensional features can more accurately achieve the point cloud segmentation of salmon parts.

[0035] (3) Aiming at the problem that there are still outliers in the rough point cloud segmentation model, in the segmentation planning module, the present invention proposes a method for removing outliers based on label discrimination, effectively removing outliers, optimizing the rough segmentation result, and improving the reliability of the segmentation scheme.

[0036] The present invention improves the efficiency and automation degree of salmon part segmentation, reduces the time and secondary pollution caused by manual segmentation, reduces the labor cost, and uses the line laser scanning technology, point cloud splicing and deep learning, combined with the three-dimensional characteristics of the fish body, and the segmentation result is more accurate and objective; the fine segmentation of different parts of salmon can improve the utilization rate and added value of salmon itself, and better meet the requirements of the consumer market for the fine processing of fish products, conforming to the development direction of industrial value addition; the present invention provides an effective reference for the fine segmentation of different parts of salmon, helping enterprises improve the processing efficiency and intelligent processing level of salmon. Brief Description of the Drawings

[0037] Figure 1 It is the overall system flow chart of the present invention;

[0038] Figure 2It is a diagram of a three-dimensional information acquisition system for the raw material surface;

[0039] Figure 3 It is a measurement schematic diagram;

[0040] Figure 4 It is a flow chart for processing point cloud data of the present invention;

[0041] Figure 5 It is a flow chart for making a data set;

[0042] Figure 6 It is a block diagram for point cloud segmentation of salmon;

[0043] Figure 7 It is a label map of salmon parts;

[0044] Figure 8 It is a diagram of a point cloud segmentation network model;

[0045] Figure 9 It is a flow chart for training a deep learning network and obtaining a rough segmentation model;

[0046] Figures 10(a), (b), and (c) are respectively a manual segmentation diagram, a PointNet++ segmentation diagram, and a PointNet++ segmentation diagram improved based on Transformer;

[0047] Figure 2 Among them: 1 industrial control computer; 2 encoder; 3 stepping motor; 4 conveyor belt; 5 dark box; 6 light source; 7 line laser sensor. Detailed implementation manner

[0048] In order to make the purpose, technical solution, and beneficial technical effects of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners, but the protection scope of the present invention is not limited to the following embodiments.

[0049] An intelligent grading system for salmon fillets proposed by the present invention, as Figure 1 shown, the technical solution of this embodiment includes the following steps:

[0050] Build a three-dimensional information acquisition system for the raw material surface;

[0051] As Figure 2 shown, the acquisition system consists of an industrial control computer, an encoder, a stepping motor, a conveyor belt, a dark box, a light source, and a line laser sensor. The speed of the conveyor belt is adjusted by the encoder. The raw material is placed on the conveyor belt, and the line laser scanner is installed on the support frame at the 1 / 2 position of the conveyor belt. The acquired raw material surface data information is transmitted to the computer data processing through an Ethernet connection method. The dark box is installed in the middle position of the frame, wrapping the scanner support frame, and two groups of strip light sources are built-in on both sides to irradiate both sides of the fish body respectively.

[0052] Example 1:

[0053] The present invention provides an example, in which under online laser scanning, the point cloud acquisition module generates a complete point cloud model, including the following steps:

[0054] In the point cloud acquisition module, a line laser sensor is used to perform real-time scanning on the salmon to be processed to obtain the point cloud data on the surface of the salmon. The method is as follows:

[0055] Start the line laser sensor and the conveyor belt. When the conveyor belt runs smoothly, place the raw material and collect the surface data of the fish body in trigger mode. The line laser measurement adopts the principle of laser triangulation reflection. In addition to the distance information (Z-axis) from the sensor to the measured surface, the controller can also calculate the position information (X-axis) along the laser line through the image information. In the two-dimensional coordinate system with the sensor as the origin, the profiler measures and outputs a set of two-dimensional coordinate values. The data of the raw material in the Y-axis direction can be determined by the conveyor belt speed and the time when the raw material triggers the line laser sensor.

[0056] The conveyor belt point cloud and the raw material point cloud are processed by PCL point cloud and effectively spliced by ICP to generate a complete raw material point cloud model. The specific steps are as follows:

[0057] Step A1: The line laser first scans and records the empty conveyor belt point cloud, and then scans the surface information of the raw material. The point cloud storage type is ASCII code type and is written as a PCD format file;

[0058] Step A2: To improve the algorithm calculation speed, voxel filtering is used for downsampling. According to the number of the original point cloud, the Leaf Size of the voxel filtering is set to 0.1f, that is, the number of point clouds is reduced to one-tenth of the original, which improves the algorithm speed while still retaining the geometric features of the point cloud;

[0059] Step A3: Use the moving least squares method to perform point cloud smoothing processing, and set the K-nearest neighbor radius for fitting to 0.005m to remove irregular data;

[0060] Step A4: Use the ICP algorithm to splice the two parts of the point cloud, calculate the LOSS and minimize the LOSS;

[0061] Step A5: Use the threshold segmentation method to remove the interference data formed on the surface of the conveyor belt on both sides of the raw material to generate a complete point cloud model. The calculation process is as follows:

[0062]

[0063] Δh1 = |M1 - M2|

[0064] Δh2 = |M2 - M3|

[0065] In the formula, [M2] is the array of the radial cross-sections of the fish body, [M1] and [M3] are the arrays of the radial cross-sections of the conveyor belts on both sides of the raw material, Δh1 is the absolute value of the difference between adjacent elements of the height value of the right endpoint of [M1] and the height value of the left endpoint of [M2], Δh2 is the absolute value of the difference between adjacent elements of the height value of the right endpoint of [M2] and the height value of the left endpoint of [M3], [M] is the array after threshold segmentation, and T is the segmentation threshold.

[0066] Embodiment 2

[0067] The technical solution of the present invention provides an embodiment. The point cloud segmentation module is based on the PointNet++ geometric deep learning network, and is improved by introducing Transformer to segment the input raw material point cloud to generate a rough segmentation result, including the following steps:

[0068] The production process of the dataset required for training the deep learning network in S1 is as follows, as Figure 5 shown:

[0069] S1.1 Use line laser scanning to obtain the point cloud data of salmon raw materials of different specifications;

[0070] S1.2 Use the point cloud processing method in step 2 to preprocess the data;

[0071] S1.3 Import the preprocessed point cloud data into the CloudCompare software, and segment different parts of the raw material according to the three-dimensional characteristics of the fish body and key point positioning, as Figure 6 shown, and manually mark the upper fish back "0", fish belly "1", fish belly "2", rear section "3" and fish tail "4" to generate labels, as Figure 7 shown.

[0072] S1.4 Export the original point cloud data and the marked point cloud data to generate the dataset required for training the deep learning network.

[0073] The process of building a deep learning network model with PointNet++ as the backbone network and improved by using the Transformer feature extraction module in S2 is as follows, and its main structure is as Figure 8 shown:

[0074] S2.1.1 Set the sampling layer to use farthest point sampling (FPS) to select 512 points from the original point set as the center points of the point cloud clusters;

[0075] S2.1.2 Set the grouping layer to select the neighborhood points of the sampling center points according to ball query (Ball query), set the radius to 0.2, and send the selected sub-point clouds into the feature layer;

[0076] S2.1.3 Set the feature layer to use PointNet to extract the local feature information of each sub-point cloud. PointNet extracts the features of the sampled central point set as the global feature of the point cloud, completing the downsampling and feature extraction of the SA1 layer;

[0077] S2.1.4 Set Transformer to perform enhanced feature processing on the local features and the global feature of the point cloud in step 3.2.3;

[0078] Set the 512 sampled points in S2.1.1 as the original point set of the SA2 layer, and repeat steps S2.1.1, S2.1.2, and S2.1.3 once in sequence. The radius of the grouping layer of SA2 is set to 0.4, completing the downsampling and feature extraction of the SA2 layer, and obtaining 2 groups of point clouds containing local spatial features after downsampling.

[0079] S2.2 Further, the hierarchical feature propagation is used to transfer features back to the original point set. The specific steps are as follows.

[0080] S2.2.1 Generate interpolation features by inverse distance weighted interpolation based on the features obtained by SA1 and SA2, and splice and stack the interpolation features with the SA features using skip-link;

[0081] The interpolation formula based on inverse distance is as follows:

[0082]

[0083] In the formula, k is the number of neighborhood points (by default, we use p = 2, k = 3), j is the index number of the point cloud, and C is an arbitrary constant.

[0084] S2.2.2 Set the stacked features to pass through unit PointNet (MLP) to perform feature extraction again: complete the full connection layer processing through data projection, max pooling operation, and MLP;

[0085] S2.2.3 Repeat the above steps 3 times to obtain the prediction scores of each point in 5 categories (fish back, fish belly, fish flank, rear section, fish tail), take the maximum score as the input point cloud category, complete the segmentation of the raw material point cloud model, and generate labels.

[0086] The Transformer is a model based on the multi-head attention mechanism. Essentially, it has an Encoder-Decoder structure, with the core being the self-attention module in the Encoder-Decoder. The operations of this model can be executed in parallel regardless of the order and it has a strong ability to capture local features. In the point cloud segmentation task, all features of the original point set need to be obtained. However, due to the layer-by-layer downsampling and feature extraction in PointNet++, there are still problems with the loss of features of the original point set. Moreover, due to the characteristics of permutation invariance and rotation invariance of point cloud data, general algorithms cannot extract its features. Since the Transformer is not sensitive to the quantity and arrangement of data, it can be used for feature extraction of point cloud data. In the two-layer SA, the first layer of SA will be used as the input of the second layer of SA. Considering the running speed and the possibility of overfitting the model due to multiple feature extractions, introducing the Transformer module in the first layer of SA can improve the performance of the original network.

[0087] The process of training the S3 deep learning network and obtaining the classification results of the part point cloud is as Figure 9 shown, specifically including the following:

[0088] S3.1 Use the point cloud data obtained in step S3.1 to train the network designed in step S3.2, and input the labeled salmon point cloud model into the improved deep learning network.

[0089] S3.2 By marking a large number of real data of the upper back, belly, belly, posterior segment, and tail of the fish as training data, train the improved fish meat segmentation network in step S3.2. Use the metric space distance to learn local features by increasing the context scale, and appropriately adjust the network parameters according to the prediction results to make them close to the best prediction results. Finally, complete the network training.

[0090] S3.3 Use the neural network trained in step S3.2 to input the real-time collected point cloud data, that is, the unlabeled salmon point cloud model data, and predict the label classification results of the upper back, belly, belly, posterior segment, and tail of the salmon point cloud model to generate a rough segmentation model.

[0091] Example 3:

[0092] The present invention provides an embodiment. The segmentation planning module performs surface reconstruction on the point cloud data obtained in the point cloud acquisition module using the rolling ball method; uses the outlier removal method based on label discrimination to optimize the rough segmentation model obtained in the point cloud segmentation module and generate a segmentation plan, including the following specific steps:

[0093] Step 1: Perform three-dimensional reconstruction on the point cloud model generated by the point cloud acquisition module to generate a mesh model. In this step, the concave hull reconstruction rolling ball method is used for surface reconstruction of the point cloud. Similarly, the rolling ball method here is a method based on the PCL library, and the radius R of the rolling ball, i.e., Alshape, needs to be set. Generally, the smaller Alshape is, the finer the model is, but it needs to be considered in combination with the actual point cloud distribution. According to the resolution of the line laser sensor and the conveyor belt speed, Alshape is set to 7.2 here;

[0094] Step 2: According to the density of the raw material point cloud, set a point in the origin set as the center point and determine the radius K. Here, K is set to 0.1. Compare the labels of all points within a distance of 0.1 with the label (one of 0, 1, 2, 3, 4) of the center point. If the labels of other points do not match it, then this point is identified as an abnormal point, otherwise it is a normal point, and so on until the optimization of the rough segmentation model is completed;

[0095] Step 3: Map the point cloud labels generated in the point cloud segmentation module to the corresponding three-dimensional mesh model, color different parts, obtain an intuitive segmentation result, and generate a segmentation scheme.

[0096] It can be seen from the comparison in Figures 10(a), 10(b), and 10(c) that the segmentation result of the improved PointNet++ based on Transfomer proposed by the present invention is closer to the human semantic segmentation of objects compared with the original network, and the boundary is more obvious, without large-area over-segmentation and under-segmentation.

[0097] Table 1 Evaluation and comparison of different segmentation methods using the Rand index

[0098]

[0099] Table 1

[0100] The Rand index measures the similarity between two segmentations of the same shape. The above results are the values generated by comparing the segmentation results of the two networks with the original segmentation result, that is, the differences between the segmentation results before and after network improvement and the human segmentation result. It can be seen from the comparison data in Table 1 that the difference between the method of the present invention and the original segmentation result is smaller, and the performance of the improved PointNet++ network based on Transformer proposed by the present invention is improved compared with the original network PointNet++.

Claims

1. An intelligent segmentation method for salmon based on line laser scanning, characterized in that, The intelligent salmon segmentation method based on line laser scanning includes a point cloud acquisition module, a point cloud segmentation module, and a segmentation planning module. The steps are as follows: Step 1: Build a three-dimensional information acquisition platform for the raw material surface; The three-dimensional information acquisition platform for the raw material surface mainly consists of an industrial control computer, an encoder, a stepping motor, a conveyor belt, a dark box, a light source, and a line laser sensor; the dark box is installed on the conveyor belt, and a light source and a line laser sensor are arranged inside the dark box; an encoder is installed on the conveyor belt, and the encoder is further connected to the stepping motor to drive the conveyor belt to run; the encoder and the line laser sensor are connected to the industrial control computer; Step 2: In the point cloud acquisition module, the line laser sensor is used to perform real-time scanning on the salmon to be processed to obtain the point cloud data of the salmon surface; based on PCL data processing and the ICP point cloud stitching algorithm, a method for generating a salmon point cloud model is designed, and the conveyor belt no-load point cloud and the salmon running point cloud are processed and effectively stitched by ICP to generate a complete salmon point cloud model; When the conveyor belt runs smoothly, the trigger mode is used to collect the surface data of the salmon. The line laser measurement of the line laser sensor adopts the laser triangulation reflection principle, and the measurement data is the point cloud data of the salmon in the Z-axis and X-axis directions. The data of the salmon in the Y-axis direction is supplemented according to the conveyor belt speed; the ICP point cloud stitching algorithm in the point cloud acquisition module is used to stitch the conveyor belt no-load point cloud and the salmon running point cloud; Step 3: In the point cloud segmentation module, a salmon point cloud segmentation network model based on PointNet++ and combined with the Transformer model to optimize the feature extraction effect is designed, that is, the Transformer model is introduced into the original network feature transfer layer SA to extract features, and the Transformer is used for another feature extraction in the first feature extraction stage to strengthen the features. The salmon point cloud model generated in the point cloud acquisition module is input into the trained salmon point cloud segmentation network model to obtain the label classification of each point cloud in the salmon point cloud model, and a rough segmentation model is generated; The backbone network of the salmon point cloud segmentation network model is PointNet++, which includes hierarchical feature learning and hierarchical feature propagation; the Transformer model is introduced in the hierarchical feature learning to optimize the extraction of local and global features of the point cloud. The hierarchical feature learning includes 2 sets of set abstraction layers SA, and each set abstraction layer SA includes a sampling layer, a grouping layer, and a feature layer. The Transformer model is introduced into the set abstraction layer SA1 to optimize the original point set features of the salmon point cloud model; the following steps are included: Step 3.1.1 Set the sampling layer to use farthest point sampling to select the center points of the point cloud clusters from the original point set of the salmon point cloud model; Step 3.1.2 Set the grouping layer to select the neighborhood points of the sampling center points according to spherical search, and form sub-point clouds from the selected neighborhood points and send them to the feature layer; Step 3.1.3 Set the feature layer to use PointNet to extract the local feature information of each sub-point cloud. PointNet extracts the features of the sampled central point set as the global feature of the point cloud, and completes the downsampling and feature extraction of the set abstraction layer SA1; Step 3.1.4 Set the Transformer model to perform enhanced feature processing on the local features and the global feature of the point cloud in Step 3.1.3; Set the sampled points in Step 3.1.1 as the original point set of the set abstraction layer SA2, and repeat the above Steps 3.1.2 and 3.1.3 once in sequence to complete the downsampling and feature extraction of the set abstraction layers SA1 and SA2, and obtain 2 groups of point clouds containing local spatial features after downsampling; Hierarchical feature propagation is used for feature transfer to return to the original point set. The specific steps are as follows. Step 3.2.1 Generate interpolation features by inverse distance weighted interpolation for the features obtained from the set abstraction layer SA1 and the set abstraction layer SA2, and splice and stack the interpolation features with the features of the set abstraction layer SA using skip-link; Step 3.2.2 Set the stacked features to pass through unitPointNet to implement feature extraction again; Repeat Steps 3.2.1 - 3.2.2 a total of 3 times to obtain the prediction scores of each point in 5 classes, take the maximum score as the input point cloud category, complete the segmentation of the salmon point cloud model, generate a rough segmentation model of the salmon, and generate labels; the point cloud label classification results in the rough segmentation model include the label of the upper fish back, the label of the fish belly, the label of the fish belly, the label of the rear section, and the label of the fish tail; Step 4: In the segmentation planning module, use the rolling ball method to perform surface reconstruction on the salmon point cloud model generated in Step 2, design an outlier removal method based on label discrimination to optimize the rough segmentation model obtained in Step 3, and finally generate a segmentation plan; The outlier removal method based on label discrimination in the segmentation planning module is as follows: the point labels in the rough segmentation model are set to 0, 1, 2, 3, 4 according to the classification results, corresponding to the fish back, fish belly, fish belly, rear section, and fish tail respectively. Designate any point as the center point, determine the radius K, compare all the point labels within a distance of K with the center point. If the labels of other points are all different from it, then this point is identified as an outlier, otherwise it is a normal point. All points are operated in this way until the rough segmentation model is optimized; The surface reconstruction method is generated using the rolling ball method, including: (1) Seed triangle selection, that is, determine a seed triangle according to the rules; (2) Triangular mesh expansion. Define the side where the seed triangle is located as the expansion side. Roll a rolling ball with a radius of R along an expansion side until the rolling ball touches a point and stops. Connect this point to the expansion side to form a new triangular mesh. The newly added side on the mesh is also defined as the expansion side. Repeat the above until all expansion sides are completed to form triangular mesh expansion, thereby generating a three-dimensional mesh model; The method steps for generating a complete point cloud model designed based on PCL point cloud processing and ICP stitching algorithm are as follows: Step 2.1 Write the salmon surface point cloud data collected by the line laser into a PCD point cloud model; Step 2.2 Optimize the data using PCL and downsample the point cloud using voxel filtering for point cloud reduction; Step 2.3 Smooth the point cloud using the moving least squares method to improve the point cloud accuracy; Step 2.4 Use the ICP algorithm to stitch the point cloud of the conveyor belt running empty and the scanned point cloud of the salmon running; Step 2.5 Use the threshold segmentation method to remove the interference data formed on the surfaces of the conveyor belts on both sides of the raw material to form a complete point cloud model of the salmon.

Images