Nondestructive method for quality detection of crabs

Abstract

The application relates to the technical field of food processing, and discloses a crab quality nondestructive detection method, which comprises the following steps: collecting basic information, two-dimensional image data and three-dimensional point cloud data of crabs to be detected; performing image processing on the two-dimensional image to extract two-dimensional features, and performing analysis on the three-dimensional point cloud to extract three-dimensional features; and calculating three-dimensional apparent density of the crabs based on the three-dimensional features and the basic information. The basic information, the two-dimensional features, the three-dimensional features, crab volume features and the three-dimensional apparent density are spliced together to form multi-source fusion features, which are taken as inputs of a quality detection model, and quality indexes of the crabs to be detected are output, so that nondestructive quality detection is realized.

Description

Technical Field

[0001] This invention belongs to the technical field of food processing, specifically relating to a non-destructive testing method for crab quality. Background Technology

[0002] The commercial value of river crabs is closely related to their weight, size, sex, plumpness, gonadal development, hepatopancreatic fullness, and overall edible content. Current production grading is usually based on weight, sex, shell length, shell width, appearance integrity, or traditional plumpness. These indicators are easy to obtain, but they mainly reflect external specifications and overall size, and are difficult to fully describe the true three-dimensional morphology, dorsal and ventral thickness distribution, and abdominal morphological differences of river crabs.

[0003] Traditional plumpness is often expressed as the ratio of body weight to the cube of shell length. This method simplifies the morphology of crabs to a one-dimensional scale correction, which cannot reflect three-dimensional spatial information such as carapace ridges, ventral undulations, abdominal smoothness, and thickness difference between the center and edge, and therefore cannot comprehensively reflect the quality of crabs. Overall size is usually measured by the displacement method, but this method is complex and inefficient, and is not suitable for online grading on the production line. While existing visual grading methods can achieve sex identification, shell length and width measurement, and appearance grading, their ability to non-destructively screen gonads, hepatopancreas, and overall edible part development level is still limited.

[0004] Therefore, there is a need for an automated method that can use three-dimensional point clouds to obtain information on the volume, thickness field, and abdominal morphology of crabs and use it for auxiliary screening of the development level of the internal edible parts. Summary of the Invention

[0005] This invention provides a non-destructive testing method for crab quality, which solves the technical problems of existing methods such as complex operation, low efficiency, and inability to comprehensively reflect the quality of crabs.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A non-destructive testing method for crab quality includes the following steps: Step 1: Collect basic information, two-dimensional image data, and three-dimensional point cloud data of the crab species to be tested; Step 2: Perform image processing on the two-dimensional image to extract two-dimensional features, and analyze the three-dimensional point cloud to extract three-dimensional features. Then, based on the three-dimensional features and basic information, calculate the three-dimensional apparent density of crabs. Step 3: Combine the basic information, two-dimensional features, three-dimensional features, crab volume features, and three-dimensional apparent density to form a multi-source fusion feature. Use this as the input to the quality inspection model and output the quality index of the crab to be tested. This can achieve non-destructive quality inspection in different application scenarios.

[0007] Furthermore, in step two, the collected back 3D point cloud data and front 3D point cloud data are first preprocessed, then the preprocessed back 3D point cloud and front 3D point cloud are projected onto a horizontal reference plane, and the projected area is divided into uniform grids, which are denoted as the effective grid sets of the front surface. Effective grille set on the back , Then calculate the effective set of rear grilles. The highest value of each grid region within the interior, and the effective grid set on the ventral side. The minimum value of each grid region is used, and then the height parameter of the three-dimensional feature volume is calculated using these maximum and minimum values ​​respectively. ventral undulation Integral features of backside point clouds Ventral point cloud integral features Local thickness Average thickness High quantile thickness Center-edge thickness difference Carapace center-edge height difference ; Simultaneously, the projected area of ​​the 3D point cloud of the abdomen onto the horizontal reference plane is calculated, and this area is used as the 3D feature ventral surface projection area. Furthermore, by using slice projection, an abdominal fitting curve is constructed, and the radius of curvature of the three-dimensional abdominal feature is calculated. Abdominal curve fitting error , Finally, the above parameters are concatenated to obtain the three-dimensional features. : .

[0008] Furthermore, three-dimensional features Input the volume prediction model to obtain the crab volume feature V, and then use the following formula to calculate the three-dimensional apparent density ρ. Where W represents the weight information in the basic information.

[0009] Furthermore, the lowest value corresponding to each grid within the ventral 3D point cloud region is taken as the height of that grid, and the integral feature of the ventral point cloud is calculated using the following formula. , in, This represents the height within the i-th,j-th ventral grid unit; Indicates the reference height on the ventral side; Indicates the set of effective grids on the ventral side. and These represent the side lengths of the grid element in the X and Y directions, respectively; Using the highest value corresponding to each grid in the back 3D point cloud region as the height of that grid, the integral feature of the back point cloud is calculated using the following formula. , in, Indicates the height within the i-th, j-th rear grille unit; Indicates the reference height on the back side; Indicates the effective set of grilles on the back. and These represent the side lengths of the grid element in the X and Y directions, respectively; Furthermore, identify the effective grid set on the ventral side. Effective grille set on the back Repeating grids are denoted as identical grids. The difference between the highest and lowest values ​​corresponding to each identical grid is the local thickness. All local thicknesses The average value is the average thickness. These local thicknesses The 90th percentile is the high quantile thickness. , Calculate the center-edge thickness difference using the following formula. , in, This indicates the local thickness within the central area of ​​the main body of the crab shell. The average value, The thickness of the local area within the edge region of the main body of the crab shell. The average value; Calculate the height difference between the center and edge of the carapace using the following formula. , in, This represents the average height of each grid within the central area of ​​the crab shell's main body. This represents the average height of each grid within the edge region of the main body of the crab shell.

[0010] Furthermore, in calculating the radius of curvature of the abdomen... Using the projection center of the ventral 3D point cloud onto the horizontal reference plane as a reference, a cross-sectional strip is selected along the head-to-tail direction of the crab. This cross-sectional strip has a preset width. The cross-sectional strip is then projected onto a vertical plane formed by the head-to-tail direction and the Z-axis, thereby obtaining a 2D cross-sectional point set of the abdomen. These 2D cross-sectional points are then fitted to obtain a fitted curve of the abdomen, the curvature of which, or the equivalent radius of the arc, is the radius of curvature of the abdomen. ; The abdominal curve fitting error is calculated using the following formula. , in, Indicates the first k The distance from each two-dimensional cross-sectional point on the abdomen to the fitted curve on the abdomen. n This indicates the number of points involved in the fitting process.

[0011] Furthermore, in step two, the main body region M of the crab shell is first segmented from the acquired two-dimensional image of the back. The principal axis length, secondary axis length, shell area, shell color parameters, shell shape parameters, and shell surface texture parameters of the main body of the crab shell are calculated. Then, these parameters are stitched together to obtain the two-dimensional features. .

[0012] Furthermore, when calculating the shell color parameters and shell texture parameters, the main body region M of the crab shell is eroded inward according to a preset pixel distance or preset ratio to obtain the inward-shrinking crab shell region. From the inward-shrinking crab shell area Extract shell color parameters, including Lab and HSV color parameters, where L* represents the shell interior metric, a* represents the red-green chromaticity, b* represents the yellow-blue chromaticity, and S... HSV Indicates color saturation; shrink the crab shell area The corresponding color image is converted into a grayscale image, and a grayscale co-occurrence matrix is ​​constructed. Then, based on the grayscale co-occurrence matrix, shell texture parameters are calculated, including texture contrast, texture homogeneity, texture correlation, and texture energy. The roundness Rc is calculated using the following formula to characterize the shell morphology parameters. in, Represents the shell area. This represents the perimeter of the outer contour of the main body area of ​​the crab shell; The shell area is obtained by calculating the actual area enclosed by the outer contour of the main body region M of the crab shell, or the projected area corresponding to the main body region M of the crab shell. .

[0013] Furthermore, we first construct the multi-source fusion feature X using the following formula. Where W represents weight information from the basic information, and S represents gender information from the basic information. Representing two-dimensional features, V represents the three-dimensional feature, V represents the volume feature of the crab shell, and ρ represents the three-dimensional apparent density. Then, using the multi-source fusion feature X as input, and the gonad quality, hepatopancreas quality, overall edible portion quality, and their corresponding gonad index, hepatopancreas index, and overall edible portion index as output, the quality detection model is trained. Then, the trained quality inspection model was used to test the gonad quality, hepatopancreas quality, overall edible portion quality, and their corresponding gonad index, hepatopancreas index, and overall edible portion index of the crabs to be tested. Based on the empirical values, the crabs to be tested were graded.

[0014] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention uses a 3D camera to acquire point cloud data of river crabs. Based on the registration and rasterization processing of dorsal and ventral point clouds, a three-dimensional feature calculation method suitable for crab morphology analysis is constructed, which can calculate the overall volume of river crabs quickly and without loss.

[0015] 2. This invention uses a 3D camera to acquire point cloud data of river crabs, which can extract the three-dimensional appearance features of river crabs, further quantify the plumpness and spatial size of river crabs, make up for the morphological and structural differences that traditional detection indicators such as sex and weight cannot describe, and can quickly determine the quality based on pure machine vision methods.

[0016] 3. This invention establishes a mathematical model based on weight, sex, 2D images, and 3D point clouds, enabling non-destructive and rapid detection of the gonads and hepatopancreas quality of river crabs, providing effective information for the refined grading and screening of river crabs. Furthermore, it can be used to develop various refined non-destructive testing methods for river crabs based on different production scenarios. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall process of the present invention; Figure 2 This is a schematic diagram of the three-dimensional point cloud data acquisition process of the present invention; Figure 3 This is a schematic diagram of the two-dimensional feature calculation process of the present invention; Figure 4 This is a schematic diagram of some three-dimensional features obtained by uniform gridding calculation according to the present invention; Figure 5 This is a schematic diagram of some of the three-dimensional features obtained by the present invention based on projection calculation of a horizontal reference plane; Figure 6 This is a schematic diagram illustrating the process of obtaining quality indicators of crabs to be tested using a quality detection model, as described in this invention. Detailed Implementation

[0018] To make the technical means, creative features, objectives and effects of the present invention easier to understand, the following embodiments, in conjunction with the accompanying drawings, specifically illustrate the non-destructive testing method for crab quality of the present invention. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0019] like Figure 1 As shown, this invention provides a non-destructive testing method for crab quality. First, basic information, two-dimensional image data, and three-dimensional point cloud data of the crab to be tested are collected. Then, image processing is performed on the two-dimensional image to extract two-dimensional features, and the three-dimensional point cloud is analyzed to extract three-dimensional features. Based on the three-dimensional features and basic information, the three-dimensional apparent density of the crab is calculated. Finally, the basic information, two-dimensional features, three-dimensional features, crab volume features, and three-dimensional apparent density are stitched together to form a multi-source fusion feature, which is used as the input to the quality testing model and outputs the quality indicators of the crab to be tested, thus achieving non-destructive quality testing.

[0020] Specifically as follows: S1. Collect basic information, two-dimensional images, and three-dimensional point cloud data of crab samples. S11. Collection of Basic Information Grab the crab sample to be tested, remove obvious moisture, mud or other debris from the sample surface, and put the crab in a relatively stable posture. Obtain the weight information of each crab using a weighing device, denoted as W. The other parameter of the basic information, gender information, is obtained by image processing of the abdominal image, as shown in step S12 below. Of course, it can also be determined manually.

[0021] S12. Acquisition of Two-Dimensional Images Two-dimensional color images of river crabs are acquired using an industrial camera, including images of the back and abdomen. The back image is used to extract two-dimensional features of the crab shell area, such as shell length, shell width, shell area, shell color, and shell texture. The abdomen image can be used for manual judgment or model identification of the sex of the river crab. The sex information is denoted as S and is represented by a numerical code, where 0 represents female crab and 1 represents male crab.

[0022] S13. Acquisition of 3D point cloud data like Figure 2 As shown, point clouds of the back and belly of the crab were collected using a 3D camera.

[0023] S131. 3D Camera Setup. An Intel RealSense D455 depth camera was used as the 3D point cloud acquisition device. This camera can acquire color and depth images. During acquisition, the color image resolution was set to 1280×720, and the frame rate was 15 fps; the depth image resolution was set to 848×480; the effective depth range was set to 0.15–1.20 m. The acquired depth frames were sequentially subjected to threshold filtering, disparity domain transformation, spatial filtering, temporal filtering, inverse disparity transform, and hole filling to reduce depth noise and minimize invalid depth regions.

[0024] S132. Camera Calibration and Intrinsic Parameter Determination. The ChArUco calibration board is used for camera pose estimation. The calibration board consists of 10×7 squares with a side length of 19.0 mm, and the ArUco markers have a side length of 14.0 mm. During calibration, multiple color images containing the ChArUco calibration board are acquired. ArUco markers are detected and interpolated to obtain ChArUco corner points. When the number of valid corner points reaches a set threshold, the frame is saved. After accumulating at least 15 frames, the camera intrinsic parameter matrix and distortion coefficients are calculated using the ChArUco calibration algorithm and saved as the calibration parameter file K for subsequent acquisitions. Each time the point cloud is saved, the corresponding calibration board rotation vector, translation vector, camera intrinsic parameter matrix, distortion coefficients, ROI coordinates, and depth validity ratio are simultaneously saved, forming a metadata file corresponding to the point cloud for subsequent point cloud coordinate transformation and quality control. The saved calibration parameter file matrix K is shown below.

[0025] S133. Point Cloud Data Acquisition. Point clouds were acquired for each crab in two poses: carapace facing upwards and plastron facing upwards. Dorsal and ventral point clouds were obtained separately and paired using the same sample number and BACK / ABD identifiers. When acquiring the dorsal point cloud, the crab's carapace was positioned facing the 3D camera; this dorsal point cloud represents the spatial morphology of the upper surface of the carapace. When acquiring the ventral point cloud, the crab's plastron was positioned facing the 3D camera; this ventral point cloud represents the morphology of the lower surface of the plastron and the abdominal region. Image acquisition can be performed in manual or automatic modes.

[0026] Manual acquisition: Select the ROI region containing a single crab on the real-time color image, and extract the corresponding point cloud from the aligned 3D vertex image according to the ROI coordinates. Remove invalid points with a depth less than 0.15m or greater than 1.20m to obtain the ROI point cloud of a single crab, and save it in PLY format.

[0027] Automatic acquisition: After the real-time color image is successfully displayed, the image and point cloud data of that frame are recorded by the YOLO model. The YOLO image is used to automatically select the crab sample, thereby obtaining the ROI area. Subsequent operations are the same as manual selection.

[0028] After acquiring 2D images and 3D point cloud data, the crab samples were dissected to obtain gonadal and hepatopancreatic mass. These two masses were then added together to determine the true value of the overall edible portion mass. Dissection measurements were only used to obtain supervised labels during model training; the trained model can then be used for non-destructive measurements on the crabs. Gonadal mass is denoted as G, hepatopancreatic mass as H, and overall edible portion mass as E. The gonadal index, hepatopancreatic index, and overall edible portion index were calculated based on body weight. in, Indicates gonadal index, Indicates the liver and pancreas index, This indicates the overall edible portion index.

[0029] S2. Processing and Feature Calculation of Two-Dimensional Images S21. Segmentation of the crab shell region The crab shell segmentation model can employ the YOLOv11-seg model, or other instance segmentation or semantic segmentation models. During the training phase of the segmentation model, some images of the crab's dorsal surface are manually labeled. The labeled area is the main region of the crab shell; legs, claws, and background areas are not considered part of the main shell region. After labeling, the images and corresponding labels are used to train the crab shell segmentation model. The main crab shell region refers to the main visible area of ​​the carapace in the crab's dorsal image, including the central area of ​​the carapace and its continuous shell edge, excluding walking legs, claws, abdomen, shadows, background, and residual areas discontinuous with the main carapace.

[0030] like Figure 3 As shown, in actual detection, the dorsal image of the crab to be detected is input into the trained crab shell segmentation model to obtain the main crab shell region, i.e., the main crab shell mask, denoted as M. To reduce the influence of legs, claws, background, edge burrs, and local noise on subsequent feature calculations, the main crab shell mask M is post-processed. This post-processing includes one or more of the following: maximum connected component preservation, hole filling, contour smoothing, shell shape prior, and branch culling, thereby obtaining a crab shell mask with complete boundaries, continuous main body, and suitable for subsequent feature calculations.

[0031] S22. Extraction of Two-Dimensional Features ① Shell length and shell width are used to represent the length and width dimensions of the crab shell body. Based on the outer contour of the crab shell body mask M, the principal axis and secondary axis of the crab shell body are calculated, thus obtaining the length of the principal axis of the carapace, denoted as Lc (shell length), and the length of the secondary axis of the carapace, denoted as Bc (shell width). In one embodiment, the shell length and shell width can be obtained from the minimum circumscribed rectangle of the crab shell body region, where the longer side of the rectangle is the shell length and the shorter side is the shell width. In another embodiment, the shell length and shell width can also be obtained from a regular circumscribed rectangle, the principal and secondary axes of principal component analysis, or the principal and secondary axes of a fitted ellipse.

[0032] ② Shell area is used to represent the actual area of ​​the crab shell or its projected size in a two-dimensional image, denoted as The shell area refers to the two-dimensional projected area of ​​the crab shell body on the image plane or the actual plane after scale calibration. It is not equivalent to the actual area of ​​the crab shell. After scale calibration, the mask area, measured in pixels, can be converted to the actual crab shell area. The shell area can be obtained from the crab shell body mask area or the contour envelope area. The mask area is closer to the actual coverage of the segmented region, while the contour envelope area is closer to the overall projected size of the crab shell. In one embodiment, the shell area is represented by the crab shell body mask area. The crab shell body mask is set as... The image width is Image height is When pixel coordinates When it belongs to the main area of ​​the crab shell ,otherwise The pixel area of ​​the crab shell main body mask can be calculated according to the following formula: in, This represents the area of ​​the crab shell mask in pixels. This is assuming consistent image acquisition distance, camera parameters, and sample placement. It can be used as a feature input to the model; after scale calibration is completed, it can be converted into actual area. in, and These represent the actual horizontal and vertical scales corresponding to a single pixel, respectively.

[0033] Another way is to define the shell area as the contour envelope area, denoted as... That is, based on the main body of the crab shell. Extract the outer contour and calculate the area enclosed by the outer contour.

[0034] In actual modeling, shell area Desirable , or One or more of them.

[0035] ③ The shell morphology parameter is used to represent the roundness of the crab shell outline, i.e., the roundness Rc, which is calculated according to the following parameters. The higher the roundness Rc, the closer the crab shell outline is to being round and smooth.

[0036] Where Rc represents roundness, Pc represents the circumference of the outer contour of the crab shell, and Pc represents the circumference of the outer contour of the crab shell, which can be determined by the crab shell mask. The outer contour is extracted. When using pixel area... Use pixel perimeter; when using actual area, The actual perimeter after scale conversion.

[0037] ④ Shell color is based on the inward-curving crab shell area Calculations were performed. To reduce the impact of highlights at the crab shell edges, background reflections, and leg residue on color calculations, a mask was applied to the crab shell body. The inward erosion process is performed according to a preset pixel distance or preset ratio to obtain the inward-shrinking crab shell region, denoted as... The preset pixel distance or preset ratio can be determined based on the image resolution, the size of the crab shell, and the model training effect. In one embodiment, the edge of the crab shell mask can be shrunk inward by a certain proportion, retaining only the relatively stable internal region of the crab shell for color and texture parameter calculation. Shell color parameters include Lab color parameters and HSV color parameters, where L* represents the crab shell interior metric, a* represents the red-green chromaticity, b* represents the yellow-blue chromaticity, and S... HSV These parameters represent color saturation. They are used to describe the color appearance of the crab shell surface.

[0038] ⑤ Shell texture parameters are based on the inward-curving crab shell region Calculations are performed. In the region of the inward-shrinking crab shell... In this process, color images are converted to grayscale images, and a gray-level co-occurrence matrix (GLCM) is constructed. Based on the GLCM, shell texture parameters are calculated, including texture contrast, texture homogeneity, texture correlation, and texture energy. These texture parameters represent the texture uniformity, roughness, and structural continuity of the crab shell surface.

[0039] The above parameters are concatenated together to obtain two-dimensional features. : In actual modeling, some or all of the two-dimensional features can be selected as input based on the model selection results. Table 1 shows the calculation results of the two-dimensional image features of the example samples. Figure 2The process of segmenting the crab shell and extracting two-dimensional features from a two-dimensional image is shown.

[0040] Table 1 Two-dimensional features of example river crabs S3. Processing of 3D Point Clouds and Extraction of 3D Features S31. Preprocessing and grid generation of 3D point clouds Preprocessing is performed on the back and ventral point clouds. First, the back and ventral point clouds corresponding to the same sample number are read, and then converted to a unified working coordinate system according to the calibration parameters saved during acquisition.

[0041] A supporting plane is fitted to the dorsal point cloud to identify the plane on which the acquisition platform or desktop is located. The Random Sample Consensus (RANSAC) algorithm or the least squares plane fitting algorithm is used to fit a plane with a large area and approximately flatness from the acquired point cloud as the supporting plane. Spatial clipping, outlier removal, and main body region preservation are then used to obtain the carapace main point cloud. In this working coordinate system, the supporting plane is used as the horizontal reference plane, defined as the XY plane, and the direction perpendicular to this plane is defined as the Z-axis direction. Outlier removal and density clustering are performed on the ventral point cloud, retaining the largest main point cluster as the ventral main point cloud. Since the crab's posture is opposite to that of the dorsal side during ventral acquisition, the ventral point cloud is spatially flipped so that it is located below the dorsal point cloud in the unified working coordinate system.

[0042] In this embodiment, the Random Sample Consensus (RANSAC) algorithm is used to perform planar fitting on the back point cloud. The planar distance threshold is set to 2.5 mm, the number of iterations is set to 8000, and the fitting is repeated 5 times. From this, a plane with a large area and approximately flatness is selected as the support plane. Subsequently, the main point cloud of the carapace is obtained through spatial clipping, outlier removal, and main body region retention. Points above the support plane by 0.75 mm are retained. Outliers are removed using the DBSCAN density clustering algorithm. The clustering neighborhood radius is set to 15 mm, the minimum number of points is set to 20, and multiple valid point clusters that meet the point number threshold are retained. Finally, a working coordinate system is established with the support plane as the horizontal reference plane, i.e., the XY plane, and the direction perpendicular to this plane is defined as the Z-axis direction.

[0043] Outlier removal and density clustering were performed on the ventral point cloud. The cluster neighborhood radius was set to 12 mm, the minimum number of points was set to 20, and the largest main point cluster was retained as the main ventral point cloud. Since the crab's posture was opposite to that of the back side during ventral acquisition, the ventral point cloud was spatially flipped by 180° so that the ventral point cloud was located below the back side point cloud in the unified working coordinate system.

[0044] like Figure 4As shown, the projection centers of the back and front point clouds on the horizontal reference plane are calculated separately. Based on the center deviations of the two in the X and Y directions, the front point cloud is horizontally translated to ensure that its projection center is essentially aligned with the back point cloud's projection center, while simultaneously ensuring that the main area of ​​the front falls within the projection range of the back point cloud. After this alignment, the vertical height difference between the back and front point clouds can be calculated at the same horizontal grid position.

[0045] The registered back and front point clouds are projected onto the same horizontal reference plane and arranged according to the same grid size. and Mesh the data so that the back face point cloud and the front face point cloud can each form their own effective grid set, i.e., the back face effective grid set. and ventral effective grid set Then, the effective grille assembly on the back side. The highest value within each grid is taken as the height of that grid, and the effective grid set on the ventral side is used as the basis. The lowest value within each grid is taken as the height of that grid.

[0046] Additionally, the back height integral is in Internal calculations, including the integral of the ventral height and the ventral projected area. Internal calculations; while the local thickness described below refers to the effective grid set shared by the back and front surfaces. Internal calculation.

[0047] S32, such as Figure 4 , 5 As shown, three-dimensional feature calculation ①Body height parameters The body height parameter represents the overall vertical height of the crab. It is obtained from the height difference between the highest area of ​​the dorsal point cloud and the reference plane, or from the height range of the dorsal and ventral point clouds under a unified coordinate reference. It can be calculated using the following formula. The larger the body height parameter, the greater the height of the crab in the dorsal and ventral direction, and the higher the overall bulge or thickness of the crab's body.

[0048] in, Indicates the effective set of grilles on the back. The maximum value of each grid height. This represents the height of the horizontal reference plane, which is set to 0 in this embodiment, meaning the horizontal reference plane is at 0 on the Z-axis.

[0049] ② The integral feature of the back point cloud represents the spatial volume of the back point cloud relative to the reference plane, reflecting the overall spatial volume and ridge degree of the carapace region in the vertical direction. The larger the value, the greater the cumulative height of the carapace relative to the reference plane, and the more pronounced the volume of the dorsal surface. It can be calculated using the following formula: in, Indicates the height within the i-th, j-th rear grille unit; This indicates the height of the horizontal reference plane, which is set to 0 in this embodiment. Indicates the effective set of grilles on the back. and These represent the side lengths of the grid element in the X and Y directions, respectively.

[0050] ③ The integral feature of the ventral point cloud is used to represent the spatial volume of the ventral point cloud relative to the reference plane, reflecting the spatial volume between the ventral armor and the reference plane. The larger the value, the greater the cumulative height of the ventral point cloud relative to the reference plane, reflecting the spatial morphology and void variations of the ventral region. It can be calculated using the following formula: in, This represents the height within the i-th,j-th ventral grid unit; This indicates the height of the horizontal reference plane, which is set to 0 in this embodiment. Indicates the set of effective grids on the ventral side. and These represent the side lengths of the grid element in the X and Y directions, respectively.

[0051] ④Aventral projection area This represents the coverage area of ​​the main ventral region on the horizontal reference plane, that is, the two-dimensional projected area of ​​the effective ventral point cloud projected onto the horizontal reference plane, reflecting the area of ​​the plastron on the XY plane. Specifically, the number of effective grid units corresponding to the ventral point cloud on the XY plane can be counted, and the number of effective grid units can be multiplied by the area of ​​a single grid unit to obtain the ventral projected area. The larger the value, the greater the coverage area of ​​the main ventral region on the horizontal reference plane. It can be calculated using the following formula: in, Indicates the number of effective grid units on the ventral side. and These represent the side lengths of the grid element in the X and Y directions, respectively.

[0052] ⑤ Abdominal undulation It indicates the degree of undulation of the entire ventral surface, reflecting the degree of undulation and unevenness of the ventral surface. The larger the value, the more pronounced the change in ventral height, indicating a fuller abdomen for the crab. This can be calculated from the standard deviation of the effective grid unit height values ​​on the ventral side. in, This represents the height within the i-th, j-th ventral grid unit.

[0053] ⑥ Abdominal radius of curvature It indicates the degree of curvature of the ventral region, reflecting the curvature of the ventral cross-section curve. The larger the value, the gentler the ventral cross-sectional curve and the lower the degree of abdominal curvature, indicating a smoother and less plump abdomen in the crab. Specifically, within the effective point cloud region of the ventral surface, one or more cross-sectional bands are selected along the head-to-tail direction of the crab, using the ventral projection center as a reference. These cross-sectional bands have a preset width to contain a sufficient number of ventral point clouds. When extracting the cross-section along the head-to-tail direction, the point cloud within the cross-sectional band can be projected onto a vertical plane formed by the head-to-tail direction and the Z-axis, thus obtaining a two-dimensional cross-sectional point set of the abdomen. This two-dimensional cross-sectional point set is then fitted to obtain the abdominal fitting curve. Finally, the abdominal curvature radius is calculated based on the curvature of the abdominal fitting curve or the equivalent radius of the circular arc. .

[0054] Calculate the abdominal radius of curvature based on the curvature of the fitted abdominal curve or the equivalent radius of the circular arc. A larger radius of curvature in the abdomen indicates a gentler ventral curve, while a smaller radius of curvature indicates a more pronounced curvature in the ventral curve.

[0055] ⑦ Abdominal curve fitting error This indicates the degree of deviation between the actual abdominal cross-section point and the fitted abdominal curve. The larger the value, the greater the deviation of the actual abdominal cross-section point from the fitted curve, and the lower the regularity of the abdominal shape. The abdominal fitted curve is the theoretical contour line constructed in section ⑥. Let the... The shortest distance from each abdominal cross-section point to the abdominal fitted curve is: The number of cross-sectional points involved in the fitting is The abdominal curve fitting error can be calculated using the following formula. The larger the abdomen, the less regular its shape. The smaller the size, the more regular the shape of the abdomen.

[0056] in, Indicates the first k The distance from each two-dimensional cross-sectional point on the abdomen to the fitted curve on the abdomen. n This indicates the number of points involved in the fitting process. This parameter is used to indicate the regularity of the abdominal contour.

[0057] ⑧ Local thickness This represents the distance between the upper and lower surfaces of the crab body at a certain horizontal grid position, and is determined by all effective grid units. The thickness distribution is used to reflect the thickness variations at different locations on the crab's body. It consists of identical grid sets on both the dorsal and ventral sides, effectively representing the height. Internal calculation of local thickness: in, Indicates the first The representative height of the dot cloud on the back of each grid unit. This represents the representative height of the corresponding ventral point cloud, and is a set of identical grids. Indicates the set of effective grids on the ventral side. Effective grille set on the back A set of repeating grids is a set of identical grids. When a grid lacks a valid height value, it can be supplemented by adjacent grids; if it cannot be supplemented, that grid is not included in the calculation.

[0058] ⑨ Average thickness of crab body Used to represent the overall thickness level within the effective area of ​​the crab's body. The larger the value, the thicker the crab's body. It can be calculated from the average local thickness of all effective grid units: in, Indicates the effective thickness grid set. Indicates the number of effective thickness grids.

[0059] ⑩ Crab body high-resolution thickness This parameter indicates the thickness level of thicker areas on the crab's body. A larger parameter indicates a more pronounced locally thicker area. It can be calculated from all effective local thicknesses. Calculation of the 90th percentile: in, This represents the 90th percentile.

[0060] Center-edge thickness difference Used to indicate the thickness advantage of the central region of the crab's body relative to the edge region. The larger the value, the fuller the central part of the crab's body, indicating a higher degree of trunk development. The thresholds for distinguishing the central and peripheral regions can be pre-set based on sample specifications, grid size, and model training results. The calculation formula is: in, Indicates the central area The average value, Indicates the edge area The average value.

[0061] Carapace center-edge height difference This parameter indicates the degree of elevation of the central region of the carapace relative to the peripheral region. A larger parameter indicates a more pronounced central elevation, meaning a more developed body structure in the crab. The division between the central and peripheral regions is consistent with the center-perimeter thickness difference calculation formula: in, This indicates the average representative height of the effective lattice in the central area of ​​the carapace. This indicates the average representative height of the effective grille in the edge region of the carapace.

[0062] By concatenating the above parameters, we obtain the three-dimensional features. : In actual modeling, some or all of the 3D features can be selected as input based on the model selection results. Figure 3 The diagram shows the extraction of 3D features after preprocessing the point cloud data of the example sample crab. The corresponding feature values ​​are shown in Table 2.

[0063] Table 2 Three-dimensional features of example river crabs S4, Point Cloud Volume and Density Calculation In this embodiment, the point cloud volume is obtained by a volume prediction model. Specifically, the three-dimensional features are input into the volume detection model: in, This represents the volume detection model, where V represents the point cloud volume.

[0064] In this embodiment, the three-dimensional morphological features As input, output point cloud volume That is, the volume of the crab to be tested. In other embodiments, the volume detection model may be one or more of Ridge, RF, ET, GBDT, or NN.

[0065] In this embodiment, the input to the volume prediction model does not include body weight data; therefore, the resulting point cloud volume V is a geometric volume that does not directly depend on body weight. After obtaining the point cloud volume V, the three-dimensional apparent density is calculated by combining it with the body weight W: Where ρ represents the three-dimensional apparent density. The three-dimensional apparent density represents the body weight per unit point cloud volume, and can be used as an indicator of the overall plumpness of the crab. Example sample crab calculated volume. Three-dimensional apparent density .

[0066] S5. Construct a quality inspection model and combine all input parameters. Weight, gender, two-dimensional features, three-dimensional features, point cloud volume, and three-dimensional apparent density are combined into the input vector of the quality detection model, and the representative features are shown in Table 3.

[0067] Where X represents the input vector of the quality inspection model, W represents weight, and S represents gender information. Representing two-dimensional features, V represents the three-dimensional feature, V represents the point cloud volume, and ρ represents the three-dimensional apparent density.

[0068] During the model training phase, the input vector X from multiple crab samples is used as the training input, and the gonadal quality, hepatopancreas quality, overall edible part quality, and their corresponding indices measured by dissection, along with their corresponding supervisory labels, are used to train the quality detection model. The number of samples should be sufficient for model training and validation to enable the model to establish a correspondence between non-destructive features and the development level of the internal edible parts.

[0069] In the actual testing phase, such as Figure 6 As shown, the input vector X of the crab to be tested is input into the trained quality detection model, which outputs one or more of the following: non-destructive detection values ​​for gonad quality, non-destructive detection values ​​for hepatopancreas quality, non-destructive detection values ​​for overall edible part quality, corresponding index detection values, high-value candidate probabilities, or grading results. These non-destructive detection values ​​are estimates based on the model's output of non-destructive features, and do not require dissection of the crab to be tested.

[0070] The quality inspection model can be represented as: in, This represents the quality inspection model. Indicates gonadal quality test value, This indicates the quality test value of the liver and pancreas. This indicates the overall quality test value of the edible portion. This indicates the result of an index or classification.

[0071] The overall edible portion quality can be directly output from the model, or it can be obtained by adding the predicted gonadal and hepatopancreatic qualities: Furthermore, the corresponding index is calculated based on the predicted mass and weight: in, This represents the gonadal quality predicted by the model. This represents the model's predicted liver and pancreas quality. The representative model predicts the quality of the liver, pancreas, and gonads. This indicates the gonadal index test value. This indicates the liver and pancreas index test value. This indicates the overall edible portion index test value.

[0072] The aforementioned quality inspection models include regression models, classification models, or a combination of both. The regression model outputs continuous indicators such as gonadal quality, hepatopancreas quality, overall edible portion quality, or corresponding indices. The classification model outputs high gonadal grade, high hepatopancreas grade, high overall edible portion grade, or high overall edible portion index grade.

[0073] Table 3 Characteristics used in quality inspection S6. Model training, validation, and high-value classification During the model training phase, the collected weight, sex, two-dimensional features, three-dimensional features, point cloud volume, and three-dimensional apparent density were used as inputs, and the gonadal mass, hepatopancreas mass, and total edible part mass measured by anatomical examination were used as the supervised targets for training the regression model.

[0074] In this embodiment, the regression model can first output the non-destructive test values ​​of gonadal quality, hepatopancreas quality, comprehensive edible part quality and their corresponding indices, and then classify them according to preset empirical thresholds, industry grading thresholds or batch quantile thresholds.

[0075] Simultaneously, labels for the classification model can be constructed based on whether gonadal quality, hepatopancreas quality, overall edible portion quality, or overall edible portion index reaches a preset high-value standard. For example, samples in the same batch with a certain indicator ranking in the top 30% can be defined as high-value samples, and the remaining samples as ordinary samples. The classification model outputs the high-value candidate probability or grading result.

[0076] In this embodiment, a total of 600 crab samples were collected. After 3D point cloud cleaning, 2D image feature extraction, and sample number matching, 586 multimodal valid samples were obtained. Model evaluation employed repeated cross-validation, with the regression task using 5-fold 10-times repeated cross-validation and the classification task using 5-fold 10-times repeated stratified cross-validation. All performance metrics were calculated based on out-of-fold prediction results. All operations were performed using Python (3.9).

[0077] Furthermore, considering the needs of actual production, the data acquisition method in this embodiment can be divided into three schemes: complete mode, rapid detection mode, and machine vision mode. Among them, the multi-source fusion mode, which collects and inputs basic information, two-dimensional features, and three-dimensional features, achieves the best results. The rapid detection mode addresses the difficulty of collecting abdominal features from crabs by inputting only weight, two-dimensional features of the back, and back point cloud features. This mode reduces the steps of flipping the sample and collecting abdominal features, and can quickly screen gonad quality, hepatopancreas quality, overall edible part quality, and their corresponding indices while maintaining high detection efficiency. The machine vision mode addresses the difficulty of collecting weight data by using a 3D camera to collect two-dimensional and three-dimensional features of the crab, achieving non-destructive machine vision detection that does not rely on measured weight. It can simultaneously collect data from multiple targets and output results using machine vision.

[0078] When using the above three modes to detect the quality indicators of crabs, the input features and model performance of each mode are shown in Table 4. The complete multi-source fusion mode comprehensively utilizes all information and has high overall detection performance; the rapid detection mode still has a certain detection capability without collecting abdominal features and is suitable for rapid initial screening on the production line; the machine vision detection mode can still achieve non-destructive auxiliary detection of the edible part development level by relying on gender visual recognition, two-dimensional shell phenotype, three-dimensional spatial structure and point cloud volume information without inputting the actual weight, and is suitable for batch automated grading.

[0079] Table 4 Performance of non-destructive testing models for crab quality under different application modes S7, Non-destructive testing and grading applications For the crab samples to be tested, body weight, two-dimensional images, and three-dimensional point cloud data were collected according to the methods described in S1 to S4, and two-dimensional features, three-dimensional features, point cloud volume, and three-dimensional apparent density were extracted. Subsequently, these were stitched together into multi-source fusion features and input into the trained quality detection model.

[0080] The quality inspection model can output one or more of the following: gonad quality inspection value, hepatopancreas quality inspection value, overall edible portion quality inspection value, gonad index inspection value, hepatopancreas index inspection value, overall edible portion index inspection value, high-value candidate probability, or high-value grade result. These outputs can be used for screening premium river crabs and grading commercial products.

[0081] Premium River Crab Selection: By setting thresholds or percentages, the system can automatically select plumper river crabs.

[0082] Product grading: The output of the detection model uses the quality of the liver, pancreas, and gonads as indicators, and can automatically set thresholds for grading. When the crab shell mask is incomplete, the point cloud coverage is insufficient, the number of effective grids is too small, the center offset of the dorsal and ventral point clouds is too large, the surface missing ratio is too high, or the model output probability is in the uncertain range, the system marks the sample as a verification sample and prompts for re-collection or manual confirmation.

[0083] It should be noted that this method is used for non-destructive auxiliary judgment and grading screening of the development level of the edible part inside the crab, and is not limited to completely replacing dissection and measurement. Other machine learning models, other grid sizes, other combinations of equivalent phenotypic parameters, or other grading thresholds may also be used without changing the basic technical concept of this invention.

Claims

1. A non-destructive testing method for crab quality, characterized in that... Includes the following steps: Step 1: Collect basic information, two-dimensional image data, and three-dimensional point cloud data of the crab species to be tested; Step 2: Perform image processing on the two-dimensional image to extract two-dimensional features, and analyze the three-dimensional point cloud to extract three-dimensional features. Then, based on the three-dimensional features and basic information, calculate the three-dimensional apparent density of crabs. Step 3: Combine the basic information, two-dimensional features, three-dimensional features, crab volume features, and three-dimensional apparent density to form a multi-source fusion feature. Use this as the input to the quality detection model and output the quality indicators of the crab to be tested to achieve non-destructive quality detection.

2. The non-destructive testing method for crab quality according to claim 1, characterized in that: In step two, the collected 3D point cloud data of the back side and the 3D point cloud data of the front side are first preprocessed. Then, the preprocessed 3D point cloud data of the back side and the 3D point cloud data of the front side are projected onto a horizontal reference plane, and the projected area is divided into uniform grids, which are denoted as the effective grid sets of the front side. Effective grille set on the back , Then calculate the effective set of rear grilles. The highest value of each grid region within the interior, and the effective grid set on the ventral side. The minimum value of each grid region is used, and then the height parameter of the three-dimensional feature volume is calculated using these maximum and minimum values ​​respectively. ventral undulation Integral features of backside point clouds Integral features of ventral point cloud Local thickness Average thickness High quantile thickness Center-edge thickness difference Carapace center-edge height difference ; Simultaneously, the projected area of ​​the 3D point cloud of the abdomen onto the horizontal reference plane is calculated, and this area is used as the 3D feature ventral surface projection area. Furthermore, by using slice projection, an abdominal fitting curve is constructed, and the radius of curvature of the three-dimensional abdominal feature is calculated. Abdominal curve fitting error , Finally, the above parameters are concatenated to obtain the three-dimensional features. : 。 3. The non-destructive testing method for crab quality according to claim 2, characterized in that: 3D features Input the volume prediction model to obtain the crab volume feature V, and then use the following formula to calculate the three-dimensional apparent density ρ. Where W represents the weight information in the basic information.

4. The non-destructive testing method for crab quality according to claim 3, characterized in that: The lowest value corresponding to each grid cell within the ventral 3D point cloud region is taken as the height of that grid cell. The integral feature of the ventral point cloud is calculated using the following formula. , in, This represents the height within the i-th, j-th ventral grid unit; Indicates the reference height on the ventral side; Indicates the set of effective grids on the ventral side. and These represent the side lengths of the grid element in the X and Y directions, respectively; Using the highest value corresponding to each grid in the back 3D point cloud region as the height of that grid, the integral feature of the back point cloud is calculated using the following formula. , in, Indicates the height within the i-th and j-th rear grille units; Indicates the reference height on the back side; Indicates the effective set of grilles on the back. and These represent the side lengths of the grid element in the X and Y directions, respectively.

5. The non-destructive testing method for crab quality according to claim 2, characterized in that: Find the effective grid set on the ventral side Effective grille set on the back Repeating grids are denoted as identical grids. The difference between the highest and lowest values ​​corresponding to each identical grid is the local thickness. All local thicknesses The average value is the average thickness. These local thicknesses The 90th percentile is the high quantile thickness. , Calculate the center-edge thickness difference using the following formula. , in, This indicates the local thickness within the central area of ​​the main body of the crab shell. The average value, The thickness of the local area within the edge region of the main body of the crab shell. The average value; Calculate the height difference between the center and edge of the carapace using the following formula. , in, This represents the average height of each grid within the central area of ​​the crab shell's main body. This represents the average height of each grid within the edge region of the main body of the crab shell.

6. The non-destructive testing method for crab quality according to claim 2, characterized in that: In calculating the abdominal curvature radius Using the projection center of the ventral 3D point cloud onto the horizontal reference plane as a reference, a cross-sectional strip is selected along the head-to-tail direction of the crab. This cross-sectional strip has a preset width. The cross-sectional strip is then projected onto a vertical plane formed by the head-to-tail direction and the Z-axis, thereby obtaining a 2D cross-sectional point set of the abdomen. These 2D cross-sectional points are then fitted to obtain a fitted curve of the abdomen, the curvature of which, or the equivalent radius of the arc, is the radius of curvature of the abdomen. ; The abdominal curve fitting error is calculated using the following formula. , in, Indicates the first k The distance from each two-dimensional cross-sectional point on the abdomen to the fitted curve on the abdomen. n This indicates the number of points involved in the fitting process.

7. The non-destructive testing method for crab quality according to claim 1, characterized in that: In step two, the main body region M of the crab shell is first segmented from the acquired two-dimensional image of the back. The principal axis length, secondary axis length, shell area, shell color parameters, shell shape parameters, and shell surface texture parameters of the main body of the crab shell are calculated. Then, these parameters are stitched together to obtain the two-dimensional features. .

8. The non-destructive testing method for crab quality according to claim 7, characterized in that: When calculating the shell color parameters and shell texture parameters, the main body region M of the crab shell is eroded inward according to a preset pixel distance or preset ratio to obtain the inward-shrinking crab shell region. From the inward-shrinking crab shell area Extract shell color parameters, including Lab and HSV color parameters, where L* represents the shell interior metric, a* represents the red-green chromaticity, b* represents the yellow-blue chromaticity, and S... HSV Indicates color saturation; Retract the crab shell area The corresponding color image is converted into a grayscale image, and a grayscale co-occurrence matrix is ​​constructed. Then, based on the grayscale co-occurrence matrix, shell texture parameters are calculated, including texture contrast, texture homogeneity, texture correlation, and texture energy. The roundness Rc is calculated using the following formula to characterize the shell morphology parameters. in, Represents the shell area. This represents the perimeter of the outer contour of the main body area of ​​the crab shell; The shell area is obtained by calculating the actual area enclosed by the outer contour of the main body region M of the crab shell, or the projected area corresponding to the main body region M of the crab shell. .

9. The non-destructive testing method for crab quality according to claim 1, characterized in that: First, construct the multi-source fusion feature X using the following formula. Where W represents weight information from the basic information, and S represents gender information from the basic information. Representing two-dimensional features, V represents the three-dimensional feature, V represents the volume feature of the crab shell, and ρ represents the three-dimensional apparent density. Then, using the multi-source fusion feature X as input, and the gonad quality, hepatopancreas quality, overall edible portion quality, and their corresponding gonad index, hepatopancreas index, and overall edible portion index as output, the quality detection model is trained. Then, the trained quality inspection model was used to test the gonad quality, hepatopancreas quality, overall edible portion quality, and their corresponding gonad index, hepatopancreas index, and overall edible portion index of the crabs to be tested. Based on the empirical values, the crabs to be tested were graded.

Images