Quality control method for premade perch dish based on machine vision

Abstract

The invention discloses a method for controlling the quality of a perch premade dish based on machine vision. The method comprises the following steps: obtaining a preprocessed perch premade dish image; inputting the pre-processed perch pre-prepared dish image into a residual dense block context feature extraction network, and outputting a multi-scale feature tensor; generating a boundary binary image, an Euler number heat image and a boundary attention weight tensor by a boundary topology branch; outputting a preliminary segmentation mask through a channel-space rearrangement mechanism; an optimized sub-pixel level segmentation network is obtained; in the inference stage, boundary refinement and topological consistency correction are carried out on the preliminarily segmented mask based on the boundary binary image and the Euler number heat image, and a target mask image is obtained; calculating a morphological integrity index set based on the target mask graph; and the integrity scoring module outputs an integrity judgment result according to a preset scoring threshold. According to the method, sub-pixel-level reconstruction and high-precision positioning of prefabricating vegetable body boundaries, tiny gaps and fractures of the weever are realized.

Description

Technical Field

[0001] This invention relates to the field of pre-prepared sea bass dishes, and in particular to a machine vision-based method for quality control of pre-prepared sea bass dishes. Background Technology

[0002] With the continuous development of automation and intelligent manufacturing in the food industry, machine vision-based food quality inspection technology is gradually being applied to various pre-prepared food production lines. In the production process of animal products such as pre-prepared sea bass, the integrity of the product's appearance directly affects its grading, sorting, and market circulation.

[0003] Traditional pixel-level segmentation methods typically employ standard convolutional neural networks or simple upsampling structures. These methods have limited accuracy in segmenting subtle boundaries, local gaps, and fractures in fish bodies against complex backgrounds. Limited by the spatial resolution capabilities of the segmentation network, they often suffer from blurred boundaries, misconnected fractures, and oversegmentation. In cases where sea bass skin is highly reflective, covered by sauce, or has complex textures, boundaries may be smoothed out or micro-cracks may be missed, leading to misjudgments of product integrity.

[0004] Deep network structures such as residual and dense connection have been applied in general image segmentation. However, conventional residual dense block networks in existing technologies have not been structurally improved for scenarios with large batch differences in pre-prepared dishes, scarce defect samples, complex morphology, and high real-time requirements. Mainstream segmentation algorithms often rely on large-scale balanced labeled data. However, in actual food production lines, the appearance defects of sea bass are distributed in a long tail, have many variations, and it is difficult to collect comprehensive training samples. Existing networks are prone to problems such as insufficient out-of-sample generalization ability and poor robustness to small targets and subtle defects. Summary of the Invention

[0005] One objective of this invention is to propose a machine vision-based method for quality control of pre-prepared sea bass dishes. This invention achieves sub-pixel-level reconstruction and high-precision positioning of the sea bass body boundary, minute gaps, and fractures in pre-prepared sea bass dishes.

[0006] A method for quality control of pre-prepared sea bass dishes based on machine vision according to an embodiment of the present invention includes:

[0007] Image data of pre-cooked sea bass during the conveying process of the pre-cooked sea bass on the production line are collected and image preprocessing is performed to obtain pre-processed images of pre-cooked sea bass.

[0008] The pre-processed image of the prepared sea bass dish is input into the residual dense block context feature extraction network, which outputs a multi-scale feature tensor.

[0009] Input the multi-scale feature tensor into the boundary topology branch, and the boundary topology branch generates a boundary binary map, an Euler number heatmap, and a boundary attention weight tensor.

[0010] The boundary attention weight tensor and the multi-scale feature tensor are fused in the pixel rearrangement feature pyramid decoder, and the initial segmentation mask is output through the channel-space rearrangement mechanism.

[0011] A composite loss function is used to jointly optimize the sub-pixel-level segmentation network composed of the residual dense block context feature extraction network, the boundary topology branch and the pixel rearrangement feature pyramid decoder. The edge preservation loss is supervised by the boundary binary map and the topology regularization term is supervised by the Euler number heatmap to obtain the optimized sub-pixel-level segmentation network.

[0012] During the inference phase, the initial segmentation mask is refined and topological consistency is corrected based on the boundary binary map and Euler number heat map to obtain the target mask map.

[0013] Calculate a set of morphological integrity indices based on the target mask image;

[0014] Input the set of morphological integrity indicators into the integrity scoring module, and the integrity scoring module outputs the integrity judgment result according to the preset scoring threshold.

[0015] Optionally, the image preprocessing includes illumination normalization, color normalization, and distortion correction.

[0016] Optionally, the step of inputting the pre-processed sea bass dish image into the residual dense block context feature extraction network includes:

[0017] Represent the pre-processed sea bass dish image in three dimensions;

[0018] By performing two-dimensional discrete convolution operations on the three dimensions using the initial convolution kernel and the initial bias, the input features of scale one are obtained.

[0019] A preset number of residual dense blocks are stacked at each scale. After all the input features of the previous layer are concatenated according to the channel dimension through the built-in dense connection layer, the dense connection output features are output.

[0020] Based on the pre-processed image of sea bass, the channels of each pixel are linearly combined using a color transformation matrix to obtain the luminance component and chrominance component. The luminance component and chrominance component are weighted and summed with a set of scalar weight coefficients and biases, and then mapped through a nonlinear activation function to obtain a reflection suppression weight map.

[0021] Based on the input features at the current scale, the Sobel operator is used to calculate the horizontal and vertical gradients respectively, and average pooling is performed to obtain structural elements. The principal axis direction angle is then calculated based on the structural elements.

[0022] The set of pre-defined directional base convolutional kernels is weighted and interpolated based on the principal axis direction angle, and the interpolated directional adaptive convolutional kernels replace the convolutional kernels of the residual dense blocks.

[0023] At each scale, a fixed Laplacian kernel is used to perform two-dimensional convolution on the input features to obtain high-frequency response features. The densely connected output features and high-frequency response features are weighted proportionally and concatenated. After channel alignment by a one-dimensional convolution kernel, they are residually superimposed with the input features to form new input features.

[0024] After stacking all residual dense blocks at each scale, a one-dimensional convolution kernel and bias are used to perform channel transformation on the new input features to output the scale features of the current scale. At the same time, a downsampling convolution kernel with a stride of two is used to perform a one-dimensional convolution operation on the scale features of the current scale to generate the input features of the next scale.

[0025] Multiple sets of dilated convolutions are applied to the scale features at each scale. The output features of each set of dilated convolutions are summed according to the channel dimension to obtain the scale context features at each scale. The scale context features of all scales are then combined into a multi-scale feature set.

[0026] Optionally, the step of inputting the multi-scale feature tensor into the boundary topology branch includes:

[0027] Select principal scale features from the multi-scale feature set, apply a two-dimensional convolution operation to the principal scale features with a bias, and then map them using an element-wise sigmoid activation function to obtain the boundary response map;

[0028] When the value of the boundary response map at the corresponding pixel position is greater than or equal to the preset threshold, the value of the boundary binary map at the corresponding pixel position is set to one; otherwise, it is set to zero, thus obtaining a binary boundary mask map.

[0029] For each channel of the main scale feature, a 3x3 two-dimensional convolution operation and element-wise ReLU activation function mapping are performed and fused into a single-channel feature map. Based on the single-channel feature map, a sliding window neighborhood is constructed for each pixel. Euler feature values ​​are calculated in each sliding window according to the local connected graph structure, and an Euler number heatmap is obtained.

[0030] Combining principal scale features with boundary response Figure 1 Using the same input boundary attention module, we obtain the boundary attention weight tensor.

[0031] Optionally, the fusion of the boundary attention weight tensor and the multi-scale feature tensor in the pixel rearrangement feature pyramid decoder includes:

[0032] Within each scale, the feature tensor of the current scale is multiplied element-wise by channel at each spatial location with the boundary attention weight tensor of the scale matching to obtain the modulation feature tensor.

[0033] Apply a one-dimensional convolution kernel to each modulation feature tensor along the channel dimension to obtain a channel compression tensor;

[0034] The channel compression tensor at higher scales is sampled from top to bottom, with the nearest neighbor upsampling operation performed sequentially to make its spatial size consistent with that of the channel compression tensor at the current scale. The fused tensor is then added to the channel compression tensor at the current scale at each spatial location and channel to obtain the fused tensor.

[0035] Apply a one-dimensional convolution kernel to each fusion tensor to obtain the channel expansion tensor;

[0036] Apply a channel-space rearrangement operation to each channel expansion tensor to decompose the channel dimension information of the channel expansion tensor by the square of the stride and rearrange it to the spatial dimension to obtain a sub-pixel-level upsampling tensor;

[0037] When the scale is at the shallowest level, a one-dimensional convolution kernel is applied to the sub-pixel-level upsampled tensor to obtain the category response tensor;

[0038] An element-wise softmax operation is applied to the category response tensor along the channel dimension to obtain a preliminary segmentation mask.

[0039] Optionally, the sub-pixel-level segmentation network undergoes joint optimization, including:

[0040] The pixel-level category mask labeled by training is used as the pixel-level category supervision signal of the segmentation network. The non-background channel in the pixel-level category mask labeled by training is downsampled using the nearest neighbor downsampling method to obtain a binary foreground annotation map.

[0041] Based on the initial segmentation mask and the pixel-level category mask of the training annotation, the category segmentation loss is calculated;

[0042] For each pixel in the binary foreground annotation map obtained by downsampling, if the value of the pixel is different from that of any eight neighboring pixels, it is assigned a value of one in the supervised boundary binary map at the corresponding pixel position; if the value of the pixel is the same as that of all eight neighboring pixels, it is assigned a value of zero in the supervised boundary binary map at the corresponding pixel position. Based on the boundary response map and the supervised boundary binary map, the edge preservation loss is calculated.

[0043] Based on the binary foreground annotation map, a fixed window sliding neighborhood is used for each pixel to calculate the supervised Euler feature value. The supervised Euler feature value of each pixel is then subtracted from the mean and divided by the standard deviation to obtain the standardized supervised Euler feature value of each pixel. The absolute difference between the Euler number heatmap and the standardized supervised Euler feature value is averaged over all pixel locations to obtain the topology regularization term.

[0044] By weighting and summing the category segmentation loss, edge preservation loss, and topology regularization term, a composite loss function is obtained, and an optimized subpixel-level segmentation network is trained.

[0045] Optionally, the step of refining the boundaries and correcting the topology consistency of the preliminary segmentation mask based on the boundary binary map and the Euler number heatmap includes:

[0046] For each pixel in the boundary response map, if the predicted probability of the pixel is greater than or equal to the set edge threshold, the corresponding pixel position in the boundary mask map is assigned a value of one; otherwise, it is assigned a value of zero.

[0047] For all non-background category channels of the initial segmentation mask, take the maximum category response at each pixel location to obtain the initial foreground mask;

[0048] The initial foreground mask is subjected to soft thresholding binarization at each pixel position to generate a foreground region mask. The skeleton extraction operation, morphological thinning operation, morphological closing operation and region connection repair operation are sequentially performed on the overlapping area of ​​the foreground region mask and the boundary mask to obtain the thinned mask.

[0049] The Euler number heatmap is normalized by performing full-image mean normalization on all pixel response values.

[0050] The normalized Euler number heatmap is adjusted by interpolation to match the spatial size of the initial segmentation mask, and used as the Euler constraint map. It is then combined with the refined mask to perform topology consistency correction and output the topology correction mask.

[0051] The topology correction mask is mapped back to the category space. At each pixel location, if the corresponding pixel of the topology correction mask is one and the category corresponding to the largest category response of the corresponding pixel in the initial segmentation mask is a certain category, the category channel of the corresponding pixel of the final target mask is assigned a value of one; otherwise, it is assigned a value of zero.

[0052] Optionally, the set of morphological integrity indicators calculated based on the target mask image includes:

[0053] Label the main body class of the bass in the category channel, take the main body class channel of the bass as the main body region of the bass, and divide the total length of the pixel segments with concave gaps in the boundary by the perimeter of the boundary of the main body region of the bass to obtain the edge gap rate;

[0054] Skeleton extraction is performed on a two-dimensional mask corresponding to the main body area of ​​the sea bass to obtain a skeleton map. All non-closed endpoints are detected in the skeleton map, and the number of non-closed endpoints is counted as the endpoint count. When the endpoint count is greater than or equal to two, the endpoint count is divided by two and rounded down to obtain the number of broken segments.

[0055] In the final target mask, the area of ​​the bone spur region is divided by the area of ​​the main body of the bass to obtain the proportion of the exposed bone spur region.

[0056] In the mask of the main body area of ​​the sea bass, the Euler feature values ​​corresponding to all the center pixels are combined to form the Euler feature value distribution. The absolute difference of the Euler feature value distribution and the normalized Euler number heatmap is calculated pixel by pixel. The absolute difference at all pixel positions is accumulated and then divided by the total number of pixels to obtain the Euler number difference.

[0057] The four indicators—edge notch rate, number of fracture segments, proportion of exposed bone spur area, and difference between Euler number—are used as a set of morphological integrity indicators.

[0058] Optionally, inputting the set of morphological integrity indicators into the integrity scoring module includes:

[0059] Input the set of morphological integrity indicators into the integrity scoring module, and output the integrity category of the prepared sea bass dish:

[0060] When the edge notch rate is less than or equal to the first notch rate threshold, the number of fracture segments is zero, the proportion of exposed bone spur area is less than or equal to the first exposed area threshold, and the difference in Euler number is less than or equal to the first set threshold, the output integrity category is complete.

[0061] When the edge notch rate is greater than the first notch rate threshold and less than or equal to the second notch rate threshold, or the number of broken segments is equal to one, or the proportion of the exposed area of ​​the bone spur is greater than the first exposed area threshold and less than or equal to the second exposed area threshold, or the difference in the Euler number is greater than the first set threshold and less than or equal to the second set threshold, the output integrity category is mild defect.

[0062] When the edge notch rate is greater than the second notch rate threshold, or the number of broken segments is greater than or equal to two, or the proportion of the exposed area of ​​bone spurs is greater than the second exposed area threshold, or the difference in the number of Eulers is greater than the second set threshold, the output integrity category is severe defect.

[0063] Optionally, the integrity scoring module outputs the defect type based on the morphological integrity index triggered by the integrity category determination process and its corresponding threshold conditions:

[0064] When the edge notch ratio is greater than the first notch ratio threshold, the defect type is edge notch;

[0065] When the number of fracture segments is greater than or equal to one, the defect type is structural fracture;

[0066] When the proportion of the exposed area of ​​bone spurs is greater than the threshold of the first exposed area, the defect type is exposed bone spurs;

[0067] When the difference in the Euler count is greater than the first set threshold, the defect type is topology anomaly.

[0068] The beneficial effects of this invention are:

[0069] (1) This invention introduces a multi-scale contextual feature extraction network based on residual dense blocks and directional adaptive convolution, combined with a sub-pixel-level feature fusion mechanism with boundary attention weights, to achieve sub-pixel-level reconstruction and high-precision localization of the boundary, micro-gap and fracture of the sea bass pre-cooked dish. It can still stably separate the main body of the fish from the edge under the background of high reflectivity, sauce occlusion and complex skin texture, and significantly improve the segmentation accuracy and connectivity maintenance ability of the integrity-related structures of micro-gap, crack and exposed bone spur.

[0070] (2) This invention proposes a multi-branch boundary topology-aware segmentation mechanism. During the training stage of the sub-pixel-level segmentation network, edge preservation loss and topology regularization term of Euler number heatmap are jointly introduced. Combined with the composite loss function of segmentation master mask, boundary response and Euler topology signal, the sub-pixel-level segmentation network can take into account the accuracy of region segmentation, the continuity of boundary geometry and the consistency of global topology during the optimization process. In the inference stage, through morphological refinement and topology consistency correction based on Euler constraints, the missegmentation problem of existing methods in the scenarios of fish body breakage, abnormal connectivity and missed detection of small breakpoints is effectively solved, and the stable quantification of multiple indicators of morphological integrity is realized in the actual industrial production line. Attached Figure Description

[0071] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0072] Figure 1 This is a flowchart of a machine vision-based method for quality control of pre-prepared sea bass dishes proposed in this invention.

[0073] Figure 2 This is a block diagram of the sub-pixel-level segmentation network in a machine vision-based pre-prepared sea bass dish quality control method proposed in this invention. Detailed Implementation

[0074] Example 1: Reference Figures 1-2 A machine vision-based method for quality control of pre-prepared sea bass dishes, comprising:

[0075] Image data of pre-cooked sea bass during the conveying process of the production line are collected and image preprocessing is performed. Image preprocessing includes illumination normalization, color normalization and distortion correction to obtain pre-processed images of pre-cooked sea bass.

[0076] The pre-processed image of the prepared sea bass dish is input into the residual dense block context feature extraction network, which outputs a multi-scale feature tensor.

[0077] In this embodiment, the pre-processed sea bass dish image is input into the residual dense block context feature extraction network, including:

[0078] Represent the pre-processed sea bass dish image in three dimensions;

[0079] The first dimension of the three-dimensional image is the height in pixels, the second dimension is the width in pixels, and the third dimension is the number of channels.

[0080] By performing two-dimensional discrete convolution operations on the three dimensions using the initial convolution kernel and the initial bias, the input features of scale one are obtained.

[0081] The number of input channels of the initial convolutional kernel is equal to the number of channels of the pre-processed sea bass dish image, and the number of output channels is the number of feature channels at scale one. The spatial size of the initial convolutional kernel is determined by the set side length. By sliding the initial convolutional kernel in three dimensions and performing element-wise multiplication and addition, while superimposing the initial bias, the input features at scale one are obtained.

[0082] A preset number of residual dense blocks are stacked at each scale. After all the input features of the previous layer are concatenated according to the channel dimension through the built-in dense connection layer, the dense connection output features are output.

[0083] The l-th block of the residual dense block receives the input features of the previous step's output, and each residual dense block contains a preset number of dense connection layers.

[0084]

[0085] in, The output features of the first layer dense connection within the l-th residual dense block at scale s are used to extract the low-level structure and texture features of the pre-processed image of sea bass at this scale. Let σ(·) represent the output feature of the k-th dense connection within the l-th residual dense block at scale s. This feature is used to integrate the multi-level features from the first k-1 layers and the input, achieving a dense representation of multi-layer information on the details, gaps, and fractures of the sea bass body surface. σ(·) represents the nonlinear activation function. Let represent the parameters of the first-layer convolutional kernel of the l-th residual dense block at scale s, which are applied to the input features to extract local spatial information. The parameters of the k-th layer convolution kernel for the l-th residual dense block at scale s are used to enhance the ability to identify sea bass edges, bones, and minor defects. Let represent the input features of the (l-1)th residual dense block at scale s. Let represent the first-level bias parameter of the l-th residual compact block at scale s. Let K represent the bias parameter of the k-th layer of the l-th residual dense block at scale s. Concat(·) represents the channel dimension splicing operation, achieving a collaborative representation of the microstructure and large-scale contour of the sea bass body surface. s This represents the number of layers in each residual dense block at scale s.

[0086] Based on the pre-processed image of sea bass, the channels of each pixel are linearly combined using a color transformation matrix to obtain the luminance component and chrominance component. The luminance component and chrominance component are weighted and summed with a set of scalar weight coefficients and biases, and then mapped through a nonlinear activation function to obtain a reflection suppression weight map.

[0087] The reflection suppression weight map is transformed into a single-channel weight consistent with the current scale through interpolation, which is used to perform pixel-by-pixel gating on features at the corresponding scale. The reflection suppression weight map suppresses the interference of surface reflection and sauce highlights on feature extraction of the pre-cooked sea bass dish.

[0088] Based on the input features at the current scale, the Sobel operator is used to calculate the horizontal and vertical gradients respectively, and average pooling is performed to obtain structural elements. The principal axis direction angle is then calculated based on the structural elements.

[0089] To align the long axis structure of the pre-cooked sea bass dish, based on the input features at the current scale, the Sobel operator is used to perform two-dimensional convolution on each channel to obtain the gradient maps of each pixel in the horizontal and vertical directions. After performing element-wise absolute value operations on the horizontal and vertical gradient maps, average pooling is performed on each channel to obtain the average structural component in the horizontal direction, the average structural component in the vertical direction, and the covariance component of the horizontal and vertical gradients at the current scale. The covariance component is obtained by performing average pooling on the element-wise product of the horizontal and vertical gradient maps. The principal axis direction angle is obtained using the average structural component and the covariance component.

[0090] The principal axis direction angle is equal to half the result of the arctangent function. The numerator of the arctangent function is twice the covariance component, and the denominator is the average structural component in the horizontal direction minus the average structural component in the vertical direction. The principal axis direction angle reflects the orientation of the main morphological structure of the pre-cooked sea bass in the current scale characteristics.

[0091] The set of pre-defined directional base convolutional kernels is weighted and interpolated based on the principal axis direction angle, and the interpolated directional adaptive convolutional kernels replace the convolutional kernels of the residual dense blocks.

[0092] In Example 1, the interpolation weight of each directional basis convolution kernel is calculated by combining the cosine similarity between the principal axis direction angle and the central angle of each directional basis with the concentration parameter. The interpolation weight is equal to the normalized exponential function of the cosine of the principal axis direction angle and the central angle of the directional basis. The sum of all interpolation weights is one. All directional basis convolution kernels are weighted and summed according to their respective interpolation weights to obtain the directional adaptive convolution kernel. The directional adaptive convolution kernel is used to replace the regular convolution kernel at the corresponding position in the residual dense block to achieve adaptive alignment between the feature extraction direction and the principal axis direction of the main body of the pre-cooked sea bass dish.

[0093] At each scale, a fixed Laplacian kernel is used to perform two-dimensional convolution on the input features to obtain high-frequency response features. The densely connected output features and high-frequency response features are weighted proportionally and concatenated. After channel alignment by a one-dimensional convolution kernel, they are residually superimposed with the input features to form new input features.

[0094] In Example 1, to enhance the high-frequency features of small cracks and exposed bones in the pre-cooked sea bass dish, a fixed Laplacian convolution kernel is used to perform two-dimensional convolution operations on the input features of each scale to obtain the high-frequency response features of the scale. The densely connected output features and the high-frequency response features are then multiplied element-wise with the texture gating coefficient and the structure gating coefficient, respectively, to obtain the texture path features and the structure path features. The pooling result of the reflection suppression weight map and its complementary value are used as the texture gating coefficient and the structure gating coefficient, respectively. The texture path features and the structure path features are concatenated along the channel dimension to form a fused feature. A one-dimensional convolution kernel is applied to the fused feature to transform the number of channels so that it is consistent with the input features of the scale. The fused feature after channel transformation is then added element-wise with the input features of the corresponding scale at the corresponding element positions to obtain new input features.

[0095]

[0096] in, This represents the new input feature of the l-th residual dense block at scale s. This represents the 1×1 channel-aligned convolution kernel corresponding to the l-th residual dense block at scale s, where * denotes a one-dimensional convolution operation. represents the texture gating coefficient at scale s, reflecting the role of fine-grained skin and flesh texture in the recognition of prepared sea bass dishes; ⊙ represents the element-wise multiplication operator. K represents the interior of the l-th residual compact block at scale s. s Intermediate features generated by dense layer connections This represents the structural gating coefficient at scale s, emphasizing the main structure and boundaries of the fish's body morphology. This represents the high-frequency features at scale s, used to highlight the minute high-frequency defects such as fractures, cracks, and bone spurs in prepared sea bass dishes.

[0097] After stacking all residual dense blocks at each scale, a one-dimensional convolution kernel and bias are used to perform channel transformation on the new input features to output the scale features of the current scale. At the same time, a downsampling convolution kernel with a stride of two is used to perform a one-dimensional convolution operation on the scale features of the current scale to generate the input features of the next scale.

[0098] Multiple sets of dilated convolutions are applied to the scale features at each scale. The output features of each set of dilated convolutions are summed according to the channel dimension to obtain the scale context features at each scale. The scale context features of all scales are then combined into a multi-scale feature set.

[0099] The dilation rate of dilated convolution is set to different positive integers. The spatial dimension of the context features at each scale in the multi-scale feature set is consistent with that scale, and the number of channels is consistent with the number of channels after transformation.

[0100] Input the multi-scale feature tensor into the boundary topology branch, and the boundary topology branch generates a boundary binary map, an Euler number heatmap, and a boundary attention weight tensor.

[0101] In this embodiment, the multi-scale feature tensor is input into the boundary topology branch, including:

[0102] Select principal scale features from the multi-scale feature set, apply a two-dimensional convolution operation to the principal scale features with a bias, and then map them using an element-wise sigmoid activation function to obtain the boundary response map;

[0103] The principal scale feature is the feature tensor corresponding to the principal scale selected from all scales in the scale context feature for boundary topology branching. It can be regarded as one of the multi-scale feature sets, with preset height, width and number of channels, and is the scale that best represents the edge contour and fracture details.

[0104] The spatial size of the convolution kernel in the two-dimensional convolution operation is three by three. The number of input channels is equal to the number of channels of the main scale feature, and the number of output channels is one. The value of each pixel in the boundary response map ranges from zero to one, representing the probability that each pixel of the pre-cooked sea bass dish belongs to the edge region.

[0105] When the value of the boundary response map at the corresponding pixel position is greater than or equal to the preset threshold, the value of the boundary binary map at the corresponding pixel position is set to one; otherwise, it is set to zero, thus obtaining a binary boundary mask map.

[0106] Each pixel in the boundary binary map is either zero or one. The boundary binary map is used as a supervision target in the training phase and as an edge guidance basis in the inference phase, and is used to highlight the outline, cross-section and gap of the sea bass pre-cooked dish.

[0107] For each channel of the main scale feature, a 3x3 two-dimensional convolution operation and element-wise ReLU activation function mapping are performed and fused into a single-channel feature map. Based on the single-channel feature map, a sliding window neighborhood is constructed for each pixel. Euler feature values ​​are calculated in each sliding window according to the local connected graph structure, and an Euler number heatmap is obtained.

[0108] In Example 1, the convolution and activation outputs of all channels are fused into a single-channel feature map by pixel-wise averaging in the channel direction. For each pixel in the single-channel feature map, a sliding neighborhood of a fixed window size is taken with the pixel as the center. The pixel values ​​in the sliding neighborhood are binarized according to a preset threshold (above the threshold is recorded as 1, otherwise it is recorded as 0). The number of connected components and the number of holes are counted in the binary neighborhood using the eight-neighbor connectivity standard. The Euler feature value is equal to the number of connected components in the neighborhood minus the number of holes. The Euler feature value is written into the corresponding pixel position of the Euler number heatmap to obtain the Euler number heatmap.

[0109] Combining principal scale features with boundary response Figure 1 Using the same input boundary attention module, we obtain the boundary attention weight tensor.

[0110] The boundary attention module linearly stretches the boundary response map into an attention weight map, and then expands it to the same number of channels as the main scale feature to obtain the boundary attention weight tensor. The boundary attention weight tensor is equal to the result of element-wise multiplication of the main scale feature and the corresponding position of the expanded attention weight map. The boundary attention weight tensor is used in the pixel rearrangement feature pyramid decoder to enhance the response to the edge, crack and bone spur areas of the sea bass pre-cooked dish.

[0111] The boundary attention weight tensor and the multi-scale feature tensor are fused in the pixel rearrangement feature pyramid decoder, and the initial segmentation mask is output through the channel-space rearrangement mechanism.

[0112] In this embodiment, the boundary attention weight tensor and the multi-scale feature tensor are fused in the pixel rearrangement feature pyramid decoder, including:

[0113] Within each scale, the feature tensor of the current scale is multiplied element-wise by channel at each spatial location with the boundary attention weight tensor of the scale matching to obtain the modulation feature tensor.

[0114] In Example 1, the boundary attention weight tensor is recorded as a three-dimensional tensor with the main scale height, main scale width, and main scale channel number. Bilinear upsampling is performed on the feature tensor of each scale in the multi-scale feature set to make the boundary attention weight tensor consistent with the feature tensor of the current scale in terms of spatial size, thus obtaining the scale-matched boundary attention weight tensor. The scale-matched boundary attention weight tensor is consistent with the feature tensor of the current scale in terms of height and width.

[0115] The value of each channel of the modulation feature tensor at each spatial location is equal to the product of the corresponding channel value of the current scale feature tensor and the boundary attention weight tensor. The modulation feature tensor is used to highlight the information representation of the edges, cracks and bone spurs of the sea bass pre-cooked dish in the channel dimension.

[0116] Apply a one-dimensional convolution kernel to each modulation feature tensor along the channel dimension to obtain a channel compression tensor;

[0117] The channel compression tensor has the same spatial dimensions as the modulation feature tensor, but the number of channels is compressed from the number of channels in the modulation feature tensor to a preset number of channels.

[0118] The channel compression tensor at higher scales is sampled from top to bottom, with the nearest neighbor upsampling operation performed sequentially to make its spatial size consistent with that of the channel compression tensor at the current scale. The fused tensor is then added to the channel compression tensor at the current scale at each spatial location and channel to obtain the fused tensor.

[0119] The fusion tensor integrates contextual information at different scales, enhancing the joint representation of the global structure and fine-grained features of pre-cooked sea bass dishes.

[0120] Apply a one-dimensional convolution kernel to each fusion tensor to obtain the channel expansion tensor;

[0121] The fusion tensor is subjected to a one-dimensional convolution kernel, which expands its number of channels to the square of the stride. The spatial size of the channel expansion tensor is the same as that of the fusion tensor. The number of channels is equal to the original number of channels multiplied by the square of the stride, and the stride is a positive integer.

[0122] Apply a channel-space rearrangement operation to each channel expansion tensor to decompose the channel dimension information of the channel expansion tensor by the square of the stride and rearrange it to the spatial dimension to obtain a sub-pixel-level upsampling tensor;

[0123] In Example 1, applying a channel-space rearrangement operation to each channel expansion tensor specifically involves: dividing the number of channels in the channel expansion tensor into several groups according to the square of the stride, with the number of channels in each group equal to the number of compressed channels; dividing all channel values ​​of each spatial position of the channel expansion tensor into stride multiplied by stride continuous subsequences, with the length of each subsequence equal to the number of compressed channels; mapping the stride multiplied by stride subsequences to the subpixel in the i-th row and j-th column of the original spatial position according to the two-dimensional coordinates (i, j) of the spatial dimension, forming a subpixel-level upsampled tensor with a spatial size that is stride times the expansion and the number of channels equal to the number of compressed channels; the information of the subpixel-level upsampled tensor in each compressed channel is equal to the result of rearranging all relevant subsequences divided by the channel expansion tensor at that spatial position to the corresponding new spatial coordinates according to the specified spatial index.

[0124] When the scale is at the shallowest level, a one-dimensional convolution kernel is applied to the sub-pixel-level upsampled tensor to obtain the category response tensor;

[0125] The spatial dimensions of the class response tensor are equal to the height and width of the preprocessed sea bass image, and the number of channels is equal to the number of classes. The class response tensor is used to quantize the response of each pixel across all classes.

[0126] An element-wise softmax operation is applied to the category response tensor along the channel dimension to obtain a preliminary segmentation mask.

[0127] The value of each spatial location of the initial segmentation mask in each category is equal to the ratio of the index of the response of that category to the sum of the indices of the responses of all categories.

[0128] A composite loss function is used to jointly optimize the sub-pixel-level segmentation network composed of the residual dense block context feature extraction network, the boundary topology branch and the pixel rearrangement feature pyramid decoder. The edge preservation loss is supervised by the boundary binary map and the topology regularization term is supervised by the Euler number heatmap to obtain the optimized sub-pixel-level segmentation network.

[0129] In engineering implementation, forward inference and backward training can only be performed by designing the operator flowchart of the network. Training is not an isolated process, but rather an end-to-end optimization of the already designed overall network structure. All composite loss functions in the claims need to be clearly defined. The edge-preserving loss is supervised by the boundary binary graph, and the topology regularization term is supervised by the Euler number heatmap. The parameters are optimized with these as the target.

[0130] In this embodiment, the sub-pixel-level segmentation network undergoes joint optimization, including:

[0131] The pixel-level category mask labeled by training is used as the pixel-level category supervision signal of the segmentation network. The non-background channel in the pixel-level category mask labeled by training is downsampled using the nearest neighbor downsampling method to obtain a binary foreground annotation map.

[0132] The pre-processed image of sea bass as the input to the sub-pixel segmentation network is used. The initial segmentation mask is used as the segmentation output of the sub-pixel segmentation network. The boundary response map is used as the edge output of the sub-pixel segmentation network. The boundary binary map is used as the edge supervision signal of the sub-pixel segmentation network. The Euler number heatmap is used as the topological supervision signal of the sub-pixel segmentation network. The pixel-level category mask for training and annotation is obtained by manually or automatically annotating the original image data of sea bass. The pixel-level categories include several preset categories such as sea bass body, boundary, bone, fracture and background. The pixel-level category mask is represented by a three-dimensional array with the same spatial size and number of channels as the pre-processed image of sea bass. Each channel is a binary distribution of the corresponding category and one-hot encoding is used.

[0133] Based on the initial segmentation mask and the pixel-level category mask of the training annotation, the category segmentation loss is calculated;

[0134] The class segmentation loss is obtained by calculating the pixel-by-pixel cross-entropy of the trained pixel-level class mask and the initial segmentation mask across all classes for each pixel location and each class channel. The class segmentation loss is used to measure the difference in class prediction between the initial segmentation mask and the trained pixel-level class mask.

[0135]

[0136] in, The class segmentation loss is represented by H0, which measures the pixel-by-pixel multi-class cross-entropy difference between the initial segmentation mask and the pixel-level class mask. A smaller value indicates better segmentation performance. H0 represents the height (in pixels) of the pre-processed sea bass dish image, W0 represents the width (in pixels) of the pre-processed sea bass dish image, and C... cls M represents the number of categories, which is equal to the total number of categories in the morphological integrity recognition of prepared sea bass dishes. gt (x,y,c) represents the label value of the pixel-level category mask at position (x,y) and category c, which takes the value 0 or 1. 1 indicates that the pixel belongs to category c, and 0 indicates that it does not. M0(x,y,c) represents the predicted probability value of the initial segmentation mask at position (x,y) and category c. ln(·) represents the natural logarithm function. max(ε,M0(x,y,c)) represents the lower bound of the ln function input to prevent it from being zero. ε is a very small positive number.

[0137] For each pixel in the downsampled binary foreground annotation map, if the value of the pixel is different from that of any eight neighboring pixels, it is assigned a value of one in the supervised boundary binary map at the corresponding pixel position. If the value of the pixel is the same as that of all eight neighboring pixels, it is assigned a value of zero in the supervised boundary binary map at the corresponding pixel position. Based on the boundary response map and the supervised boundary binary map, the edge preservation loss is calculated.

[0138] Each pixel in the supervised boundary binary map is either zero or one, marking the spatial location of the foreground region of the pre-cooked sea bass dish relative to the background or boundaries of different structures. This is an encoding of abrupt changes in the spatial distribution of the foreground label map after downsampling, reflecting the structural edge information of the pre-cooked sea bass dish.

[0139] The edge preservation loss is obtained by calculating the binary cross-entropy of the predicted probability log of the boundary response map and the actual label for each pixel at the position where the supervised boundary binary map is one, and by calculating the binary cross-entropy of the predicted probability log of the boundary response map and the actual label at the position where the supervised boundary binary map is zero. The two results are added together and averaged to obtain the edge preservation loss.

[0140]

[0141] in, This represents the edge-preservation loss, measuring the consistency between the predicted boundary and the true labeled boundary, emphasizing the ability to discriminate edges, breaks, and gaps in pre-cooked catfish dishes. H m W represents the height of the principal scale feature. m B represents the width of the principal scale feature. sup (x,y) represents the value at pixel position (x,y) in the supervised boundary binary map, R edge (x,y) represents the value of the boundary response map at pixel position (x,y).

[0142] Based on the binary foreground annotation map, a fixed window sliding neighborhood is used for each pixel to calculate the supervised Euler feature value. The supervised Euler feature value of each pixel is then subtracted from the mean and divided by the standard deviation to obtain the standardized supervised Euler feature value of each pixel. The absolute difference between the Euler number heatmap and the standardized supervised Euler feature value is averaged over all pixel locations to obtain the topology regularization term.

[0143] In Example 1, the number of connected components and the number of holes within a fixed window are counted. The number of connected components minus the number of holes is used to obtain the supervised Euler feature value of each pixel. The mean and standard deviation of the supervised Euler feature values ​​of all pixels are calculated over the entire image. The standardized supervised Euler feature value of each pixel is obtained by subtracting the mean from the supervised Euler feature value of each pixel and dividing by the standard deviation.

[0144] By weighting and summing the category segmentation loss, edge preservation loss, and topology regularization term, a composite loss function is obtained, and an optimized subpixel-level segmentation network is trained.

[0145] The parameter set of the residual dense block context feature extraction network, the boundary topology branch, and the pixel rearrangement feature pyramid decoder is used as a joint optimization variable. In the same training batch, the forward propagation results of the preprocessed sea bass dish image are used to generate an initial segmentation mask, boundary response map, and Euler number heatmap. The parameter set is synchronously updated through backpropagation with the composite loss function as the target until the composite loss function meets the preset convergence condition. The optimized sub-pixel-level segmentation network is then output, which is jointly defined by the parameter set of the residual dense block context feature extraction network, the boundary topology branch, and the pixel rearrangement feature pyramid decoder.

[0146] During the inference phase, the initial segmentation mask is refined and topological consistency is corrected based on the boundary binary map and Euler number heat map to obtain the target mask map.

[0147] In this embodiment, the preliminary segmentation mask is refined in terms of boundary and corrected in terms of topological consistency based on the boundary binary map and Euler number heatmap, including:

[0148] For each pixel in the boundary response map, if the predicted probability of the pixel is greater than or equal to the set edge threshold, the corresponding pixel position in the boundary mask map is assigned a value of one; otherwise, it is assigned a value of zero.

[0149] A boundary mask is a binary description of whether each pixel is a predicted edge pixel.

[0150] For all non-background category channels of the initial segmentation mask, take the maximum category response at each pixel location to obtain the initial foreground mask;

[0151] The preliminary foreground mask reflects the maximum response probability of each pixel across all non-background categories, providing a preliminary probability estimate of the main body region of the pre-cooked sea bass dish.

[0152] The initial foreground mask is subjected to soft thresholding binarization at each pixel position to generate a foreground region mask. The skeleton extraction operation, morphological thinning operation, morphological closing operation and region connection repair operation are sequentially performed on the overlapping area of ​​the foreground region mask and the boundary mask to obtain the thinned mask.

[0153] In Example 1, the initial foreground mask is compared with a preset probability threshold at each pixel position. When the probability value of the initial foreground mask at the corresponding pixel position is greater than or equal to the set threshold, the corresponding pixel position of the foreground region mask is assigned a value of one; otherwise, it is assigned a value of zero, thus obtaining the foreground region mask. The boundary mask image is adjusted to be consistent with the spatial size of the foreground region mask through interpolation. At each pixel position, if the boundary mask image is one at the corresponding pixel position, the corresponding pixel position is regarded as a boundary pixel. For the set of pixels where both the foreground region mask and the boundary mask image have a spatial position of one, a skeleton extraction operation is performed to preserve the main structure within the corresponding set. Then, a morphological thinning operation is applied to the skeleton extraction result to obtain a continuous curve with a single pixel width. A morphological closing operation is applied to the thinning result to fill the discontinuities or small holes in the skeleton curve. Finally, a region connection repair operation is applied to the closing operation result to merge small broken regions and enhance structural connectivity, thus obtaining a refined mask. Each pixel position of the refined mask being one indicates that the corresponding pixel belongs to the foreground region of the main body of the pre-cooked sea bass dish after boundary refinement and topology repair.

[0154] The Euler number heatmap is normalized by performing full-image mean normalization on all pixel response values.

[0155] The normalized Euler number heatmap is adjusted by interpolation to match the spatial size of the initial segmentation mask, and used as the Euler constraint map. It is then combined with the refined mask to perform topology consistency correction and output the topology correction mask.

[0156] In Example 1, the topology consistency correction is specifically as follows: combined with the refined mask, for each local region, when the Euler constraint map shows that the number of connected components in the corresponding region is significantly lower than the local average, the corresponding region is determined to be a pseudo-fracture region, and a connection operation is performed in the corresponding region to achieve region closure; when the Euler constraint map shows that the Euler eigenvalue of the corresponding region is higher than the local average, it is determined that there is redundant adhesion in the corresponding region, and a morphological erosion operation is performed in the corresponding region to achieve boundary separation.

[0157] The topology correction mask is mapped back to the category space. At each pixel location, if the corresponding pixel of the topology correction mask is one and the category corresponding to the largest category response of the corresponding pixel in the initial segmentation mask is a certain category, the category channel of the corresponding pixel of the final target mask is assigned a value of one; otherwise, it is assigned a value of zero.

[0158] The final target mask serves as the final segmentation result for identifying the morphological integrity of pre-cooked sea bass, ensuring both boundary accuracy and topological consistency.

[0159] The final target mask is represented as a three-dimensional array. The first dimension of the three-dimensional array is the height of the pre-processed sea bass pre-prepared image in pixels, the second dimension is the width of the pre-processed sea bass pre-prepared image in pixels, and the third dimension is the total number of categories in the sea bass pre-prepared image morphological integrity recognition. In each category channel, the two-dimensional mask of the corresponding category channel is used as the category region. The category region boundary set is extracted along the outer contour of the category region. The number of pixels in the category region boundary set is used as the category boundary perimeter, and the unit of the category boundary perimeter is the number of pixels.

[0160] Calculate a set of morphological integrity indices based on the target mask image;

[0161] This embodiment includes:

[0162] Label the main body class of the bass in the category channel, take the main body class channel of the bass as the main body region of the bass, and divide the total length of the pixel segments with concave gaps in the boundary by the perimeter of the boundary of the main body region of the bass to obtain the edge gap rate;

[0163] In Example 1, the boundary curvature value is calculated for each pixel of the boundary curve of the main body region of the sea bass. The boundary curvature value is compared with a preset curvature threshold. If the boundary curvature value at a certain pixel is greater than the curvature threshold, the corresponding pixel is determined to be a pixel of the boundary depression gap. All consecutive pixels determined to be depression gaps are grouped into pixel segments along the boundary curve, and the lengths of all pixel segments are summed to obtain the total length of the pixel segments of the depression gap in the boundary. The edge gap rate is used to measure the local defects of the overall boundary of the pre-cooked sea bass dish. The skeleton is extracted on the two-dimensional mask corresponding to the main body region of the sea bass to obtain the skeleton map. All non-closed endpoints are detected in the skeleton map, and the number of non-closed endpoints is counted as the number of endpoints. When the number of endpoints is greater than or equal to two, the number of endpoints is divided by two and rounded down to obtain the number of broken segments.

[0164] In Example 1, the skeleton extraction operation includes: in the final target mask, the bass body category channel is selected as the bass body region mask, and morphological erosion and thinning operations are repeatedly performed on the bass body region mask until all regions shrink to a connected structure with a width of one pixel, so as to obtain a skeleton map that retains only the main trunk shape. When the value of each pixel in the skeleton map is one, it means that the corresponding position belongs to the fish skeleton, and when it is zero, it means that it does not belong. The number of broken segments is used to evaluate the number of cracks and cross-sections in the main body of the pre-cooked bass dish.

[0165] In the final target mask, the area of ​​the bone spur region is divided by the area of ​​the main body of the bass to obtain the proportion of the exposed bone spur region.

[0166] In the final target mask, the bone spur category channel is selected as the bone spur region mask. The number of all pixels in the bone spur region mask is accumulated to obtain the area of ​​the bone spur region. The number of all pixels in the sea bass body region mask is accumulated to obtain the area of ​​the sea bass body region. The percentage of exposed bone spur area represents the relative proportion of the exposed bone spur area in the fish body region.

[0167] In the mask of the main body area of ​​the sea bass, the Euler feature values ​​corresponding to all the center pixels are combined to form the Euler feature value distribution. The absolute difference of the Euler feature value distribution and the normalized Euler number heatmap is calculated pixel by pixel. The absolute difference at all pixel positions is accumulated and then divided by the total number of pixels to obtain the Euler number difference.

[0168] The Euler number difference is used to measure the overall topological consistency between the final segmentation result and the network's predicted Euler structure. The larger the Euler number difference, the greater the deviation between the topological structure of the segmentation result and the expectation.

[0169] The four indicators—edge notch rate, number of fracture segments, proportion of exposed bone spur area, and difference between Euler number—are used as a set of morphological integrity indicators.

[0170] Input the set of morphological integrity indicators into the integrity scoring module, and the integrity scoring module outputs the integrity judgment result according to the preset scoring threshold.

[0171] In this embodiment, the set of morphological integrity indicators is input into the integrity scoring module, including:

[0172] Input the set of morphological integrity indicators into the integrity scoring module, and output the integrity category of the prepared sea bass dish:

[0173] When the edge notch rate is less than or equal to the first notch rate threshold, the number of fracture segments is zero, the proportion of exposed bone spur area is less than or equal to the first exposed area threshold, and the difference in Euler number is less than or equal to the first set threshold, the output integrity category is complete.

[0174] When the edge notch rate is greater than the first notch rate threshold and less than or equal to the second notch rate threshold, or the number of broken segments is equal to one, or the proportion of the exposed area of ​​the bone spur is greater than the first exposed area threshold and less than or equal to the second exposed area threshold, or the difference in the Euler number is greater than the first set threshold and less than or equal to the second set threshold, the output integrity category is mild defect.

[0175] When the edge notch rate is greater than the second notch rate threshold, or the number of fracture segments is greater than or equal to two, or the proportion of the exposed area of ​​the bone spur is greater than the second exposed area threshold, or the difference in the Euler number is greater than the second set threshold, the output integrity category is severe defect.

[0176] In Example 1, the first gap rate threshold, the second gap rate threshold, the first exposed area threshold, the second exposed area threshold, the first set threshold, and the second set threshold are determined by historical sample statistics or industry standards, respectively.

[0177] The integrity scoring module calculates the confidence probability of the integrity category based on the statistical distribution of the training samples using a normalized scoring function. Based on the relationship between the confidence probability and the threshold for dividing the confidence level, it outputs the confidence level.

[0178] When the confidence probability is greater than or equal to the first confidence level threshold, the output confidence level is high confidence.

[0179] When the confidence probability is greater than or equal to the second confidence threshold and less than the first confidence threshold, the output confidence level is medium confidence.

[0180] When the confidence probability is less than the second confidence threshold, the output confidence level is low confidence.

[0181] The first confidence threshold and the second confidence threshold are determined based on the industrial scenario, and the first confidence threshold is greater than the second confidence threshold.

[0182] The integrity scoring module outputs the defect type based on the morphological integrity index triggered during the integrity category determination process and its corresponding threshold conditions:

[0183] When the edge notch ratio is greater than the first notch ratio threshold, the defect type is edge notch;

[0184] When the number of fracture segments is greater than or equal to one, the defect type is structural fracture;

[0185] When the proportion of the exposed area of ​​bone spurs is greater than the threshold of the first exposed area, the defect type is exposed bone spurs;

[0186] When the difference in the Euler count is greater than the first set threshold, the defect type is topology anomaly.

[0187] A machine vision-based method for quality control of pre-cooked sea bass dishes further includes: sending the integrity determination result and the location information corresponding to the target mask image to the sorting control module; the sorting control module drives the sorting equipment to perform defective product rejection or alarm; and storing the integrity determination result and the target mask image in the quality database.

[0188] Example 2: On a certain batch of sea bass pre-cooked food production line, raw material pretreatment was carried out: sea bass with qualified freshness (TVB-N value ≤12mg / 100g, pH value 6.2-6.8) were selected, and a gradient thawing method was adopted: thawing at -4℃ for 8 hours → thawing at 4℃ for 4 hours. After thawing, the core temperature of the fish meat was controlled at 0-2℃. After rinsing with clean water, the scales, gills and internal organs were removed, and the fish was cut into 3-5cm pieces.

[0189] Compound deodorization process: Soak fish pieces in compound deodorization solution (0.8% perilla leaf extract + 1.2% citric acid + 0.5% ginger extract, solvent is deionized water), liquid to material ratio 10:1 (mL:g), soak at 25℃ for 30 minutes, rinse twice with deionized water after soaking, and drain.

[0190] Precise degreasing treatment: The "enzymatic hydrolysis-ultrasound synergistic degreasing method" is adopted. The fish pieces are soaked in lipase solution (enzyme activity 5000U / g, added at 0.3% of the fish weight) and enzymatically hydrolyzed for 20 minutes at 45℃. At the same time, ultrasound with power of 300W and frequency of 40kHz is applied to assist the process. After degreasing, the fish pieces are centrifuged (3000r / min, 5 minutes) to remove free fat.

[0191] Low-temperature sterilization process: First, soak the fish pieces in a 0.6% honeysuckle-forsythia compound extract solution (mass ratio 1:1) for 15 minutes, then treat them with low-temperature plasma (power 80W, treatment time 60 seconds), and then quickly cool them to below 4℃ after sterilization.

[0192] Quality Inspection: The system continuously detected images of the finished sea bass with the number A059. The conveyor belt speed was controlled at 60 samples per minute. The industrial camera automatically captured the original image of sample A059. The image resolution was 2048×1536 pixels. The original image contained obvious highly reflective areas and sauce patches, and the background had some distortion.

[0193] The system automatically performs illumination normalization on the A059 image, normalizing the maximum pixel value to 180 and the minimum pixel value to 18; after color normalization, the mean values ​​of the RGB three channels are [120, 115, 118]; after distortion correction, the offset of the fish's principal axis linear fitting is less than 1.2 degrees. The final preprocessed image is sent into the analysis process in the form of a three-dimensional tensor (2048×1536×3).

[0194] The residual dense block context feature extraction network performs convolution operations on the A059 preprocessed image. The initial convolution kernel space size is 3×3, and the number of channels is expanded from 3 to 64. The scale-1 feature output tensor space size is 2048×1536×64. The mean value of the output features of the first layer of the dense block is 0.037, and that of the second layer is 0.062. After concatenation of the three layers, the channel dimension reaches 192. Through the color transformation matrix, the peak value of the image luminance component is concentrated at 120, and the chrominance component is evenly distributed. The mean weight of the reflection suppression weight map in the highlight area of ​​the fish body is 0.41, and the mean weight of the dark area is 0.86. After interpolation, it is completely aligned with the feature tensor in space. The calculated principal axis direction angle is 5.3 degrees. The weighting coefficient of the convolution weight of the orientation adaptive convolution kernel in the principal axis direction of the fish body is 0.88. The maximum value of the high-frequency response feature appears in the crack area of ​​the fish's abdomen.

[0195] In the multi-scale feature set, the feature tensor space size of scale 1 is 2048×1536×64, scale 2 is 1024×768×128, and scale 3 is 512×384×256. Each scale feature is fused by dilated convolution, and the context feature response map is enhanced by an average of 0.18 near the fracture boundary of the fish body.

[0196] After inputting the principal scale feature tensor (1024×768×128) into the boundary topology branch, a 2D convolution is performed to obtain the boundary response map. The average boundary response map value is 0.73 at the edge of the fish's head, and the maximum value is 0.95 in the abdominal fracture area. With an edge threshold set to 0.68, the boundary mask map is judged as one on the main boundary of the fish and as zero in the small segment of the sauce-covered area.

[0197] For sample A059, convolution and activation were fused into a single-channel feature map with a window size of 7×7. The mean Euler feature of the main body window of the fish was 1.02, and the Euler feature value at the break point dropped to 0.33. The mean Euler number heatmap after standardization was 0, and the variance was 0.37.

[0198] Combining the principal scale features with the boundary response map, the maximum weight after stretching appears at the main fracture (weight 0.94), and the minimum weight appears at the edge of the fish's tail (weight 0.13). After expansion, the number of channels is consistent with the principal scale features, and the modulation feature tensor is enhanced by 27% in the crack region.

[0199] Multi-scale modulation features are compressed through one-dimensional convolution channels, with the channel compression tensor space size being 1 / 4 of the original number of channels. The high-scale channel compression tensors are upsampled to the main scale, added to the main scale channel compression tensors, and then fused, resulting in an average response improvement of 0.11 at the main boundary of the fused tensor. The fused tensor is then expanded through one-dimensional convolution with a stride of 2, and the number of channels in the expanded tensor is 4 times the number of compressed channels. The channel-space rearrangement operation has a stride of 2, expanding the sub-pixel-level upsampled tensor space size to 2048×1536×64.

[0200] The subpixel-level upsampling tensor is used to generate a class response tensor through one-dimensional convolution. The class response tensor space size is 2048×1536×5, and the channels correspond to the main body of the sea bass, the bones, the fracture, the gap, and the background. After the Softmax operation, the initial segmentation mask has a main class probability of 0.97 in the main body region and a maximum probability of 0.82 in the bone region.

[0201] During model training, the initial segmentation mask and annotation mask for the A059 samples had a cross-entropy loss of 0.047 for the main category, an edge preservation loss of 0.019, a topological regularization term of 0.012, and an overall composite loss of 0.027. After mini-batch training with A059 samples, the model parameters were further adjusted, the convergence speed of the boundary response map at the gaps was significantly accelerated, and the local consistency of the Euler number heatmap was improved by 12%.

[0202] During the inference phase, after binarization of the boundary response map of sample A059 with a threshold of 0.68, the number of pixels in the main boundary of the boundary mask map is 1783. The maximum response probability of the main category channel in the initial segmentation mask is 0.97. With a soft threshold of 0.6, the area of ​​the main region of the binarized foreground region mask is 17876 pixels. After skeleton extraction, the width of the main structure is one pixel, the number of non-closed endpoints is 2, and the number of broken segments is 1. After morphological thinning operations, the boundary connectivity index increases to 0.99, and the closing operation repairs 45 pixels of small broken areas.

[0203] The Euler number heatmap, after normalization of the overall mean, has a maximum absolute value of 0.64. After interpolation to the segmentation mask space, the absolute mean difference between the Euler features in the topology-corrected mask region and the segmented region is 0.08. Topology consistency correction repairs the tiny connectivity breaks in misjudged fish bodies, resulting in a final mask structure connectivity of 1.

[0204] Based on the target mask image, edge curvature analysis of sample A059 detected a total edge gap pixel length of 42, a fish body boundary perimeter of 1835, and an edge gap rate of 0.023. Skeleton analysis revealed 1 broken segment, a bone spur area of ​​78 pixels, a main fish body area of ​​17876 pixels, and a bone spur exposed area ratio of 0.0043. Comparison of Euler feature distribution with the normalized Euler number heatmap showed a mean absolute difference of 0.061.

[0205] The system inputs the indicators into the integrity scoring module. Based on the first gap rate threshold, the first exposure area threshold, and the first set threshold of the historical sample statistics of this system, the system determines that the integrity category of sample A059 is "slight defect", the confidence level is "medium confidence", and the defect type is "structural fracture".

[0206] During the same period, samples from batches A066 and A092 were randomly selected. After system identification, A066 had an edge notch rate of 0.011, 0 fracture segments, 0.0016% of the area exposed by bone spurs, and a Euler number difference of 0.034, and was judged as "intact"; A092 had 2 fracture segments, 0.0078% of the area exposed by bone spurs, and a Euler number difference of 0.12, and was judged as "severely defective". The defect types included "structural fracture" and "exposed bone spurs".

[0207] Compared with the traditional U-Net method for detecting sample A059, the U-Net output mask main class probability was 0.89, the length of the blurred boundary segment was relatively long, the number of broken segments was not detected correctly, and the Euler number difference was as high as 0.14, which misclassified it as "intact" and missed the detection of structural fracture defects. The traditional method mistakenly classified the sauce highlight area of ​​sample A066 as a gap and judged it as "slight defect", while the actual manual inspection was "intact".

[0208] On the test set for integrity category determination, this system achieved a minor defect detection rate of 94.6%, a severe defect detection rate of 91.2%, and a false positive rate of 2.7%, which is superior to the traditional pixel-level segmentation methods of 86.7%, 83.4%, and 7.1%, respectively. The average inference time per fish is 41 milliseconds, meeting the actual requirements of the production line.

[0209] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for quality control of pre-prepared sea bass dishes based on machine vision, characterized in that, include: Image data of pre-cooked sea bass during the conveying process of the pre-cooked sea bass on the production line are collected and image preprocessing is performed to obtain pre-processed images of pre-cooked sea bass. The pre-processed image of the prepared sea bass dish is input into the residual dense block context feature extraction network, which outputs a multi-scale feature tensor. Input the multi-scale feature tensor into the boundary topology branch, and the boundary topology branch generates a boundary binary map, an Euler number heatmap, and a boundary attention weight tensor. The boundary attention weight tensor and the multi-scale feature tensor are fused in the pixel rearrangement feature pyramid decoder, and the initial segmentation mask is output through the channel-space rearrangement mechanism. A composite loss function is used to jointly optimize the sub-pixel-level segmentation network composed of the residual dense block context feature extraction network, the boundary topology branch and the pixel rearrangement feature pyramid decoder. The edge preservation loss is supervised by the boundary binary map and the topology regularization term is supervised by the Euler number heatmap to obtain the optimized sub-pixel-level segmentation network. During the inference phase, the initial segmentation mask is refined and topological consistency is corrected based on the boundary binary map and Euler number heat map to obtain the target mask map. Calculate a set of morphological integrity indices based on the target mask image; Input the set of morphological integrity indicators into the integrity scoring module, and the integrity scoring module outputs the integrity judgment result according to the preset scoring threshold.

2. The method for quality control of pre-prepared sea bass dishes based on machine vision according to claim 1, characterized in that, The image preprocessing includes illumination normalization, color normalization, and distortion correction.

3. The method for quality control of pre-prepared sea bass dishes based on machine vision according to claim 1, characterized in that, The step of inputting the pre-processed sea bass dish image into the residual dense block context feature extraction network includes: Represent the pre-processed sea bass dish image in three dimensions; By performing two-dimensional discrete convolution operations on the three dimensions using the initial convolution kernel and the initial bias, the input features of scale one are obtained. A preset number of residual dense blocks are stacked at each scale. After all the input features of the previous layer are concatenated according to the channel dimension through the built-in dense connection layer, the dense connection output features are output. Based on the pre-processed image of sea bass, the channels of each pixel are linearly combined using a color transformation matrix to obtain the luminance component and chrominance component. The luminance component and chrominance component are weighted and summed with a set of scalar weight coefficients and biases, and then mapped through a nonlinear activation function to obtain a reflection suppression weight map. Based on the input features at the current scale, the Sobel operator is used to calculate the horizontal and vertical gradients respectively, and average pooling is performed to obtain structural elements. The principal axis direction angle is then calculated based on the structural elements. The set of pre-defined directional base convolutional kernels is weighted and interpolated based on the principal axis direction angle, and the interpolated directional adaptive convolutional kernels replace the convolutional kernels of the residual dense blocks. At each scale, a fixed Laplacian kernel is used to perform two-dimensional convolution on the input features to obtain high-frequency response features. The densely connected output features and high-frequency response features are weighted proportionally and concatenated. After channel alignment by a one-dimensional convolution kernel, they are residually superimposed with the input features to form new input features. After stacking all residual dense blocks at each scale, a one-dimensional convolution kernel and bias are used to perform channel transformation on the new input features to output the scale features of the current scale. At the same time, a downsampling convolution kernel with a stride of two is used to perform a one-dimensional convolution operation on the scale features of the current scale to generate the input features of the next scale. Multiple sets of dilated convolutions are applied to the scale features at each scale. The output features of each set of dilated convolutions are summed according to the channel dimension to obtain the scale context features at each scale. The scale context features of all scales are then combined into a multi-scale feature set.

4. The method for quality control of pre-prepared sea bass dishes based on machine vision according to claim 1, characterized in that, The step of inputting multi-scale feature tensors into the boundary topology branch includes: Select principal scale features from the multi-scale feature set, apply a two-dimensional convolution operation to the principal scale features with a bias, and then map them using an element-wise sigmoid activation function to obtain the boundary response map; When the value of the boundary response map at the corresponding pixel position is greater than or equal to the preset threshold, the value of the boundary binary map at the corresponding pixel position is set to one; otherwise, it is set to zero, thus obtaining a binary boundary mask map. For each channel of the main scale feature, a 3x3 two-dimensional convolution operation and element-wise ReLU activation function mapping are performed and fused into a single-channel feature map. Based on the single-channel feature map, a sliding window neighborhood is constructed for each pixel. Euler feature values ​​are calculated in each sliding window according to the local connected graph structure, and an Euler number heatmap is obtained. The principal scale features and the boundary response map are input into the boundary attention module to obtain the boundary attention weight tensor.

5. The method for quality control of pre-prepared sea bass dishes based on machine vision according to claim 1, characterized in that, The fusion of the boundary attention weight tensor and the multi-scale feature tensor in the pixel rearrangement feature pyramid decoder includes: Within each scale, the feature tensor of the current scale is multiplied element-wise by channel at each spatial location with the boundary attention weight tensor of the scale matching to obtain the modulation feature tensor. Apply a one-dimensional convolution kernel to each modulation feature tensor along the channel dimension to obtain a channel compression tensor; The channel compression tensor at higher scales is sampled from top to bottom, with the nearest neighbor upsampling operation performed sequentially to make its spatial size consistent with that of the channel compression tensor at the current scale. The fused tensor is then added to the channel compression tensor at the current scale at each spatial location and channel to obtain the fused tensor. Apply a one-dimensional convolution kernel to each fusion tensor to obtain the channel expansion tensor; Apply a channel-space rearrangement operation to each channel expansion tensor to decompose the channel dimension information of the channel expansion tensor by the square of the stride and rearrange it to the spatial dimension to obtain a sub-pixel-level upsampling tensor; When the scale is at the shallowest level, a one-dimensional convolution kernel is applied to the sub-pixel-level upsampled tensor to obtain the category response tensor; An element-wise softmax operation is applied to the category response tensor along the channel dimension to obtain a preliminary segmentation mask.

6. The method for quality control of pre-prepared sea bass dishes based on machine vision according to claim 1, characterized in that, The sub-pixel-level segmentation network is jointly optimized, including: The pixel-level category mask of the training annotation is used as the pixel-level category supervision signal of the segmentation network. The non-background channel in the pixel-level category mask of the training annotation is downsampled using the nearest neighbor downsampling method to obtain a binary foreground annotation map. Based on the initial segmentation mask and the pixel-level category mask of the training annotation, the category segmentation loss is calculated; For each pixel in the binary foreground annotation map obtained by downsampling, if the value of the pixel is different from that of any eight neighboring pixels, it is assigned a value of one in the supervised boundary binary map at the corresponding pixel position; if the value of the pixel is the same as that of all eight neighboring pixels, it is assigned a value of zero in the supervised boundary binary map at the corresponding pixel position. Based on the boundary response map and the supervised boundary binary map, the edge preservation loss is calculated. Based on the binary foreground annotation map, a fixed window sliding neighborhood is used for each pixel to calculate the supervised Euler feature value. The supervised Euler feature value of each pixel is then subtracted from the mean and divided by the standard deviation to obtain the standardized supervised Euler feature value of each pixel. The absolute difference between the Euler number heatmap and the standardized supervised Euler feature value is averaged over all pixel locations to obtain the topology regularization term. By weighting and summing the category segmentation loss, edge preservation loss, and topology regularization term, a composite loss function is obtained, and an optimized subpixel-level segmentation network is trained.

7. The method for quality control of pre-prepared sea bass dishes based on machine vision according to claim 1, characterized in that, The process of refining the boundaries and correcting the topology consistency of the initial segmentation mask based on the boundary binary map and Euler number heatmap includes: For each pixel in the boundary response map, if the predicted probability of the pixel is greater than or equal to the set edge threshold, the corresponding pixel position in the boundary mask map is assigned a value of one; otherwise, it is assigned a value of zero. For all non-background category channels of the initial segmentation mask, take the maximum category response at each pixel location to obtain the initial foreground mask; The initial foreground mask is subjected to soft thresholding binarization at each pixel position to generate a foreground region mask. The skeleton extraction operation, morphological thinning operation, morphological closing operation and region connection repair operation are sequentially performed on the overlapping area of ​​the foreground region mask and the boundary mask to obtain the thinned mask. The Euler number heatmap is normalized by performing full-image mean normalization on all pixel response values. The normalized Euler number heatmap is adjusted by interpolation to match the spatial size of the initial segmentation mask, and used as the Euler constraint map. It is then combined with the refined mask to perform topology consistency correction and output the topology correction mask. The topology correction mask is mapped back to the category space. At each pixel location, if the corresponding pixel of the topology correction mask is one and the category corresponding to the largest category response of the corresponding pixel in the initial segmentation mask is a certain category, the category channel of the corresponding pixel of the final target mask is assigned a value of one; otherwise, it is assigned a value of zero.

8. The method for quality control of pre-prepared sea bass dishes based on machine vision according to claim 1, characterized in that, The set of morphological integrity indicators calculated based on the target mask image includes: Label the main body class of the bass in the category channel, take the main body class channel of the bass as the main body region of the bass, and divide the total length of the pixel segments with concave gaps in the boundary by the perimeter of the boundary of the main body region of the bass to obtain the edge gap rate; Skeleton extraction is performed on a two-dimensional mask corresponding to the main body area of ​​the sea bass to obtain a skeleton map. All non-closed endpoints are detected in the skeleton map, and the number of non-closed endpoints is counted as the endpoint count. When the endpoint count is greater than or equal to two, the endpoint count is divided by two and rounded down to obtain the number of broken segments. In the final target mask, the area of ​​the bone spur region is divided by the area of ​​the main body of the bass to obtain the proportion of the exposed bone spur region. In the mask of the main body area of ​​the sea bass, the Euler feature values ​​corresponding to all the center pixels are combined to form the Euler feature value distribution. The absolute difference of the Euler feature value distribution and the normalized Euler number heatmap is calculated pixel by pixel. The absolute difference at all pixel positions is accumulated and then divided by the total number of pixels to obtain the Euler number difference. The four indicators—edge notch rate, number of fracture segments, proportion of exposed bone spur area, and difference between Euler number—are used as a set of morphological integrity indicators.

9. The method for quality control of pre-prepared sea bass dishes based on machine vision according to claim 1, characterized in that, The step of inputting the set of morphological integrity indicators into the integrity scoring module includes: Input the set of morphological integrity indicators into the integrity scoring module, and output the integrity category of the prepared sea bass dish: When the edge notch rate is less than or equal to the first notch rate threshold, the number of fracture segments is zero, the proportion of exposed bone spur area is less than or equal to the first exposed area threshold, and the difference in Euler number is less than or equal to the first set threshold, the output integrity category is complete. When the edge notch rate is greater than the first notch rate threshold and less than or equal to the second notch rate threshold, or the number of broken segments is equal to one, or the proportion of the exposed area of ​​the bone spur is greater than the first exposed area threshold and less than or equal to the second exposed area threshold, or the difference in the Euler number is greater than the first set threshold and less than or equal to the second set threshold, the output integrity category is mild defect. When the edge notch rate is greater than the second notch rate threshold, or the number of broken segments is greater than or equal to two, or the proportion of the exposed area of ​​bone spurs is greater than the second exposed area threshold, or the difference in the number of Eulers is greater than the second set threshold, the output integrity category is severe defect.

10. A method for quality control of pre-prepared sea bass dishes based on machine vision according to claim 9, characterized in that, The integrity scoring module outputs the defect type based on the morphological integrity index triggered by the integrity category determination process and its corresponding threshold conditions: When the edge notch ratio is greater than the first notch ratio threshold, the defect type is edge notch; When the number of fracture segments is greater than or equal to one, the defect type is structural fracture; When the proportion of the exposed area of ​​bone spurs is greater than the threshold of the first exposed area, the defect type is exposed bone spurs; When the difference in the Euler count is greater than the first set threshold, the defect type is topology anomaly.

Images