Visual analysis and screening method for appearance characteristics of fish maw

Abstract

The invention discloses a visual analysis and screening method for fish maw appearance features, and relates to the technical field of computer vision and intelligent sorting, and the method comprises the steps: carrying out the polarization reflection feature analysis of a fish maw main region image, constructing a surface direction-reflection relation tensor, and carrying out the clustering to generate an anisotropic reflection texture feature pattern; gradient pyramid fusion is carried out on the anisotropic reflection texture feature map and the RGB image, contour information is extracted, and a fish maw boundary enhanced image is generated; inputting the fish maw boundary enhanced image into a contour detection network, performing automatic modeling, outputting a high-precision boundary thermodynamic diagram, and generating a high-precision boundary contour diagram; and carrying out feature fusion on the anisotropic reflection texture feature map and the high-precision boundary contour map, constructing a fish maw appearance description vector, carrying out appearance grade classification through a multi-layer neural network model in combination with a historical evaluation sample, and outputting a fish maw sample classification result. The accuracy and efficiency of fish maw quality detection are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and intelligent sorting, in particular to a visual analysis and screening method for the appearance features of fish glue. BACKGROUND

[0002] In the field of food processing and quality control of traditional Chinese medicinal materials, the appearance features (such as texture uniformity, boundary integrity, surface glossiness, etc.) of fish glue are important basis for judging its quality grade. The traditional manual visual detection method relies on experienced quality inspection personnel, and has the problems of strong subjectivity and low efficiency. In recent years, automatic detection technology based on machine vision has been gradually applied in the field of agricultural products and traditional Chinese medicinal materials sorting, such as using hyperspectral imaging or structured light three-dimensional reconstruction technology to obtain the surface morphology information of the object, and combining support vector machine (SVM) or convolutional neural network (CNN) for classification decision. In the prior art, the HSV features are extracted through color space conversion, and the preliminary grading is realized by combining the texture analysis algorithm.

[0003] Further research found that the surface quality of fish glue is closely related to the arrangement direction of its collagen fibers, and the uniformity of fiber arrangement will directly affect the distribution mode of polarized reflected light. The existing technical solutions based on RGB or single light source imaging lack multi-angle polarized light field data acquisition and fusion mechanism, and cannot effectively capture the anisotropy features of the surface microstructure. In addition, the existing grading model relies on static feature extraction (such as LBP or GLCM), and it is difficult to dynamically fuse multi-modal optical features (such as polarized reflection and RGB space information), which limits the generalization ability of the classification model. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a visual analysis and screening method for the appearance features of fish glue to solve the problems of insufficient analysis of the surface microstructure of fish glue and limited generalization ability of the classification model in the prior art.

[0006] To solve the above technical problems, the present application provides the following technical solutions: In a first aspect, the present application provides a visual analysis and screening method for the appearance features of fish glue, which comprises: collecting an RGB image and a multi-angle polarized reflection image group of fish glue; The multi-angle polarized reflection image group is preprocessed to form a fish glue multi-light source image dataset, an image segmentation algorithm is used to extract a fish glue main region mask, and smoothing processing is performed to obtain a fish glue main body region image; The fish glue main body region image is subjected to polarized reflection feature analysis, a surface direction-reflection relationship tensor is constructed, and an anisotropic reflection texture feature map is generated by clustering; The anisotropic reflectance texture feature map is fused with the RGB image using a gradient pyramid method and the contour information is extracted to generate a fish maw boundary enhanced image. The fish maw boundary enhancement image is input into the contour detection network, and automatic modeling is performed to output a high-precision boundary heat map and generate a high-precision boundary contour map; The anisotropic reflective texture feature map and the high-precision boundary contour map are fused to construct the fish maw appearance description vector. The appearance grade classification is performed through a multi-layer neural network model combined with historical evaluation samples, and the fish maw sample grading results are output.

[0007] As a preferred solution of the visual analysis and screening method of the appearance characteristics of fish maw described in the present invention, the multi-angle polarized reflection image group is sequentially subjected to image size unification, color space conversion, brightness normalization, image denoising and gamma correction to form a fish maw multi-light source image data set.

[0008] As a preferred solution of the visual analysis and screening method of the appearance characteristics of fish maw of the present invention, the image segmentation algorithm is used to extract the mask of the main area of ​​the fish maw, and smoothing is performed to obtain the image of the main area of ​​the fish maw. The specific steps are as follows: By calculating the reflection intensity difference of the corresponding pixels of each group of images with different polarization angles in the fish maw multi-light source image dataset and obtaining the global average reflection difference value, the polarization angle image with the largest average reflection difference value is selected as the significant polarization image, and then converted to the standard color space to extract the brightness component image; Perform edge detection on the brightness component image to extract the brightness change mutation area, generate brightness edge information, and extract the main area mask of the fish maw through image segmentation algorithm; Perform contour detection on the edge of the mask of the main area of ​​the fish maw to obtain an ordered boundary point set, and perform smoothing to generate the edge contour of the fish maw; The fish maw multi-light source image dataset was subjected to mask cropping and pixel normalization processing using the edge contour of the fish maw to form the image of the main area of ​​the fish maw.

[0009] As a preferred embodiment of the visual analysis and screening method for the appearance characteristics of fish maws of the present invention, the polarized reflection characteristic analysis is performed on the image of the main area of ​​the fish maw, a surface direction-reflection relationship tensor is constructed, and an anisotropic reflection texture feature map is generated by clustering. The specific steps are as follows: For each pixel in the image of the main area of ​​the fish maw, a corresponding virtual pulse neuron is constructed, the reflection intensity difference is input into the corresponding virtual pulse neuron, and the membrane potential accumulation calculation is performed to obtain the membrane potential value; The membrane potential value is compared with the pulse triggering threshold. When the membrane potential value exceeds the pulse triggering threshold, a pulse signal is triggered to generate a spatiotemporal pulse response matrix. Taking the spatiotemporal impulse response matrix as input excitation, a surface direction-reflection relationship tensor field modulated by pulse intensity is constructed in a three-dimensional space coordinate system, and the steady-state tensor field is obtained by solving the nonlinear diffusion equation. According to the local eigenvector distribution of the steady-state tensor field, adaptive weight learning is performed in combination with the temporal correlation of the pulse signals of adjacent pixels to divide the phase-consistent texture area. The phase-consistent texture area is mapped to the two-dimensional image space according to the anisotropic response level to generate an anisotropic reflectance texture feature map.

[0010] As a preferred embodiment of the visual analysis and screening method for the appearance characteristics of fish maws of the present invention, the anisotropic reflective texture feature map is subjected to gradient pyramid fusion with the RGB image and contour information is extracted to generate a fish maw boundary enhanced image. The specific steps are as follows: Convert the RGB image to LAB color space and extract the L channel as the brightness channel of the RGB image; The anisotropic reflectance texture feature map and the RGB image brightness channel are decomposed into Gaussian pyramids respectively, and downsampled by a fixed scale factor to generate a gradient pyramid hierarchical structure; At each level of the gradient pyramid structure, the Sobel operator is used to calculate the horizontal and vertical gradients of the anisotropic reflectance texture feature map and the luminance channel of the RGB image, respectively. The gradient amplitudes are compared and the larger gradient amplitude is retained to generate an optimized gradient amplitude map. Starting from the top layer of the gradient pyramid hierarchy, the optimized gradient magnitude map is upsampled layer by layer and superimposed. The superposition results are histogram-equalized to generate a fused base map. Based on the gradient amplitude distribution of the fused base image, contour points are extracted and connected to form closed contour segments; Closed contour segments are superimposed on the RGB image to generate a fish maw boundary enhanced image.

[0011] As a preferred solution of the visual analysis and screening method of the appearance characteristics of fish maws of the present invention, the fish maw boundary enhancement image is input into the contour detection network, and automatic modeling is performed to output a high-precision boundary heat map and generate a high-precision boundary contour map. The specific steps are as follows: The fish maw boundary enhancement image is subjected to local contrast normalization to generate a normalized boundary enhancement image, which is then input into the contour detection network for multi-scale boundary feature extraction, generating skip connection features and dilated convolution features. The skip connection features are fused with the dilated convolution features to generate enhanced boundary features, and channel weighted response processing is performed to generate the initial boundary heat map; Perform non-maximum suppression on the initial boundary heat map to generate a high-precision boundary heat map; The local maximum points are extracted from the high-precision boundary heat map, and the high-precision boundary contour map is generated after verification of gradient direction connectivity.

[0012] As a preferred solution of the visual analysis and screening method of the appearance characteristics of fish maw described in the present invention, the anisotropic reflective texture feature map and the high-precision boundary contour map are spatially aligned, the anisotropic reflective texture feature map is used as the first input channel, the high-precision boundary contour map is used as the second input channel, the first input channel and the second input channel are spliced ​​along the channel dimension to generate a multi-channel fusion feature map, and global average pooling operation and maximum and minimum normalization processing are performed to generate a fish maw appearance description vector.

[0013] As a preferred embodiment of the visual analysis and screening method for the appearance characteristics of fish maw of the present invention, the method comprises the following steps: performing appearance grade classification by combining historical evaluation samples with a multi-layer neural network model, and outputting the grading results of the fish maw samples. The historical fish maw appearance description vector is used as training data, and the multi-layer neural network model parameters are updated through the back propagation algorithm to obtain the trained multi-layer neural network model; The fish maw appearance description vector is input into the trained multi-layer neural network model, and deep classification features are extracted through fully connected layer operations; The deep classification features are sequentially activated by ReLU and normalized by Softmax to generate a level probability distribution; The first-level standard probability interval, the second-level standard probability interval and the third-level standard probability interval are extracted from the historical evaluation samples, and the grade probability distribution is graded and evaluated to obtain the grading results of fish maw samples.

[0014] In a second aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the visual analysis and screening method of fish maw appearance characteristics as described in the first aspect of the present invention is implemented.

[0015] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the visual analysis and screening method of fish maw appearance characteristics as described in the first aspect of the present invention is implemented.

[0016] The beneficial effects of the present invention are: through polarization reflection feature analysis (constructing a pulse neuron model and tensor field clustering to generate anisotropic reflection texture feature maps) and gradient pyramid fusion (multi-scale optimization of RGB and polarization features to enhance contours), the problems of insufficient micro-texture analysis and poor robustness of contour detection in traditional fish maw sorting technology are solved, high-precision surface defect recognition, multi-modal feature collaborative optimization and automated intelligent grading are achieved, and the accuracy and efficiency of fish maw quality detection are significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 Flowchart of the visual analysis and screening method for fish maw appearance characteristics.

[0019] Figure 2 A flowchart constructed for a virtual spiking neuron.

[0020] Figure 3 Flowchart for the generation and processing of space-time impulse response matrix.

[0021] Figure 4 Flowchart built for anisotropic reflectance texture feature maps. DETAILED DESCRIPTION

[0022] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0023] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0025] Reference Figures 1 to 4 , is an embodiment of the present invention, which provides a method for visual analysis and screening of fish maw appearance characteristics, comprising the following steps: S1. Collect RGB images and multi-angle polarized reflectance images of fish maw.

[0026] RGB images include standard red, green, and blue three-channel color images acquired under visible light illumination.

[0027] The multi-angle polarized reflective image group includes fish maw surface reflective images collected at different polarization angles.

[0028] S2. Preprocess the multi-angle polarized reflection image group to form a fish maw multi-light source image dataset, use an image segmentation algorithm to extract the fish maw main area mask, and perform smoothing to obtain the fish maw main area image.

[0029] The multi-angle polarized reflectance image group was subjected to image size unification, color space conversion, brightness normalization, image denoising and gamma correction in sequence to form a multi-light source image dataset of fish maw.

[0030] Furthermore, the image size is unified: each fish maw surface reflection image in the multi-angle polarized reflection image group is resampled to a unified target size, and the pixel width and height of each fish maw surface reflection image are adjusted to the same size. This ensures that the subsequent fish maw surface reflection images have a consistent pixel correspondence in the spatial dimension, facilitating the analysis and comparison of reflection features at the same spatial location. Color space conversion: Each fish maw surface reflection image in the multi-angle polarized reflection image group is converted from RGB color space to Lab color space. By extracting the L channel, the image component reflecting the brightness information is obtained, thereby reducing the impact of color interference on the image brightness characteristics, providing a basis for subsequent brightness normalization and edge feature extraction; Brightness normalization: Perform linear normalization on the L channel image after color space conversion to map pixel brightness values ​​to a unified standardized range, such as [0, 1]. This reduces overall brightness differences caused by different polarization angles or lighting conditions and improves the comparability of brightness features. Image denoising: Bilateral filtering is applied to the brightness-normalized fish maw surface reflection image to perform image denoising. This method smoothes local brightness fluctuations caused by illumination disturbances or acquisition noise while preserving edge details, thereby improving the overall structural stability of the image and the accuracy of subsequent feature extraction. Gamma correction: Gamma correction is performed on the denoised fish maw surface reflection image. Nonlinear grayscale mapping is used to enhance the details of low-brightness areas while suppressing the overexposure of high-brightness areas. For example, the value γ=0.5 or γ=2.2 (where γ represents gamma) is used. This optimizes the image's brightness distribution structure, enhances the image's contrast and layering, and ultimately forms a fish maw multi-light source image dataset.

[0031] By calculating the reflection intensity difference of the corresponding pixels of each group of images with different polarization angles in the fish maw multi-light source image dataset and obtaining the global average reflection difference value, the polarization angle image with the largest average reflection difference value is selected as the significant polarization image, and then converted to the standard color space to extract the brightness component image; It should be noted that for each group of images with different polarization angles in the fish maw multi-light source image dataset, the same spatial position is registered at the pixel level, and the difference in reflection intensity of the pixels at the same position at different polarization angles is calculated. The expression is: ; Where, is the two-dimensional spatial coordinate of the pixel in the fish maw multi-light source image dataset The reflection intensity difference value, It is The two-dimensional spatial coordinates of pixels in the fish maw image at different polarization angles brightness, is the two-dimensional space coordinate of the pixel The average brightness in the image at all polarization angles, is the index variable of the fish maw image under polarization angle, is the number of maw images under polarization angle; The reflection intensity difference values ​​of all pixels in the entire set of images are summed and the number of pixels is normalized to obtain the global average reflection difference value of the group polarization angle image; The global average reflectance difference values ​​of all polarization angle image groups are compared in turn, and the polarization angle image with the largest average reflectance difference value is selected as the significant polarization image. The color space conversion operation is performed on the significant polarization image, and the significant polarization image is converted from RGB color space to Lab color space. The L channel image is extracted as the brightness component image.

[0032] Perform edge detection on the brightness component image to extract the brightness change mutation area, generate brightness edge information, and extract the main area mask of the fish maw through image segmentation algorithm; It should be noted that the Sobel operator is used to calculate the horizontal and vertical gradients of the brightness component image, and the edge area with drastic brightness changes is determined by calculating the gradient amplitude. The edge information of the brightness mutation area is represented in the form of a binary image to form a brightness edge information map; According to the spatial distribution of edge lines in the brightness edge information map, an image segmentation algorithm based on region growing or graph cutting method is used to segment the brightness component image, extract the connected areas corresponding to the main structural contour of the fish maw, and finally generate the main area mask of the fish maw.

[0033] The edge of the mask of the main area of ​​the fish maw is detected to obtain an ordered boundary point set, which is then smoothed to generate the edge contour of the fish maw.

[0034] It should be noted that an edge pixel sequence is extracted from the fish maw main region mask using an edge extraction method based on contour tracking, the edge of the fish maw main region mask is contour detected, and a complete and ordered boundary point set is generated according to the pixel connectivity sequence of the edge of the fish maw main region mask; The ordered boundary point set is constructed into a continuous curve according to the point set arrangement order. The curvature of the continuous curve is estimated and redundant points are identified. The boundary point set is smoothed by B-spline fitting or moving average filtering to eliminate high-frequency jagged deformation and local discrete mutations, and finally a fish maw edge contour with higher morphological continuity is generated.

[0035] The fish maw multi-light source image dataset was subjected to mask cropping and pixel normalization processing using the edge contour of the fish maw to form the image of the main area of ​​the fish maw.

[0036] It should be explained that, with the edge contour of the fish maw as the spatial boundary constraint, a pixel-level masking operation is performed on each image with different polarization angles in the fish maw multi-light source image dataset. The image area surrounded by the edge contour of the fish maw is taken as the valid area, and the pixels in the remaining areas are set to zero or shielded to complete the mask cropping of the polarization angle image. In the polarization angle image after mask cropping, the brightness values ​​and color channel values ​​of all non-zero pixels in the effective area are extracted, and the minimum-maximum linear normalization processing is performed on each channel to map the pixel values ​​to a standardized interval, such as [0,1]. This eliminates the grayscale dimension differences and dynamic range inconsistencies between multi-angle polarization reflection images, and finally forms an image of the main area of ​​the fish maw with a unified scale, unified spatial range and unified brightness standard.

[0037] S3. Perform polarization reflection feature analysis on the main area image of the fish maw, construct a surface direction-reflection relationship tensor and cluster it to generate anisotropic reflection texture feature map.

[0038] For each pixel point in the image of the main area of ​​the fish maw, a corresponding virtual pulse neuron is constructed, the reflection intensity difference is input into the corresponding virtual pulse neuron, and the membrane potential accumulation calculation is performed to obtain the membrane potential value.

[0039] It should be noted that in the image of the main area of ​​the fish maw, each pixel point is spatially mapped one-to-one with the corresponding position in the image of each polarization angle in the fish maw multi-light source image dataset, and the reflection intensity difference value of the pixel point at all polarization angles is collected; the average reflection intensity of each pixel point at all polarization angles is calculated, and the difference between the reflection intensity of each polarization angle and the average value is calculated one by one, a virtual pulse neuron structure is constructed for each pixel point, the initial membrane potential is set to the resting value, and each reflection intensity difference value is injected into the virtual pulse neuron structure as a pulse input sequence; after each pulse input, the membrane potential is updated according to the superposition result of the membrane potential at the previous moment and the current input, and an attenuation factor is applied to simulate the charge leakage effect; all reflection intensity difference values ​​are input in sequence according to the order of polarization angle change to complete the time integration process of the membrane potential, and finally the membrane potential value at the end of the integration is recorded.

[0040] The membrane potential value is compared with the pulse trigger threshold. When the membrane potential value exceeds the pulse trigger threshold, a pulse signal is triggered to generate a spatiotemporal pulse response matrix.

[0041] It should be explained that the membrane potential values ​​of all pixels are compared with the pulse trigger threshold one by one. When the membrane potential value of a certain pixel exceeds the pulse trigger threshold, it is considered that this pixel generates an impulse response in the corresponding polarization angle sequence and is recorded as a pulse signal occurrence; The impulse responses of all pixels under different polarization angle input sequences are encoded according to spatial position and time order, and a three-dimensional matrix structure is constructed, in which the two-dimensional plane corresponds to the pixel spatial distribution and the third dimension represents the time step under the polarization angle input sequence, finally generating a spatiotemporal impulse response matrix. It should also be noted that the pulse trigger threshold is set using the empirical method or quantile method by statistically analyzing the distribution of membrane potential values ​​of the reflection intensity differences of all pixels in the fish maw multi-light source image dataset at various polarization angles, combined with the range of membrane potential values ​​of pixel areas reflecting significant anisotropy characteristics in historical samples.

[0042] Taking the spatiotemporal impulse response matrix as input excitation, a surface direction-reflection relationship tensor field modulated by pulse intensity is constructed in a three-dimensional spatial coordinate system, and the steady-state tensor field is obtained by solving the nonlinear diffusion equation.

[0043] It should be explained that the two-dimensional pulse signal graph at each time step in the spatiotemporal pulse response matrix is ​​mapped to a three-dimensional spatial coordinate system according to the order in the polarization angle sequence, and a three-dimensional coordinate system is established with the spatial pixel position as the two-dimensional reference coordinate and the polarization angle input order as the time axis direction. The pulse signal intensity is used as the tensor component amplitude to construct the response change structure of each position in the main area of ​​the fish maw under polarized irradiation; The initial tensor distribution of the surface direction-reflection relationship is defined based on the response variation structure. Each non-zero pulse value in the spatiotemporal pulse response matrix is ​​used as a pulse excitation source, and the response amplitude is used as a reference for the degree of surface direction modulation. The tensor direction distribution and amplitude are modulated by the pulse intensity to preliminarily generate the surface direction-reflection relationship tensor field affected by the pulse modulation. Based on the initial state of the surface direction-reflection relationship tensor field, a nonlinear diffusion equation based on spatial gradient is established, and an iterative numerical method is used to perform diffusion calculations in the spatial domain. The diffusion rate is controlled by the tensor direction consistency and response intensity difference in the neighborhood, realizing the spatial continuity constraint and abnormal smoothness suppression of the surface direction-reflection relationship, and finally obtaining a stable and structurally consistent steady-state tensor field.

[0044] According to the local eigenvector distribution of the steady-state tensor field and the temporal correlation of the pulse signals of adjacent pixels, adaptive weight learning is performed to divide the phase-consistent texture area.

[0045] It should be noted that based on the tensor main direction eigenvector of each spatial position in the steady-state tensor field, the tensor main direction angle and the direction consistency index between adjacent pixels are calculated to reflect the change trend of the local texture direction. The expression is: ; Where, is the pixel position and adjacent pixel positions The angle between the main directions of the tensor, is the first in the steady-state tensor field The main direction unit vector of the pixel position, It is The main direction unit vectors of adjacent pixel positions, is the index of the adjacent pixel position, is the index of the pixel position; ; Where, is the pixel position and adjacent pixel positions Directional consistency index; Combined with the pulse trigger timing corresponding to each pixel in the spatiotemporal impulse response matrix, the impulse response time differences between adjacent pixels are counted. By calculating the degree of time delay consistency, the strength of the timing correlation between adjacent pixels is quantified. The expression is: ; Where, is the pixel position and adjacent pixel positions The temporal correlation strength of the impulse response of is the pixel position and adjacent pixel positions The difference in impulse response time, It is the Gaussian kernel width parameter that controls the influence range of time difference. For example, the value of ; Based on the tensor main direction consistency index and the impulse response temporal correlation index, a weighted distance measurement function containing dual coupling features is constructed. Each pixel neighborhood is weightedly aggregated, and the clustering judgment threshold is adaptively adjusted according to the distance measurement value to achieve the aggregation division of texture areas with similar directions and strong temporal linkage, and output phase-consistent texture areas.

[0046] It should also be noted that the clustering determination threshold is set by statistically analyzing the angle distribution and response intensity differences of local eigenvectors in the direction-reflection relationship tensor field of the fish maw surface, combined with empirical analysis.

[0047] The phase-consistent texture area is mapped to the two-dimensional image space according to the anisotropic response level to generate an anisotropic reflectance texture feature map.

[0048] It should be explained that the phase-consistent texture area is divided into multiple categories according to the anisotropic reflection response level, and different response intensity values ​​or codes are assigned to each category. The texture area corresponding to each category is projected onto the two-dimensional image plane through spatial mapping to form an anisotropic reflection texture feature map with hierarchical distribution and directional characteristics.

[0049] S4. Perform gradient pyramid fusion on the anisotropic reflectance texture feature map and the RGB image and extract the contour information to generate a fish maw boundary enhanced image.

[0050] Convert the RGB image to the LAB color space and extract the L channel as the brightness channel of the RGB image.

[0051] It should be explained that the color space conversion operation is performed on the RGB image, and the red, green, and blue channel values ​​of each pixel in the RGB image are mapped to the Lab color space according to the standard color space conversion formula. The color information of the RGB image is split into a brightness channel (L channel) and two color channels (a channel and b channel), where the L channel reflects the brightness information of the pixel. The L channel image is extracted from the converted Lab color space image as the brightness channel representing the brightness characteristics of the original RGB image.

[0052] The anisotropic reflectance texture feature map and the RGB image brightness channel are decomposed into Gaussian pyramids respectively, and downsampled by a fixed scale factor to generate a gradient pyramid hierarchical structure.

[0053] It should be noted that by applying a Gaussian smoothing filter to remove high-frequency details on the anisotropic reflection texture feature map and the RGB image luminance channel, a blurred version image is obtained. The anisotropic reflection texture feature map and the RGB image luminance channel are down-sampled according to a fixed scale factor, the spatial resolution of the anisotropic reflection texture feature map and the RGB image luminance channel is reduced in proportion, a multi-layer image sequence with gradually reduced resolution is generated, and the processing process is iterated. On each layer, Gaussian filtering is first performed, and then a lower resolution image is obtained by down-sampling, finally forming a gradient pyramid structure containing multiple scale levels. It should also be noted that the fixed scale factor is usually set according to image processing requirements and experience, and a constant value greater than 1 is selected as the proportion of image size reduction during each down-sampling.

[0054] On each gradient pyramid level structure, the Sobel operator is used to calculate the horizontal and vertical gradients of the anisotropic reflection texture feature map and the RGB image luminance channel, respectively. The gradient amplitudes are compared and the larger gradient amplitudes are retained to generate an optimized gradient amplitude map.

[0055] It should be noted that in each gradient pyramid level structure, the Sobel operator is applied to the anisotropic reflection texture feature map and the RGB image luminance channel respectively to calculate the horizontal and vertical gradients of the image. The specific operation includes convolution operation on the image pixel neighborhood, extraction of horizontal and vertical gradient values, reflection of local brightness change intensity and direction of the image, comparison of gradient amplitudes of the anisotropic reflection texture feature map and the RGB image luminance channel at corresponding positions, retention of larger gradient amplitudes to highlight more significant edges and texture features, and finally generation of an optimized gradient amplitude map.

[0056] Starting from the top layer of the gradient pyramid level structure, the optimized gradient amplitude map is up-sampled layer by layer and superimposed. The superimposed result is histogram equalized to generate a fusion base map.

[0057] It should be noted that starting from the top layer of the gradient pyramid level structure, the optimized gradient amplitude map is up-sampled layer by layer. The current layer image is interpolated and reconstructed according to the resolution of the lower layer image, so that the spatial size is consistent with that of the next layer image. After up-sampling is completed, the current layer image and the next layer image are superimposed point by point at corresponding positions to fuse gradient features at different scales. The up-sampling and superimposition operations are performed in turn until the lowest layer of the gradient pyramid level structure is reached, and the fusion of the optimized gradient amplitude maps of all scale levels is completed. The histogram equalization method is applied to the final superimposed image to redistribute the gray level distribution of the pixels, enhance the contrast and edge structure clarity of the image, and finally generate a fusion base map.

[0058] Based on the gradient amplitude distribution of the fused base image, contour points are extracted and connected to form closed contour segments.

[0059] It should be noted that the gradient amplitude threshold is used to extract pixels with significant gradient amplitude in the fusion base image as candidate contour points. These candidate contour points are concentrated in areas with sudden brightness changes and have high edge response values. Based on the pixel adjacency relationship, pixel sequences with spatial continuity and directional consistency are screened. In the extracted connected pixel sequences, the contour points are connected one by one according to the gradient direction and position order, and the broken points and isolated points are excluded to form closed contour segments. It should also be noted that the gradient amplitude distribution of all pixels in the statistical fusion base image is statistically analyzed, the statistical characteristics of the gradient amplitude, such as the mean or median, are calculated, and a certain proportion or weighted value of the statistical characteristics is set according to experience or algorithmic rules to obtain the gradient amplitude threshold.

[0060] Closed contour segments are superimposed on the RGB image to generate a fish maw boundary enhanced image.

[0061] It should be explained that the specific position and shape of the boundary line are determined based on the point set of the closed contour segment, all pixel points of the closed contour segment are traversed, and the color values ​​of these pixel points are adjusted to high-contrast colors, such as bright boundary colors, to highlight the contour structure. By superimposing the boundary line pixels in the RGB image, the visual performance of the fish maw boundary is enhanced, the boundary contour is made clearer and obvious, and an enhanced fish maw boundary image is generated.

[0062] S5. Input the fish maw boundary enhanced image into the contour detection network, perform automatic modeling, output a high-precision boundary heat map, and generate a high-precision boundary contour map.

[0063] The local contrast of the fish maw boundary enhanced image is normalized to generate a normalized boundary enhanced image, which is then input into the contour detection network for multi-scale boundary feature extraction, generating skip connection features and hole convolution features.

[0064] It should be noted that the fish maw boundary enhancement image is subjected to local contrast normalization processing. By calculating the mean and variance of the local area of ​​the image, the grayscale value of each pixel is normalized to improve the contrast performance of local details and form a normalized boundary enhancement image; The normalized boundary-enhanced image is input into the contour detection network. A multi-scale feature extraction method is used to capture boundary information at different scales. The skip connection mechanism is used to fuse shallow and deep features to enhance the expression of boundary details. At the same time, the dilated convolution operation is applied to expand the receptive field and extract richer contextual boundary features. Finally, skip connection features and dilated convolution features are generated. It should also be noted that the contour detection network is based on historical annotated samples and is trained using supervised learning. It extracts boundary features through a multi-layer convolution structure, combines jump connections with void convolution mechanisms to optimize feature expression capabilities, uses a loss function to calculate the error between the predicted boundary and the true boundary, and continuously adjusts the contour detection network weights through the back-propagation algorithm.

[0065] The skip connection features are fused with the dilated convolution features to generate enhanced boundary features, and channel weighted response processing is performed to generate the initial boundary heat map.

[0066] It should be explained that the jump connection features and the hole convolution features are spliced ​​along the channel dimension to achieve information fusion and complementarity, forming a multi-level feature expression, and applying channel weighted response processing to the fused feature map. By calculating the importance weight of each channel, the channel features are weighted and adjusted to improve the expression ability of key boundary information. The weighted feature map is activated to generate an initial boundary heat map reflecting the boundary significance.

[0067] The initial boundary heat map is processed by non-maximum suppression to generate a high-precision boundary heat map.

[0068] It should be explained that the initial boundary heat map is scanned pixel by pixel, the gradient amplitude is compared in the local neighborhood of each pixel, the pixel with the largest gradient amplitude is identified and retained, the pixel values ​​of non-maximum response are suppressed, the boundary width redundancy and weak response interference are eliminated, so as to highlight the accurate position and continuity of the boundary, and through non-maximum suppression processing, a high-precision boundary heat map with more accurate boundary position and clearer details is generated.

[0069] The local maximum points are extracted from the high-precision boundary heat map, and the high-precision boundary contour map is generated after verification of gradient direction connectivity.

[0070] It should be explained that local maximum points are extracted from the high-precision boundary heat map as potential boundary points. According to the gradient direction of each local maximum point, the connectivity of adjacent points is verified, the consistency and coherence of the gradient directions of adjacent points are judged, and the point set with continuous gradient direction characteristics is screened out. By connecting the connected local maximum points, a high-precision boundary contour map with complete structure and continuity is formed.

[0071] S6. Fuse the anisotropic reflective texture feature map with the high-precision boundary contour map to construct a fish maw appearance description vector. Use a multi-layer neural network model and combine it with historical evaluation samples to perform appearance grade classification and output the fish maw sample grading results.

[0072] The anisotropic reflective texture feature map and the high-precision boundary contour map are spatially aligned. The anisotropic reflective texture feature map is used as the first input channel, and the high-precision boundary contour map is used as the second input channel. The first and second input channels are spliced ​​along the channel dimension to generate a multi-channel fusion feature map. Global average pooling and maximum and minimum normalization are then performed to generate a fish maw appearance description vector.

[0073] It should be explained that the anisotropic reflective texture feature map and the high-precision boundary contour map are spatially aligned to ensure that the anisotropic reflective texture feature map and the high-precision boundary contour map accurately correspond in spatial coordinates. Through spatial alignment, each pixel position of the anisotropic reflective texture feature map and the high-precision boundary contour map is made to correspond one to one; The spatially aligned anisotropic reflectance texture feature map is used as the first input channel, and the high-precision boundary contour map is used as the second input channel. The two input channels are spliced ​​along the channel dimension to generate a multi-channel fusion feature. The multi-channel fusion feature map contains the joint expression of texture features and boundary information, enriching the feature representation capability of the image. Perform a global average pooling operation on the multi-channel fusion feature map to compress the multi-channel fusion feature map into a feature vector of fixed length. By averaging the pixel values ​​of all spatial positions, the global statistical information of the fusion feature map is extracted to obtain the feature vector after global average pooling. The feature vector after global average pooling is normalized to its maximum and minimum values, and mapped to a uniform range through linear transformation to improve the stability and contrast of feature expression. The result obtained after normalization is the fish maw appearance description vector, which is used to comprehensively characterize the appearance characteristics of fish maw.

[0074] The historical fish maw appearance description vectors are used as training data, and the back propagation algorithm is used to update the multi-layer neural network model parameters to obtain the trained multi-layer neural network model.

[0075] It should be explained that the historical fish maw appearance description vector is used as input data and passed to each layer of the multi-layer neural network model in sequence. After weighted summation and nonlinear activation function processing, the predicted output is generated, and the error between the predicted output and the actual label is calculated. The error signal is then passed back from the output layer to each hidden layer through the backpropagation algorithm, and the gradient information of each layer is calculated in sequence. According to the calculated gradient information, the gradient descent method is used to adjust the weight parameters and bias parameters of each layer of the multi-layer neural network model, gradually optimize the performance of the multi-layer neural network model, and repeat multiple rounds of iterative training until the loss function converges or the performance index is met, and the training is terminated. Finally, a trained multi-layer neural network model is obtained.

[0076] The fish maw appearance description vector is input into the trained multi-layer neural network model, and deep classification features are extracted through fully connected layer operations.

[0077] It should be explained that the fish maw appearance description vector is taken as input and passed to the fully connected layer of the trained multi-layer neural network model. After matrix multiplication and bias addition operations, the deep classification features in the vector are extracted. The fully connected layer performs a linear transformation on the input vector through weight parameters, gradually mining the abstract expression in the fish maw appearance description vector, and finally generating deep classification features with higher discriminative ability.

[0078] The deep classification features are sequentially activated with ReLU and normalized with Softmax to generate the level probability distribution.

[0079] It should be explained that the deep classification features are processed element by element through the ReLU activation function, all eigenvalues ​​less than zero are set to zero, and positive values ​​are retained, so as to improve the nonlinear expression ability of the deep classification features and prevent the gradient disappearance phenomenon. The feature vector activated by ReLU is input into the Softmax function, and the Softmax function calculates the exponential value of each element and normalizes it to obtain a level probability distribution in which the sum of the probability values ​​of all elements is one.

[0080] The first-level standard probability interval, the second-level standard probability interval and the third-level standard probability interval are extracted from the historical evaluation samples, and the grade probability distribution is graded and evaluated to obtain the grading results of fish maw samples.

[0081] It should be noted that the historical evaluation samples are derived from manual evaluation and quality inspection of a large number of fish maw samples, covering the appearance characteristics of fish maws of different grades and the corresponding probability distribution information; By statistically analyzing the probability distribution of each grade of fish maw in these historical evaluation samples, statistical methods were used to determine the first-level standard probability interval, the second-level standard probability interval, and the third-level standard probability interval, which correspond to the distribution ranges of different quality characteristics, such as complete boundary contour and uniform texture (the first-level probability interval has the highest requirements), slightly irregular boundary and locally uneven texture (the second-level probability interval has an appropriate relaxation), and obvious boundary defects and messy texture (the third-level probability interval has the widest range). By comparing the grade probability distribution of the sample with the above three probability intervals, when the grade probability distribution falls into the first-level standard probability interval, the sample is judged to be a first-level fish maw, indicating that the boundary contour is complete and the texture is uniform; when it falls into the second-level standard probability interval, it is judged to be a second-level fish maw, indicating that the boundary is slightly irregular and the texture is locally uneven; when it falls into the third-level standard probability interval, it is judged to be a third-level fish maw, indicating that the boundary has obvious defects and the texture is messy, and the fish maw sample grading results are obtained.

[0082] This embodiment also provides a computer device suitable for the visual analysis and screening method of fish maw appearance characteristics, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the visual analysis and screening method of fish maw appearance characteristics proposed in the above embodiment.

[0083] The computer device may be a terminal, comprising a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage media. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. Wireless communication may be achieved via Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. The display of the computer device may be a liquid crystal display or an electronic ink display. The input device may be a touchscreen overlay on the display, buttons, a trackball, or a touchpad on the computer device housing, or an external keyboard, touchpad, or mouse.

[0084] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the visual analysis and screening method for the appearance characteristics of fish maw as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk.

[0085] To sum up, the present application solves the problems of insufficient micro-texture analysis and poor robustness of contour detection in traditional fish glue sorting technology by polarization reflection feature analysis (constructing pulse neuron model and generating anisotropic reflection texture feature map by tensor field clustering) and gradient pyramid fusion (optimizing RGB and polarization features in multiple scales to enhance the contour), and realizes high-precision surface defect recognition, multi-modal feature collaborative optimization and automatic intelligent grading, significantly improving the accuracy and efficiency of fish glue quality detection.

[0086] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A method for visual analysis and screening of fish maw appearance characteristics, characterized by: include, Collect RGB images and multi-angle polarized reflectance images of fish maw; The multi-angle polarized reflectance image group was preprocessed to form a fish maw multi-light source image dataset. The image segmentation algorithm was used to extract the fish maw main area mask, and then smoothed to obtain the fish maw main area image. Polarized reflectance characteristics of the fish maw main area image were analyzed to construct the surface direction-reflectance relationship tensor and cluster it to generate anisotropic reflectance texture feature map. The anisotropic reflectance texture feature map is fused with the RGB image using a gradient pyramid method and the contour information is extracted to generate a fish maw boundary enhanced image. The fish maw boundary enhancement image is input into the contour detection network, and automatic modeling is performed to output a high-precision boundary heat map and generate a high-precision boundary contour map; The anisotropic reflective texture feature map and the high-precision boundary contour map are fused to construct the fish maw appearance description vector. The appearance grade classification is performed through a multi-layer neural network model combined with historical evaluation samples, and the fish maw sample grading results are output.

2. The method for visually analyzing and screening the appearance characteristics of fish maw according to claim 1, wherein: The multi-angle polarized reflectance image group was subjected to image size unification, color space conversion, brightness normalization, image denoising and gamma correction in sequence to form a multi-light source image dataset of fish maw.

3. The method for visually analyzing and screening the appearance characteristics of fish maw according to claim 1, wherein: The image segmentation algorithm is used to extract the mask of the main area of ​​the fish maw, and smoothing is performed to obtain the image of the main area of ​​the fish maw. The specific steps are as follows: By calculating the reflection intensity difference of the corresponding pixels of each group of images with different polarization angles in the fish maw multi-light source image dataset and obtaining the global average reflection difference value, the polarization angle image with the largest average reflection difference value is selected as the significant polarization image, and then converted to the standard color space to extract the brightness component image; Perform edge detection on the brightness component image to extract the brightness change mutation area, generate brightness edge information, and extract the main area mask of the fish maw through image segmentation algorithm; Perform contour detection on the edge of the mask of the main area of ​​the fish maw to obtain an ordered boundary point set, and perform smoothing to generate the edge contour of the fish maw; The fish maw multi-light source image dataset was subjected to mask cropping and pixel normalization processing using the edge contour of the fish maw to form the image of the fish maw main area.

4. The method for visually analyzing and screening the appearance characteristics of fish maw according to claim 3, wherein: The polarized reflection feature analysis is performed on the image of the main area of ​​the fish maw, the surface direction-reflection relationship tensor is constructed, and the anisotropic reflection texture feature map is generated by clustering. The specific steps are as follows: For each pixel in the image of the main area of ​​the fish maw, a corresponding virtual pulse neuron is constructed, the reflection intensity difference is input into the corresponding virtual pulse neuron, and the membrane potential accumulation calculation is performed to obtain the membrane potential value; Compare the membrane potential value with the pulse trigger threshold. When the membrane potential value exceeds the pulse trigger threshold, a pulse signal is triggered to generate a spatiotemporal pulse response matrix. Taking the spatiotemporal impulse response matrix as input excitation, a surface direction-reflection relationship tensor field modulated by pulse intensity is constructed in a three-dimensional space coordinate system, and the steady-state tensor field is obtained by solving the nonlinear diffusion equation. According to the local eigenvector distribution of the steady-state tensor field, adaptive weight learning is performed in combination with the temporal correlation of the pulse signals of adjacent pixels to divide the phase-consistent texture area. The phase-consistent texture area is mapped to the two-dimensional image space according to the anisotropic response level to generate an anisotropic reflectance texture feature map.

5. The method for visually analyzing and screening fish maw appearance characteristics according to claim 1, wherein: The anisotropic reflectance texture feature map is fused with the RGB image by gradient pyramid and contour information is extracted to generate a fish maw boundary enhanced image. The specific steps are as follows: Convert the RGB image to LAB color space and extract the L channel as the brightness channel of the RGB image; The anisotropic reflectance texture feature map and the RGB image brightness channel are decomposed into Gaussian pyramids respectively, and downsampled by a fixed scale factor to generate a gradient pyramid hierarchical structure; At each level of the gradient pyramid structure, the Sobel operator is used to calculate the horizontal and vertical gradients of the anisotropic reflectance texture feature map and the luminance channel of the RGB image, respectively. The gradient amplitudes are compared and the larger gradient amplitude is retained to generate an optimized gradient amplitude map. Starting from the top layer of the gradient pyramid hierarchy, the optimized gradient magnitude map is upsampled layer by layer and superimposed. The superposition results are histogram-equalized to generate a fused base map. Based on the gradient amplitude distribution of the fused base image, contour points are extracted and connected to form closed contour segments; Closed contour segments are superimposed on the RGB image to generate a fish maw boundary enhanced image.

6. The method for visually analyzing and screening the appearance characteristics of fish maw according to claim 5, characterized in that: The fish maw boundary enhancement image is input into the contour detection network, and automatic modeling is performed to output a high-precision boundary heat map and generate a high-precision boundary contour map. The specific steps are as follows: The fish maw boundary enhancement image is subjected to local contrast normalization to generate a normalized boundary enhancement image, which is then input into the contour detection network for multi-scale boundary feature extraction, generating skip connection features and dilated convolution features. The skip connection features are fused with the dilated convolution features to generate enhanced boundary features, and channel weighted response processing is performed to generate the initial boundary heat map; Perform non-maximum suppression on the initial boundary heat map to generate a high-precision boundary heat map; The local maximum points are extracted from the high-precision boundary heat map, and the high-precision boundary contour map is generated after verification of gradient direction connectivity.

7. The method for visual analysis and screening of fish maw appearance characteristics according to claim 1, characterized in that :The anisotropic reflective texture feature map and the high-precision boundary contour map are spatially aligned, the anisotropic reflective texture feature map is used as the first input channel, the high-precision boundary contour map is used as the second input channel, the first input channel and the second input channel are spliced ​​along the channel dimension to generate a multi-channel fusion feature map, and global average pooling operation and maximum and minimum normalization processing are performed to generate the fish maw appearance description vector.

8. The method for visually analyzing and screening fish maw appearance characteristics according to claim 1, wherein: The multi-layer neural network model is used to combine historical evaluation samples to perform appearance grade classification and output the grading results of fish maw samples. The specific steps are as follows: The historical fish maw appearance description vector is used as training data, and the multi-layer neural network model parameters are updated through the back propagation algorithm to obtain the trained multi-layer neural network model; The fish maw appearance description vector is input into the trained multi-layer neural network model, and deep classification features are extracted through fully connected layer operations; The deep classification features are sequentially activated by ReLU and normalized by Softmax to generate a level probability distribution; The first-level standard probability interval, the second-level standard probability interval and the third-level standard probability interval are extracted from the historical evaluation samples, and the grade probability distribution is graded and evaluated to obtain the grading results of fish maw samples.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the visual analysis and screening method of fish maw appearance characteristics described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the visual analysis and screening method of the appearance characteristics of fish maw described in any one of claims 1 to 8 are implemented.

Images