Fish fillet freshness detection method and device fusing vision and electrical impedance

Abstract

The invention provides a fillet freshness detection method and device fusing vision and electrical impedance, and relates to the technical field of prefabricated aquatic product nondestructive testing, fillet fixation and temperature data detection are realized through a fixed detection device, fillet humidity detection is realized through a capacitance hygrometer, and the detection accuracy is improved. The electrical impedance detection device is used for measuring the electrical impedance of fish fillets, the lighting device is used for providing a stable light source to enable the image shooting device to complete image acquisition, the control unit is used for controlling the detection device to work through a preset instruction, and the acquired image and electrical impedance information are sent to the computer through the connecting line. After data acquisition and screening are completed, image features of fillets are extracted, then extracted electrical impedance features are added, the features are fused together, and finally, a prediction model is used for predicting the freshness of the fillets. According to the invention, high-efficiency and lossless fillet freshness detection can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of non-destructive testing of prefabricated aquatic products, and particularly to a method and device for detecting the freshness of fish fillets by integrating vision and electrical impedance. Background Art

[0002] With the development of the global economy and the improvement of living standards, consumers' demand for fast and healthy food is increasing continuously. As a kind of food material with rich nutrition, convenient to eat and unique flavor, fish fillets have gradually become an important raw material in the prefabricated food market. However, in order to ensure the freshness of prefabricated food raw materials, the demand for detecting the freshness of fish fillets has become more urgent. Traditional methods for detecting the freshness and flavor of aquatic products usually take a long time and are complex to operate, making it difficult to meet the requirements of fast and efficient detection in prefabricated food processing. Therefore, developing a precise, fast and non-destructive freshness detection technology, combining computer simulation and machine learning to obtain raw material quality information, is of great significance for improving the quality and safety of prefabricated food products. Therefore, it is very necessary to design a method and device for detecting the freshness of fish fillets by integrating vision and electrical impedance. Summary of the Invention

[0003] In order to overcome the deficiencies of the prior art, the purpose of the present invention is to provide a method and device for detecting the freshness of fish fillets by integrating vision and electrical impedance.

[0004] To achieve the above purpose, the present invention provides the following solutions:

[0005] The present invention provides a method for detecting the freshness of fish fillets by integrating vision and electrical impedance, including:

[0006] S1: Collect the image, temperature, and humidity information of the fish fillet to be tested through a fish fillet freshness detection device, obtain the electrical impedance data of the fish fillet to be tested by using the four-electrode method, and measure the content of total volatile basic nitrogen in the fish fillet by using a Kjeldahl nitrogen analyzer to calibrate its freshness;

[0007] S2: Screen the quality of the image data of the fish fillet, delete the data with inconsistent quality, and prompt for re-collection;

[0008] S3: Calculate the size of the fish fillet based on the obtained image data of the fish fillet by combining the vision measurement method and the fine needle carving substrate measurement method, and extract its color, gloss, and texture features;

[0009] S4: Check the change rate of the detected temperature and humidity information of the fish fillet, compare the measured electrical impedance amplitude and phase of the fish fillet by using the four-electrode method with the preset intervals respectively, and screen the data quality;

[0010] S5: Calculate the transverse and longitudinal resistivity of the fish fillet under standard conditions based on the resistance, size, temperature, and humidity information of the fish fillet. Use the size, temperature, and humidity information of the fish fillet to correct the collected transverse and longitudinal impedance information, and extract the corresponding impedance characteristics;

[0011] S6: Standardize and splice and fuse the obtained color characteristics, gloss characteristics, texture characteristics, and impedance characteristics of the fish fillet. Determine the freshness grading standard according to the use of the fish fillet, construct a freshness prediction model for the fish fillet, use the color characteristics, gloss characteristics, texture characteristics, and impedance characteristics of the fish fillet as input values, and use the freshness grading of the fish fillet as the output to predict the freshness of the fish fillet;

[0012] S7: Add an artificial comparison program, calculate the error rate of the prediction model, and confirm the accuracy of the prediction result.

[0013] Preferably, in S2, screen the quality of the image data of the fish fillet, delete the data with inconsistent quality and prompt for re-acquisition. Specifically:

[0014] Convert the original RGB image I of the fish fillet into a grayscale image I gray , which is:

[0015]

[0016] In the formula, the value of ω is 0.2989, The value of is 0.5870, and the value of τ is 0.1140;

[0017] Judge the noise pollution, local and overall quality in the image data, perform block processing on the input image, the image size is M T ×N T , and divide it into p T ×q T small pieces, and each small piece is defined as S ij , where i and j represent the indexes of each small piece, which is:

[0018]

[0019] In the formula, H ij is the local complexity of each small piece S ij , P k represents the probability that the gray value k appears in S ij , σ H represents the standard deviation of the local complexity, represents the average value of all local complexities;

[0020] Screen the clarity of the image data, map the image to the frequency domain, and judge the concentration of high-frequency energy, which is:

[0021]

[0022]

[0023] Wherein, F(u, v) is the complex value in the output frequency domain, u represents the frequency in the horizontal direction, v represents the frequency in the vertical direction, j is the imaginary unit, and S f represents the concentration of high-frequency energy, and H is the index set of the high-frequency region |F(u, v)| is the amplitude of (u, v) in the frequency domain;

[0024] Establish a scoring mechanism for the quality of image data, standardize the scores, and screen the image data. When the final score is greater than 70, it indicates that the quality of the image data meets the standard, and the next step is carried out. Otherwise, it is prompted that the quality does not meet the standard, and re-acquisition is carried out. The scoring mechanism is specifically as follows:

[0025] Q = μ1·exp(-σH) + μ2·S f ;

[0026]

[0027] Wherein, Q is the score calculated by the image data quality scoring mechanism, and σ H represents the standard deviation of local complexity, μ1 and μ2 represent weight coefficients, which are determined through experiments, and S f represents the concentration of high-frequency energy, and Q n represents the standardized score, and Q min is the minimum value of the scores in the experiment, and Q max is the maximum value of the scores in the experiment.

[0028] Preferably, in S3, based on the obtained image data of the fish fillet, combine the visual measurement method and the fine needle carving substrate measurement method to calculate the size of the fish fillet, and extract its color, gloss and texture features, which specifically include the following steps:

[0029] Perform the calculation of the fish fillet size. First, identify the image edge, and separate the outermost contour image of the fish fillet from it, and find the smallest rectangle that encloses the outermost contour of the fish fillet This rectangle has a pixel size of a×b;

[0030] Taking the actual distance L2 between two adjacent needles in the fine needle carving substrate, whose pixel size is c, as the reference size, calculate the size 1 of the fish fillet as:

[0031]

[0032] Wherein, A is the length of the size 1 of the fish fillet, and B is the width of the size 1 of the fish fillet;

[0033] In the outermost contour image, find the number of needles m and n blocked by the longest length and width diameters of the fish fillet. Among them, calculate the distance D(x, y) from any point (x, y) in the input fish fillet image to the nearest point on the fish fillet contour, which is:

[0034]

[0035] In the formula, D(x, y) is the distance from any point (x, y) in the image to the nearest point on the fish fillet contour, (x * , y * ) ∈ fish is the set of all points on the fish fillet edge contour, which is used to calculate the shortest straight-line distance between two points;

[0036] Use U(x, y) to map D(x, y) of the fish fillet into a periodic function to determine whether the point (x, y) is near the needle, which is:

[0037]

[0038] In the formula, U(x, y) describes the feature intensity of the point (x, y) under the needle carving substrate, and L2 represents the distance between adjacent needles;

[0039] For the position (x i , y j ) of each needle, judge the possibility of it being blocked by the fish fillet, which is:

[0040]

[0041] In the formula, r is the radius of the needle, and P cover (x i , y j ) is the possibility of the needle being blocked by the fish fillet;

[0042] Calculate the total number N of needles blocked by the fish fillet cover which is:

[0043]

[0044] In the formula, M and N are the numbers of needles in the length and width directions on the needle carving substrate, τ * is the threshold of the blocking probability, 1[P cover (x i , y j ) > τ * means that if the condition P cover (x i , y j ) > τ * is satisfied, its value is 1, otherwise it is 0;

[0045] Calculate the number of needles blocked m and n along the length and width directions of the fish fillet respectively as follows:

[0046]

[0047] In the formula, projection(x i ) and projection(y i ) map the positions of the needles in the length and width directions. projection(x i ) = x i , projection(y i ) = y i . projection(x i ) ∩ U and projection(y i ) ∩ U represent the intersections of U with the projections projection(x i ) and projection(y i ) of the needles in the length and width directions;

[0048] Perform accuracy verification based on the total number of needles blocked by the fish fillet and the number of needles blocked in the length and width directions as follows:

[0049] N cover = m × n;

[0050] Furthermore, use the distance L2 between adjacent needles to find the length S and width P of the fish fillet size 2, and combine the two to find the final length X and width Y as follows:

[0051]

[0052] In the formula, S is the length of the fish fillet size 2 obtained by using the fine needle carving substrate measurement method, and P is the width of the fish fillet size 2 obtained by using the fine needle carving substrate measurement method;

[0053] Extract the color features of the fish fillet, use the original RGB color space of the image, and then calculate the average value of its three color channels as the color feature as follows:

[0054]

[0055] In the formula, a × b is the pixel size of the smallest rectangle enclosing the outermost contour of the fish fillet. P R , P G and P B represent the average values of the red, green, and blue channels respectively, and I i,j represents the pixel at the i-th row and j-th column of the image I;

[0056] Extract the gloss feature of the fish fillet, where the gloss feature is represented by the average brightness of the image:

[0057]

[0058] In the formula, G p represents the average brightness of the image, and a·b is the smallest rectangle surrounding the outermost contour of the fish fillet. The pixel size is Represents the pixel brightness of the i-th row and j-th column of the image;

[0059] Extract the texture features of the fish fillet. The texture features are obtained through contrast, entropy, and homogeneity, which are:

[0060]

[0061] D=∑ i,j (ij) 2 P(i,j);

[0062] S=-∑ i,j P(i,j)log(P(i,j));

[0063]

[0064] Where P(i,j) is the gray level co-occurrence matrix, I gray is a grayscale image, i and j are grayscale levels, Δx, Δy are relative displacements, D is contrast, S is entropy, and T is homogeneity.

[0065] Preferably, in S4, the change rate of the detected temperature and humidity information of the fish fillet is checked, and the impedance amplitude and phase of the fish fillet measured by the four-electrode method are compared with the preset intervals to screen the data quality, specifically:

[0066] The temperature and humidity information of the fish fillet detected is adjusted by dynamic parameters. Starting from the fourth set of data, the change range of the first three sets of data is compared, the parameters are dynamically determined, the mean of the first three sets of data is obtained, the threshold change range is established, and the current measured data is compared with the threshold. If the current measured data is within the range, it means that the quality of the temperature and humidity data meets the standard and the next step is performed. Otherwise, it prompts that the quality does not meet the standard and re-collection is performed.

[0067]

[0068] Where, T n-1 , T n-2 , T n-3 is the current temperature measurement value T n The previous three sets of temperature data, is the current temperature measurement value Tn The average value of the previous three groups of temperature data, k T is the parameter of temperature change, y T is the temperature threshold range, S n-1 , S n-2 , S n-3 is the current humidity measurement value S n The previous three groups of humidity data is the current humidity measurement value S n The average value of the previous three groups of humidity data, k S is the parameter of humidity change, y S is the humidity threshold range;

[0069] Compare the impedance amplitude and phase of the fish fillet measured by the four-electrode method with the preset ranges respectively. Among them, the preset amplitude range is [α Ω , β Ω , and the preset phase range is [α°, β°]. If the currently measured data is within the range, it means that the quality of the impedance data meets the standard, and the next step is carried out. Otherwise, it is prompted that the quality does not meet the standard, and re-acquisition is carried out.

[0070] Preferably, in S5, calculate the transverse and longitudinal resistivity of the fish fillet in the standard environment according to the resistance, size, temperature and humidity information of the fish fillet, use the size, temperature and humidity information of the fish fillet to correct the collected transverse and longitudinal impedance information, and extract the corresponding impedance characteristics. Specifically:

[0071] At different frequencies, measure the impedance amplitude and phase of the fish fillet by the four-electrode method, and calculate the rectangular coordinate form of the fish fillet impedance, which is:

[0072]

[0073] Z 横 / 纵 =R f +jx f ;

[0074] In the formula, Z 横 / 纵 is the impedance measured under the transverse or longitudinal arrangement of the electrodes, |Z 横 / 纵 | is the impedance amplitude measured under the transverse or longitudinal arrangement of the electrodes, is the phase difference between the voltage and the current, R f横 / 纵 is the real part in the rectangular coordinate form of the fish fillet impedance, x f横 / 纵 is the imaginary part in the rectangular coordinate form of the fish fillet impedance;

[0075] Calculate the transverse and longitudinal resistivity of the fish fillet in the standard environment according to the resistance, size, temperature and humidity information of the fish fillet as:

[0076]

[0077] Where ρ H / ρ Z is the transverse / longitudinal resistivity of the fish fillet under standard conditions of T0℃ and S0%, R f横 / R f横 is the transverse / longitudinal resistance of the fish fillet under the current detection environment of T℃ and S%, X is the length of the fish fillet, Y is the width of the fish fillet, H is the thickness of the fish fillet, d VV h is the fixed distance between the two voltage electrodes when the electrodes are arranged transversely, d VV z is the fixed distance between the two voltage electrodes when the electrodes are arranged longitudinally, α is the resistance temperature coefficient, β is the resistance humidity coefficient, γ is the non-linear coefficient of humidity and temperature, T0 is the standard temperature, S0 is the standard humidity, T is the temperature during detection, and S is the humidity during detection;

[0078] According to the temperature, humidity, and size information of the fish fillet, the impedance Z 横 、Z 纵 is corrected, and the corrected impedance is expressed as:

[0079]

[0080] Where α is the resistance temperature coefficient; β is the resistance humidity coefficient, γ is the non-linear coefficient of humidity and temperature, T0 is the standard temperature, S0 is the standard humidity, T is the temperature during detection, S is the humidity during detection, X is the length of the fish fillet, Y is the width of the fish fillet, X 标 is the reference standard fish fillet length, Y 标 is the reference standard fish fillet width;

[0081] Feature extraction is performed on the impedance of the fish fillet, and the extracted features include the relative resistance peak ratio K and the impedance modulus change rate Δ|Z f |. The relative resistance peak ratio can analyze the frequency-dependent characteristics in the impedance spectrum and the relative relationship between capacitance and resistance. The impedance modulus change rate represents the dynamic characteristics of the conductivity and capacitance of the fish fillet tissue changing with frequency, and is:

[0082]

[0083] Where R flow represents the low-frequency resistance measured for the fish fillet, R fhigh represents the high-frequency resistance measured for the fish fillet, x max represents the reactance peak value in the graph plotted with the real part of the impedance as the horizontal axis and the imaginary part as the vertical axis, R max represents the maximum resistance value in the graph plotted with the real part of the impedance as the horizontal axis and the imaginary part as the vertical axis, R fis the real part in the impedance, x f is the imaginary part in the impedance, f a 、f b are two adjacent frequency points;

[0084] Convert the obtained impedance features into feature vectors so that they can be fused with visual features in a high-dimensional space, which is:

[0085] K = ReLU(W1·K + b1);

[0086] Δ|Z f | = ReLU(W2·Δ|Z f | + b1);

[0087] In the formula, K, Δ|Z f | are the impedance features after being converted into feature vectors, ReLU is the activation function, W is the corresponding weight matrix, K, Δ|Z f | are the impedances before conversion, and b is the bias vector.

[0088] Preferably, in S6, standardize and splice and fuse the obtained fish fillet color features, gloss features, texture features and impedance features, determine the freshness grading standard according to the use of the fish fillets, construct a fish fillet freshness prediction model, use the fish fillet color features, gloss features, texture features and impedance features as input values, and use the fish fillet freshness grading as the output to predict the freshness of the fish fillets, specifically:

[0089] Standardize and perform multi-modal feature fusion on the obtained fish fillet color features, gloss features, texture features and impedance features, and establish a fish fillet freshness prediction model. Among them, the input layer of the prediction model is to receive the fused feature vector F = [P R ,P G ,P B ,G p ,D,S,T,ρ H ,ρ Z ,K,Δ|Z f |]. First, perform normalization processing on the fused feature vector F to obtain F°, and each neuron in the input layer is fully connected to each neuron in the hidden layer 1 to form a weight matrix W o1 , the number of neurons in the hidden layer 1 is δ1, the dropout rate of the Dropout layer is 50%, the activation function is ReLU, and the input feature F° is linearly combined with the weight matrix of the hidden layer 1 through the fully connected layer and added with the bias vector b o1 , that is, Z o1 = W o1 ·F° + b o1 , and further processed by the activation function to obtain H o1 = ReLU(Zo1 ) and output to the Dropout layer. The Dropout layer randomly discards the outputs of some neurons, and the outputs of the remaining neurons are H o1 * Pass to the second hidden layer. The number of neurons in the second hidden layer is The second hidden layer receives the output H from the first hidden layer o1 * , and perform a linear transformation through the fully connected layer to obtain Z o2 = W o2 ·H o1 * + b o2 . After the fully connected layer, the second hidden layer introduces a batch normalization layer to standardize the output after the linear transformation to obtain BN(Z o2 ). The output after batch normalization passes through the ReLU activation function to introduce non - linear characteristics to obtain H o2 = ReLU(BN(Z o2 )). The output layer receives the output H from the second hidden layer o2 . The number of neurons in the output layer varies according to the number of levels of the required freshness, so as to predict the freshness of the fish fillets. After passing through the prediction model, the softmax activation function is used to classify the fish fillets. The formula is:

[0090]

[0091] In the formula, is a four - dimensional vector representing the probabilities of four classifications.

[0092] Preferably, in S7, an artificial comparison program is added to calculate the error rate of the prediction model and confirm the accuracy of the prediction result. Specifically:

[0093] Randomly select a part of the fish fillet samples X f , and manually use standard detection tools to verify the freshness of the fish fillets, confirm the accuracy of the prediction result, compare the predicted output of the model with the actual detection result, and count the number of prediction errors X c . If the error rate is lower than the threshold θ c , it is considered that the predicted output is accurate and continue the detection. If the error rate is relatively high, prompt for system correction. Among them, the formula for the error rate is:

[0094]

[0095] In the formula, μ d is the error rate of the predicted output in the fish fillet samples.

[0096] The present invention also provides a freshness detection device for fish fillets that integrates vision and electrical impedance, comprising: a mounting base, an industrial camera, a top light source board, a support column, electrode patches, a fine needle carving substrate, a control unit, an electrical impedance detector, a reduction device, an illumination device, a humidity monitoring device, and an infrared thermometer. The reduction device, the support column, and the control unit are arranged on the mounting base. The fine needle carving substrate is arranged on the mounting base through a vertical rod, and the illumination device is arranged on the vertical rod. The fine needle carving substrate is located above the reduction device. The top light source board is arranged corresponding to the fine needle carving substrate on the front side of the support column, and the industrial camera is arranged at the bottom of the top light source board. The reduction device is used to restore the fine needle carving substrate. The electrical impedance detector is also arranged on the mounting base. The electrical impedance detector is connected with 4 electrode patches for collecting the electrical impedance of the fish fillet. The humidity monitoring device is used to collect the humidity information of the fish fillet, and the infrared thermometer is used to collect the temperature information of the fish fillet.

[0097] Preferably, the reduction device includes a fixed frame, a stepping motor, a stepping motor fixing frame, a coupling, a fixed end, a ball screw, a support column, a slide table, and a flat plate. The fixed frame is arranged on the mounting base, the stepping motor fixing frame is arranged on the fixed frame, the stepping motor is arranged on the stepping motor fixing frame, the fixed end is arranged in the middle of the fixed frame, the support columns are arranged on both sides of the top of the fixed end, the slide table is slidably connected to the support columns, both sides of the slide table are connected to the flat plate through connecting rods, the coupling passes through the fixed end to connect the ball screw, and the ball screw is slidably connected to the slide table for driving the slide table to slide up and down along the support columns. The flat plate is located directly below the fine needle carving substrate.

[0098] Preferably, the humidity monitoring device includes a capacitive humidity board, an LED display screen, a wire, and a detection patch. The LED display screen is arranged on the capacitive humidity board, and the capacitive humidity board is connected to the detection patch through a wire for detecting the humidity of the fish fillet through the detection patch.

[0099] According to the specific embodiments provided by the present invention, the following technical effects are disclosed by the present invention:

[0100] The present invention provides a method and device for detecting the freshness of fish fillets by integrating vision and impedance. The fish fillets are fixed and temperature data is detected through a fixed detection device. The humidity of the fish fillets is detected through a capacitance hygrometer. The impedance of the fish fillets is measured through an impedance detection device. A stable light source is provided by a lighting device to enable an image capture device to complete image acquisition. A control unit controls the detection device to work through preset instructions, and sends the collected images and impedance information to a computer through a connection line. After data collection and screening, the image features of the fish fillets are extracted, and then the extracted impedance features are added, and then these features are fused together. Finally, a prediction model is used to predict the freshness of the fish fillets. The present invention can achieve high-efficiency and non-destructive detection of the freshness of fish fillets. Description of the Drawings

[0101] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0102] Figure 1 Schematic flow chart of the method for detecting the freshness of fish fillets by integrating vision and impedance provided by the embodiment of the present invention;

[0103] Figure 2 Timing diagram of the non-destructive detection method for the freshness of fish fillets provided by the embodiment of the present invention;

[0104] Figure 3 Schematic diagram of the horizontal layout of the electrode patches of the non-destructive detection method for the freshness of fish fillets provided by the embodiment of the present invention;

[0105] Figure 4 Schematic diagram of the vertical layout of the electrode patches of the non-destructive detection method for the freshness of fish fillets provided by the embodiment of the present invention;

[0106] Figure 5 Bar chart of the detection results of the non-destructive detection method for the freshness of fish fillets provided by the embodiment of the present invention;

[0107] Figure 6 Structural diagram of the device for detecting the freshness of fish fillets by integrating vision and impedance provided by the embodiment of the present invention;

[0108] Figure 7 Partial enlarged view of the reduction device;

[0109] Figure 8 Structural diagram of the capacitance hygrometer.

[0110] Reference numerals: 1, industrial camera; 2, top light source board; 3, support column; 4, electrode patch; 5, fine needle engraving substrate; 6, control unit; 7, impedance detector; 8, reduction device; 801, fixed frame; 802, stepper motor; 803, stepper motor fixing bracket; 804, coupling; 805, fixed end; 806, ball screw; 807, support column; 808, slide table; 809, flat plate; 9, lighting device; 10, capacitance humidity board; 11, LED display screen; 12, wire; 13, detection patch; 14, mounting base. Detailed implementation manners

[0111] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0112] The purpose of the present invention is to provide a method and device for detecting the freshness of fish fillets by integrating vision and impedance, which can not only meet the high-quality requirements of fish fillets in the production of prefabricated dishes, but also significantly improve the efficiency and accuracy of detecting the freshness of aquatic products, providing an innovative and practical solution for the quality monitoring of the prefabricated dish industry.

[0113] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0114] Figure 6 This is the schematic structural diagram of the device provided by the embodiment of the present invention, as Figure 6As shown in the figure, the present invention provides a fish fillet freshness detection device integrating vision and electrical impedance, comprising: a mounting base 14, an industrial camera 1, a top light source board 2, a support column 3, electrode patches 4, a fine needle carving substrate 5, a control unit 6, an electrical impedance detector 7, a reduction device 8, an illumination device 9, a humidity monitoring device and an infrared thermometer. The reduction device 8, the support column 3 and the control unit 6 are arranged on the mounting base 14. The fine needle carving substrate 5 is arranged on the mounting base 14 through a vertical rod, and the illumination device 9 is arranged on the vertical rod. The fine needle carving substrate 5 is located above the reduction device 8. The top light source board 2 is arranged corresponding to the fine needle carving substrate 5 on the front side of the support column 3, and the industrial camera 1 is arranged at the bottom of the top light source board 2. The reduction device 8 is used to restore the fine needle carving substrate 5. The electrical impedance detector 7 is also arranged on the mounting base 14. The electrical impedance detector 7 is connected with 4 electrode patches 4 for collecting the electrical impedance of the fish fillet. The humidity monitoring device is used to collect the humidity information of the fish fillet, and the infrared thermometer is used to collect the temperature information of the fish fillet.

[0115] As Figure 7 As shown in the figure, the reduction device comprises a fixed frame 801, a stepping motor 802, a stepping motor fixing bracket 803, a coupling 804, a fixed end 805, a ball screw 806, a support column 807, a sliding table 808 and a flat plate 809. The fixed frame 801 is arranged on the mounting base 14. The stepping motor fixing bracket 803 is arranged on the fixed frame 801, and the stepping motor 802 is arranged on the stepping motor fixing bracket 803. The fixed end 805 is arranged in the middle of the fixed frame 801. The support columns 807 are arranged on both sides of the top of the fixed end 805. The sliding table 808 is slidably connected to the support columns 807. Both sides of the sliding table 808 are connected to the flat plate 809 through connecting rods. The coupling 804 passes through the fixed end to connect the ball screw 806. The ball screw 806 is slidably connected to the sliding table 808 for driving the sliding table 808 to slide up and down along the support column 807. The flat plate 809 is located directly below the fine needle carving substrate 5;

[0116] The substrate of the fine needle carving substrate is made of 316 stainless steel. The diameter of the needle is 1.2 mm and the length is 25 mm. Rubber coatings are added at both ends of the needle, and the interval between adjacent needles is 4 mm;

[0117] The impedance detector uses a precision LCR digital bridge of the TH2829X series automatic transformer test system. The TH2829X series equipment integrates a high-precision LCR digital bridge, which can accurately measure the parameters of components such as inductance, capacitance, and resistance. The measurement accuracy can reach 0.01%, and it supports a wide measurement frequency range from 20 Hz to 1 MHz. The impedance detector transmits the acquired impedance information to the control unit through a USB interface. The impedance measurement uses the four-electrode method. The four-electrode method uses four electrodes for impedance detection: including two current electrodes and two voltage electrodes. The current electrodes are used to apply an alternating current to the salmon fillet, and the voltage electrodes are used to measure the voltage drop across the salmon fillet.

[0118] The acquisition of impedance data is completed using two methods: horizontal electrode arrangement and vertical electrode arrangement. Sweep-frequency impedance measurement is used, and the measurement frequencies are 20 Hz, 2 kHz, and 20 kHz. The electrode patches use snap-on ECG patches, and the patches are embedded with electrodes made of highly conductive silver / silver chloride material. The snap-on design is adopted, and the patch substrate is made of soft and biocompatible polyurethane. The size of the four electrode patches is 1 cm × 1 cm. For the horizontal electrode arrangement, the electrode patches are attached parallel to the width direction of the fish fillet, with four electrodes attached. The current electrodes are on the outside, and the voltage electrodes are on the inside. When the electrodes are arranged horizontally, the fixed distance between the two voltage electrodes is d VV h , and the fixed distance between the voltage and current electrodes is d AV h and d VA h , and d AV h = d VA h , as Figure 3 shown. The horizontal impedance is used to analyze the freshness status of the local part of the fish fillet. For the vertical electrode arrangement, the electrode patches are attached along the length direction of the fish fillet, with four electrodes attached. The current electrodes are at both ends, and the voltage electrodes are near the middle. When the electrodes are arranged vertically, the fixed distance between the two voltage electrodes is dVVz, and the fixed distance between the voltage and current electrodes is d AV z and d VA z , and d AV z = d VA z , as Figure 4 shown. The vertical impedance is used to analyze the freshness status of the entire fish fillet. The measured horizontal impedance is denoted as Z 横 , and the measured vertical impedance is denoted as Z 纵 .

[0119] AsFigure 8 As shown, the humidity monitoring device includes a capacitive humidity plate 10, an LED display screen 11, a wire 12, and a detection patch 13. The LED display screen 11 is arranged on the capacitive humidity plate 10. The capacitive humidity plate 10 is connected to the detection patch 13 through the wire 12, and the humidity of the fish fillet is detected through the detection patch 13.

[0120] The control unit 6 can control the detection device to work through preset instructions, and send the collected image and impedance information to the computer through the connection line, so as to predict the freshness.

[0121] The operation process of this device is as follows: After the device is initialized, the fish fillet to be detected is placed on the fixed detection device. At this time, the needles on the fine needle carving substrate move under the gravity of the fish fillet, so as to fix the position of the fish fillet. Further, the lighting device (including a set of LED light sources on each side and an LED light source board directly above, with a power of 0.5W for each LED, providing sufficient light of 500 - 1000 lumens) is turned on, and the industrial camera (fixed focal length, resolution of 4112x3008 pixels, equipped with a 20cm fixed-focus lens) collects the image data of the fish fillet.

[0122] Further, the electrode patch is attached in the Figure 3 shown manner to measure the transverse impedance of the fish fillet. Specifically, the four-electrode method is used, including two current electrodes and two voltage electrodes. When the electrodes are arranged horizontally, the fixed distance between the two voltage electrodes is 30mm, and the fixed distance between the voltage and current electrodes is 25mm. At the same time, the position of the electrode patch is changed, and it is attached in the Figure 4 shown manner to measure the longitudinal impedance of the fish fillet. When the electrodes are arranged longitudinally, the fixed distance between the two voltage electrodes is 60mm, and the fixed distance between the voltage and current electrodes is 50mm.

[0123] In addition, the humidity of the fish fillet surface is measured by a capacitive hygrometer (relative humidity measurement range is 0% - 100%, accuracy is ±3%RH, and the LED display screen displays the humidity in real time). The measured humidity is about 80%. At the same time, the temperature information of the fish fillet is measured by an infrared thermometer (measurement range is -50°C to +300°C, accuracy ±2°C) on the top of the fish fillet, and the measured temperature is about 4°C.

[0124] After the above data measurement is completed, the control unit (model: Arduino Mega 2560, capable of controlling the detection device to work through preset instructions) transmits the obtained data to the computer through the data line for freshness prediction. At the same time, the fine needle carving substrate restoration device starts to operate. The stepper motor (model: NEMA17, rated current 1.2A, step angle 1.8°) controls the ball screw through the coupling. The lead of the ball screw is 4mm. When the stepper motor rotates forward 12.5 turns, the flat plate rises 5cm to restore the fine needle carving substrate to its original state. Further, when the stepper motor rotates backward 12.5 turns, the flat plate descends 5cm to return the flat plate to its initial position.

[0125] As Figure 1 shown, the present invention also provides a method for detecting the freshness of fish fillets by integrating vision and impedance, which is applied to the above device and includes:

[0126] S1: Collect the image, temperature, and humidity information of the fish fillet to be measured through the fish fillet freshness detection device, obtain the impedance data of the fish fillet to be measured by the four-electrode method, and measure the content of total volatile basic nitrogen in the fish fillet by the Kjeldahl method to calibrate its freshness;

[0127] S2: Screen the quality of the image data of the fish fillet, delete the data with inconsistent quality and prompt for re-collection;

[0128] S3: Calculate the size of the fish fillet based on the obtained image data of the fish fillet by combining the vision measurement method and the fine needle carving substrate measurement method, and extract its color, gloss, and texture features;

[0129] S4: Check the change rate of the temperature and humidity information of the detected fish fillet, compare the impedance amplitude and phase of the fish fillet measured by the four-electrode method with the preset intervals respectively, and screen the data quality;

[0130] S5: Calculate the transverse and longitudinal resistivity of the fish fillet under standard conditions according to the resistance, size, temperature, and humidity information of the fish fillet, correct the collected transverse and longitudinal impedance information with the size, temperature, and humidity information of the fish fillet, and extract the corresponding impedance features;

[0131] S6: Standardize and splice and fuse the obtained color features, gloss features, texture features, and impedance features of the fish fillet, determine the freshness grading standard according to the use of the fish fillet, construct a fish fillet freshness prediction model, use the color features, gloss features, texture features, and impedance features of the fish fillet as input values, and use the freshness grading of the fish fillet as the output to predict the freshness of the fish fillet;

[0132] S7: Add an artificial comparison program, calculate the error rate of the prediction model, and confirm the accuracy of the prediction result.

[0133] First of all, it should be noted that in this invention, salmon slices are selected as the embodiment to elaborate on the method described in this invention.

[0134] In S1, the image, temperature, and humidity information of the fish slices to be measured are collected by a fish slice freshness detection device, the impedance data of the fish slices to be measured is obtained by the four-electrode method, and the total volatile basic nitrogen content of the fish slices is measured by a Kjeldahl nitrogen analyzer to calibrate its freshness. Specifically:

[0135] Purchase 300 fresh salmon slices of different sizes. After transporting them to the laboratory, store them in a constant temperature incubator at 4°C. Every day, randomly select 30 slices from these 300 salmon slices for image collection and impedance measurement. After the collection, measure the volatile base nitrogen content of the salmon slices to determine the freshness of the salmon slices (in this embodiment, the four-classification method is adopted: fresh, TVB-N content ≤ 10mg / 100g; sub-fresh, TVB-N content: 10 - 20mg / 100g; not fresh, TVB-N content: 20 - 30mg / 100g; spoiled, TVB-N content ≥ 30mg / 100g). Continuously collect according to the above steps for 10 days to obtain a total of 300 samples. Among them, 260 samples are used as the training set, and 40 samples are used as the test set to test the accuracy of the model. The flow of the entire detection method can be represented by a timing diagram, as Figure 2 shown;

[0136] In this embodiment, the relative humidity measured by the capacitance hygrometer is about 80%, and the temperature measured by the infrared thermometer is about 4°C.

[0137] In S2, screen the quality of the image data of the fish slices, delete the data with inconsistent quality, and prompt for re-collection. Specifically:

[0138] Convert the original RGB image I of the fish slice into a grayscale image I gray , as:

[0139]

[0140] In the formula, the value of ω is 0.2989, the value of is 0.5870, and the value of τ is 0.1140;

[0141] Judge the noise pollution, local and overall quality in the image data, and perform block processing on the input image. The image size is M T ×N T , and it is divided into p T ×q T small blocks, and each small block is defined as S ij , where i, j represent the index of each small block, as:

[0142]

[0143] Wherein, H ij is the local complexity of each small piece S ij , P k represents the probability that the gray value k appears in S ij , σ H represents the standard deviation of the local complexity, represents the average value of all local complexities;

[0144] In this embodiment, the image size is 4112×3008 pixels, divided into 10×10 small pieces, and the normal local complexity of each small piece S ij is about 0.556, and the standard deviation of the local complexity is about 0.08;

[0145] Screen the clarity of the image data, map the image to the frequency domain, and judge the concentration of high-frequency energy, which is:

[0146]

[0147] Wherein, F(u, v) is the complex value of the output frequency domain, u represents the frequency in the horizontal direction, v represents the frequency in the vertical direction, j is the imaginary unit, S f represents the concentration of high-frequency energy, and H is the index set of the high-frequency region |F(u, v)| is the amplitude of (u, v) in the frequency domain;

[0148] In this embodiment, the concentration of high-frequency energy S f is 0.7;

[0149] Establish a scoring mechanism for the quality of image data, standardize the scores, screen the image data. When the final score is greater than 70, it means that the quality of the image data meets the standard and the next step is carried out. Otherwise, it is prompted that the quality does not meet the standard and re-acquisition is carried out. The scoring mechanism is specifically:

[0150] Q = μ1·exp(-σ H ) + μ2·S f ;

[0151]

[0152] Wherein, Q is the score calculated by the image data quality scoring mechanism, σ H represents the standard deviation of the local complexity, μ1 and μ2 represent the weight coefficients, which are determined through experiments, S f represents the concentration of high-frequency energy, Q n represents the standardized score, Q min is the minimum value of the scores in the experiment, Qmax is the maximum score in the experiment;

[0153] In this embodiment, μ1 and μ2 are 0.7 and 0.3, the minimum score is 0.7, the maximum score is 0.9. Among the selected image data, the score calculated by the image data quality scoring mechanism is 0.8561, and the standardized score is 78.05. The image quality score has reached 78.05, which is greater than 70 and meets the standard, so there is no need to re - collect.

[0154] In S3, based on the obtained image data of the fish fillet, combined with the vision measurement method and the fine needle carving substrate measurement method, calculate the size of the fish fillet, and extract its color, gloss and texture features. The specific steps are as follows:

[0155] The industrial camera is equipped with a fixed - focus lens of 20 cm, and the obtained image size is 4112×3008 pixels;

[0156] For calculating the size of the fish fillet, first identify the image edge, and separate the outermost contour image of the fish fillet from it, and find the smallest rectangle that encloses the outermost contour of the fish fillet This rectangle has pixel dimensions of a×b;

[0157] In this embodiment, the pixel dimension of the long side a is about 3396 pixels, and the pixel dimension of the short side b is about 1230 pixels;

[0158] Taking the actual distance L2 between two adjacent needles in the fine needle carving substrate, whose pixel dimension is c, as a reference dimension, calculate the size 1 of the fish fillet as:

[0159]

[0160] In the formula, A is the length of the size 1 of the fish fillet, and B is the width of the size 1 of the fish fillet;

[0161] In this embodiment, the actual distance L2 between two adjacent needles in the fine needle carving substrate is 4 mm;

[0162] In the outermost contour image, find the number m and n of the needles blocked by the longest length and width diameters of the fish fillet. Among them, calculate the distance D(x,y) from any point (x,y) in the input fish fillet image to the nearest point on the fish fillet contour as:

[0163]

[0164] In the formula, D(x,y) is the distance from any point (x,y) in the image to the nearest point on the fish fillet contour, (x * ,y * ) ∈ fish is the set of all points on the fish fillet edge contour, For calculating the shortest straight-line distance between two points;

[0165] Use U(x,y) to map D(x,y) of the fish fillet into a periodic function to determine whether the point (x,y) is near the needle, which is:

[0166]

[0167] In the formula, U(x,y) describes the characteristic intensity of the point (x,y) under the needle carving substrate, and L2 represents the distance between adjacent needles;

[0168] In this embodiment, the distance L2 between adjacent needles is 4 mm;

[0169] For the position (x i ,y j ) of each needle, determine the possibility of it being blocked by the fish fillet, which is:

[0170]

[0171] In the formula, r is the radius of the needle, and P cover (x i ,y j ) is the possibility of the needle being blocked by the fish fillet;

[0172] Calculate the total number N of needles blocked by the fish fillet cover which is:

[0173]

[0174] In the formula, M and N are the numbers of needles in the length and width directions on the needle carving substrate, τ * is the threshold of the occlusion probability, 1[P cover (x i ,y j )>τ * means that if the condition P cover (x i ,y j )>τ * is satisfied, its value is 1, otherwise it is 0;

[0175] In this embodiment, the radius of the needle is 0.6 mm; the threshold τ of the occlusion probability * is 0.5; the numbers M and N of needles in the length and width directions on the needle carving substrate are 100 and 75 respectively; the distance L2 between adjacent needles is 4 mm;

[0176] Calculate the occlusion numbers m and n of the needles along the length and width directions of the fish fillet respectively, which are:

[0177]

[0178] In the formula, projection(xi )、projection(y i ) maps the positions of the needles in the length and width directions. projection(x i ) = x i , projection(y i ) = y i , projection(x i ) ∩ U and projection(y i ) ∩ U represent the intersections of U with the projections projection(x i ) and projection(y i ) of the needle in the length and width directions;

[0179] Perform accuracy verification based on the total number of needles blocked by the fish fillet and the number of needles blocked in the length and width directions, which is:

[0180] N cover = m × n;

[0181] In this embodiment, the total number of needles blocked by the fish fillet is approximately 1250; the number of needles blocked in the length and width directions is approximately 50 and 25 respectively;

[0182] Furthermore, use the distance L2 between adjacent needles to find the length S and width P of the fish fillet size 2, and combine the two to find the final size length X and width Y, which is:

[0183]

[0184] In the formula, S is the length of the fish fillet size 2 obtained by using the fine needle carving substrate measurement method, and P is the width of the fish fillet size 2 obtained by using the fine needle carving substrate measurement method;

[0185] Extract the color features of the fish fillet, use the original RGB color space of the image, and then calculate the average value of its three color channels as the color feature, which is:

[0186]

[0187] In the formula, a × b is the pixel size of the smallest rectangle enclosing the outermost contour of the fish fillet, P R , P G and P B respectively represent the average values of the red, green, and blue channels, and I i,j represents the pixel at the i-th row and j-th column of the image I;

[0188] In this embodiment, the extracted color feature is approximately RGB(120, 100, 90);

[0189] Extract the gloss feature of the fish fillet, where the gloss feature is represented by the average brightness of the image, which is:

[0190]

[0191] In the formula, G p represents the average brightness of the image, and a×b is the pixel size of the smallest rectangle enclosing the outermost contour of the fish fillet ; represents the pixel brightness of the i-th row and j-th column of the image;

[0192] In this embodiment, the gloss feature can be expressed as: the average brightness is about 70;

[0193] Extract the texture feature of the fish fillet. The texture feature is obtained through contrast, entropy, and homogeneity, which is:

[0194]

[0195] D = ∑ i,j (i - j) 2 P(i,j);

[0196] S = -∑ i,j P(i,j)log(P(i,j));

[0197]

[0198] In the formula, P(i,j) is the gray-level co-occurrence matrix, I gray is the grayscale image, i and j are gray levels, Δx, Δy are relative displacements, D is the contrast, S is the entropy, and T is the homogeneity;

[0199] In this embodiment, the approximate values obtained are: contrast 25.5, entropy 1.35, and homogeneity 0.92.

[0200] In S4, perform a rate verification on the temperature and humidity information of the detected fish fillet. Compare the impedance amplitude and phase of the fish fillet measured by the four-electrode method with the preset intervals respectively to screen the data quality. Specifically:

[0201] Adopt the method of dynamic parameter adjustment for the detected temperature and humidity information of the fish fillet. Starting from the fourth group of data, compare the variation ranges of the first three groups of data, dynamically determine the parameters, calculate the mean value of the first three groups of data, establish the threshold variation range, and compare the currently measured data with the threshold. If the currently measured data is within the range, it indicates that the quality of the temperature and humidity data meets the standard, and proceed to the next step. Otherwise, prompt that the quality does not meet the standard and perform re-acquisition, which is:

[0202]

[0203] Wherein, T n-1 、T n-2 、T n-3 are three groups of temperature data before the current temperature measurement value T n , is the average value of three groups of temperature data before the current temperature measurement value T n , k T is the parameter of temperature change, y T is the temperature threshold interval, S n-1 、S n-2 、S n-3 are three groups of humidity data before the current humidity measurement value S n , is the average value of three groups of humidity data before the current humidity measurement value S n , k S is the parameter of humidity change, y S is the humidity threshold interval;

[0204] Compare the impedance amplitude and phase of the fish fillet measured by the four-electrode method with the preset intervals respectively. Among them, the preset amplitude interval is [α Ω , β Ω , and the preset phase interval is [α°, β°]. If the currently measured data is within the range, it means that the quality of the impedance data meets the standard, and the next step is carried out. Otherwise, it is prompted that the quality does not meet the standard, and re-acquisition is carried out;

[0205] In this embodiment, k T ∈[0, 0.2], y T ∈[4 - 4k T , 4 + 4k T , k S ∈[0, 0.25], y T ∈[0.8 - 0.8k S , 0.8 + 0.8k S , and the preset amplitude interval is [50, 500], and the preset phase interval is [-5, -45].

[0206] In S5, calculate the transverse and longitudinal resistivity of the fish fillet in the standard environment according to the resistance, size, temperature and humidity information of the fish fillet, correct the collected transverse and longitudinal impedance information using the size, temperature and humidity information of the fish fillet, and extract the corresponding impedance characteristics. Specifically:

[0207] At different frequencies, measure the impedance amplitude and phase of the fish fillet by the four-electrode method, and calculate the rectangular coordinate form of the fish fillet impedance, which is:

[0208]

[0209] Z横 / 纵 = R f + jx f ;

[0210] Wherein, Z 横 / 纵 is the impedance measured under the transverse or longitudinal arrangement of the electrodes, |Z 横 / 纵 | is the amplitude of the impedance measured under the transverse or longitudinal arrangement of the electrodes, is the phase difference of the voltage relative to the current, R f横 / 纵 is the real part in the rectangular coordinate form of the impedance of the fish fillet, x f横 / 纵 is the imaginary part in the rectangular coordinate form of the impedance of the fish fillet;

[0211] According to the resistance, size, temperature and humidity information of the fish fillet, the transverse and longitudinal resistivity of the fish fillet under the standard environment is calculated as:

[0212]

[0213]

[0214] Wherein, ρ H / ρ Z is the transverse / longitudinal resistivity of the fish fillet under the standard conditions of T0°C, S0%, R f横 / R f横 is the transverse / longitudinal resistance of the fish fillet under the current detection environment of T°C, S%, X is the length of the fish fillet, Y is the width of the fish fillet, H is the thickness of the fish fillet, d VV h is the fixed distance between the two voltage electrodes when the electrodes are arranged transversely, d VV z is the fixed distance between the two voltage electrodes when the electrodes are arranged longitudinally, α is the resistance temperature coefficient, β is the resistance humidity coefficient, γ is the non-linear coefficient of humidity and temperature, T0 is the standard temperature, S0 is the standard humidity, T is the temperature during detection, S is the humidity during detection;

[0215] In this embodiment, the resistance temperature coefficient α is 0.005°C -1 , the resistance humidity coefficient β is 0.002% -1 , γ is 0.2, the standard temperature T0 is 25°C, the standard humidity S0 is 60%, the temperature T during detection is 4°C, the humidity S during detection is 80%, the thickness H of the fish fillet is 10 mm, the fixed distance between the two voltage electrodes when the electrodes are arranged transversely is 30 mm, and the fixed distance between the two voltage electrodes when the electrodes are arranged longitudinally is 60 mm. Under the specific detection environment, when the frequency is 20 Hz, the transverse resistivity ρ H of the salmon fillet is 1.5 Ω·m, the longitudinal resistivity ρ Z is 1.8 Ω·m, and when the frequency is 2 kHz, the transverse resistivity ρ His 1.0 Ω·m, the longitudinal resistivity ρ Z is 1.2 Ω·m, the transverse resistivity ρ of the salmon fillet at a frequency of 20 kHz H is 0.8 Ω·m, the longitudinal resistivity ρ Z is 1.0 Ω·m;

[0216] According to the temperature, humidity, and size information of the fish fillet, the impedance Z 横 、Z 纵 is corrected, and the corrected impedance is expressed as:

[0217]

[0218] In the formula, α is the temperature coefficient of resistance; β is the humidity coefficient of resistance, γ is the non-linear coefficient of humidity and temperature, T0 is the standard temperature, S0 is the standard humidity, T is the temperature during detection, S is the humidity during detection, X is the length of the fish fillet, Y is the width of the fish fillet, X 标 is the reference standard fish fillet length, Y 标 is the reference standard fish fillet width;

[0219] In this embodiment, X 标 The reference standard fish fillet length is 200 mm, Y 标 The reference standard fish fillet width is 100 mm;

[0220] Feature extraction is performed on the impedance of the fish fillet, and the extracted features include the relative peak resistance ratio K and the impedance modulus change rate Δ|Z f |. The relative peak resistance ratio can analyze the frequency-dependent characteristics in the impedance spectrum and the relative relationship between capacitance and resistance. The impedance modulus change rate represents the dynamic characteristics of the conductivity and capacitance of the fish fillet tissue changing with frequency, and is:

[0221]

[0222] In the formula, R flow represents the low-frequency resistance measured of the fish fillet, R fhigh represents the high-frequency resistance measured of the fish fillet, x max represents the reactance peak value in the graph plotted with the real part of the impedance as the horizontal axis and the imaginary part as the vertical axis, R max represents the maximum resistance value in the graph plotted with the real part of the impedance as the horizontal axis and the imaginary part as the vertical axis, R f is the real part in the impedance, x f is the imaginary part in the impedance, f a 、f b are two adjacent frequency points;

[0223] In this embodiment, the measured relative impedance peak ratio K ranges from 1.2 to 1.4, and 1.39 is taken here. The change rate of the impedance modulus Δ|Z f | ranges from -0.009 to -0.005, and -0.007 is taken here;

[0224] The obtained impedance characteristics are transformed into feature vectors so that they can be fused with visual features in a high-dimensional space, as follows:

[0225] K = ReLU(W1·K + b1);

[0226] Δ|Z f | = ReLU(W2·Δ|Z f | + b1);

[0227] In the formula, K and Δ|Z f | are the impedance characteristics after being transformed into feature vectors. ReLU is the activation function, W is the corresponding weight matrix, and K and Δ|Z f | are the impedance before transformation, and b is the bias vector;

[0228] In this embodiment, the dimension of the weight matrix W and the bias vector b is 1×1, W1 = 0.5, b1 = 2, W2 = 0.5, and b2 = 2.

[0229] In S6, the obtained color characteristics, gloss characteristics, texture characteristics, and impedance characteristics of the fish fillets are standardized and spliced and fused. The freshness grading standard is determined according to the use of the fish fillets, and a fish fillet freshness prediction model is constructed. Using the color characteristics, gloss characteristics, texture characteristics, and impedance characteristics of the fish fillets as input values and the freshness grading of the fish fillets as output, the freshness of the fish fillets is predicted, specifically as follows:

[0230] The obtained color characteristics, gloss characteristics, texture characteristics, and impedance characteristics of the fish fillets are standardized and multi-modal feature fusion is performed to establish a fish fillet freshness prediction model. Among them, the input layer of the prediction model receives the fused feature vector F = [P R , P G , P B , G p , D, S, T, ρ H , ρ Z , K, Δ|Z f |]. First, the fused feature vector F is normalized to obtain F°. Each neuron in the input layer is fully connected to each neuron in the hidden layer 1 to form a weight matrix W o1 . The number of neurons in the hidden layer 1 is δ1, the dropout rate of the Dropout layer is 50%, the activation function is ReLU, and the input feature F° is linearly combined with the weight matrix of the hidden layer 1 through the fully connected layer and the bias vector b o1, that is, Z o1 =W o1 ·F° + b o1 , after further processing by the activation function, H is obtained o1 = ReLU(Z o1 ), output to the Dropout layer, and the Dropout layer randomly discards the outputs of some neurons. The output H of the remaining neurons o1 * is passed to the hidden layer 2, and the number of neurons in the hidden layer 2 is The hidden layer 2 receives the output H from the hidden layer 1 o1 * , and through the fully connected layer for linear transformation to obtain Z o2 =W o2 ·H o1 * +b o2 , the hidden layer 2 introduces a batch normalization layer after the fully connected layer to standardize the output after linear transformation to obtain BN(Z o2 ), and the output after batch normalization passes through the ReLU activation function to introduce non-linear characteristics to obtain H o2 = ReLU(BN(Z o2 ))), the output layer receives the output H from the hidden layer 2 o2 , the number of neurons in the output layer varies according to the number of levels of the required freshness, so as to predict the freshness of the fish fillets. After passing through the prediction model, the fish fillets are classified using the spftmax activation function. The formula is:

[0231]

[0232] In the formula, is a four-dimensional vector representing the probabilities of four classifications.

[0233] In S7, an artificial comparison program is added to calculate the error rate of the prediction model and confirm the accuracy of the prediction result. Specifically:

[0234] Randomly select a part of the fish fillet samples X f , and manually use standard detection tools to verify the freshness of the fish fillets, confirm the accuracy of the prediction result, compare the predicted output of the model with the true detection result, and count the number of prediction errors X c , if the error rate is lower than the threshold θ c , it is considered that the predicted output is accurate and continue the detection. If the error rate is high, it is prompted to perform system correction. Among them, the formula for the error rate is:

[0235]

[0236] In the formula, μ dis the error rate of the model prediction output in the fish fillet sample, and the threshold θ is set in the present invention c to be 8%.

[0237] Forty samples of the detection set are used for verification respectively using different detection methods, and the correct rates obtained are as Figure 5 shown. According to Figure 5 it can be seen that: the detection method provided by the embodiment of the present invention has the highest correct rate under various freshness levels.

Claims

1. A method for detecting the freshness of fish fillets by integrating vision and electrical impedance, characterized in that, Including: S1: Collect the image, temperature, and humidity information of the fish fillet to be tested through a fish fillet freshness detection device, obtain the impedance data of the fish fillet to be tested using the four-electrode method, and measure the total volatile basic nitrogen content of the fish fillet by a Kjeldahl nitrogen analyzer to calibrate its freshness; S2: Screen the quality of the image data of the fish fillet, delete the data with inconsistent quality, and prompt for re-collection; S3: Calculate the size of the fish fillet based on the obtained image data of the fish fillet by combining the visual measurement method and the fine needle carving substrate measurement method, and extract its color, gloss, and texture features; S4: Perform a change rate verification on the detected temperature and humidity information of the fish fillet, compare the measured impedance amplitude and phase of the fish fillet by the four-electrode method with the preset intervals respectively, and screen the data quality; S5: Calculate the transverse and longitudinal resistivity of the fish fillet under standard conditions according to the resistance, size, temperature, and humidity information of the fish fillet, correct the collected transverse and longitudinal impedance information using the size, temperature, and humidity information of the fish fillet, and extract the corresponding impedance features; S6: Standardize and splice and fuse the obtained color features, gloss features, texture features, and impedance features of the fish fillet, determine the freshness grading standard according to the use of the fish fillet, construct a fish fillet freshness prediction model, use the color features, gloss features, texture features, and impedance features of the fish fillet as input values, and the freshness grading of the fish fillet as the output to predict the freshness of the fish fillet; S7: Add an artificial comparison program, calculate the error rate of the prediction model, and confirm the accuracy of the prediction result.

2. The method according to claim 1, wherein In S2, when screening the quality of the image data of the fish fillet, deleting the data with inconsistent quality, and prompting for re-collection, specifically: Convert the original RGB image I of the fish fillet into a grayscale image I gray , as follows: where ω takes a value of 0.2989, takes a value of 0.5870, and τ takes a value of 0.1140; Judge the noise pollution, local and overall quality in the image data, and perform block processing on the input image. The size of the image is M T ×N T , and it is divided into p T ×q T small blocks, and each small block is defined as S ij , where i and j represent the indices of each small block, which are: Where, H ij is the local complexity of each small piece S ij , P k represents the probability that the gray value k appears in S ij , σ H represents the standard deviation of the local complexity, and represents the average value of all local complexities; Screen the clarity of the image data, map the image to the frequency domain, and judge the concentration of high-frequency energy, specifically: Wherein, F(u, v) is the complex value in the output frequency domain, u represents the frequency in the horizontal direction, v represents the frequency in the vertical direction, j is the imaginary unit, S f represents the concentration of high-frequency energy, and H is the index set of the high-frequency region |F(u, v)| is the amplitude of (u, v) in the frequency domain; Establish a scoring mechanism for the quality of image data, standardize the scores, screen the image data. When the final score is greater than 70, it indicates that the quality of the image data meets the standard, and proceed to the next step. Otherwise, prompt that the quality does not meet the standard and re-collect. The scoring mechanism is specifically: Q = μ1·exp(-σ H ) + μ2·S f ; Where Q is the score calculated by the image data quality scoring mechanism, and σ H represents the standard deviation of local complexity, μ1 and μ2 represent weight coefficients determined through experiments, and S f represents the concentration of high-frequency energy, Q n represents the standardized score, Q min is the minimum value of the scores in the experiment, and Q max is the maximum value of the scores in the experiment.

3. The method according to claim 2, wherein In S3, when calculating the size of the fish fillet based on the obtained image data of the fish fillet by combining the visual measurement method and the fine needle carving substrate measurement method, and extracting its color, gloss, and texture features, it specifically includes the following steps: To calculate the size of the fish fillet, first identify the edges of the image, separate the outermost contour image of the fish fillet from it, and find the smallest rectangle that encloses the outermost contour of the fish fillet. This rectangle has pixel dimensions of a × b. Taking the actual distance L2 between two adjacent needles in the fine needle carving substrate, whose pixel size is c, as the reference size, calculate the size 1 of the fish fillet, specifically: In the formula, A is the length of the size 1 of the fish fillet, and B is the width of the size 1 of the fish fillet; In the outermost contour image, find the number of needles m and n blocked by the longest diameter of the length and width of the fish fillet. Among them, calculate the distance D(x, y) from any point (x, y) in the input fish fillet image to the nearest point on the fish fillet contour, specifically: where D(x, y) is the distance from any point (x, y) in the image to the nearest point on the fish fillet contour, (x * , y * ) ∈ fish is the set of all points on the edge contour of the fish fillet, used to calculate the shortest straight-line distance between two points; Use U(x, y) to map the D(x, y) of the fish fillet to a periodic function to judge whether the point (x, y) is near the needle, specifically: In the formula, U(x, y) describes the feature intensity of the point (x, y) under the needle carving substrate, and L2 represents the distance between adjacent needles; For the position (x i , y j ) of each needle, determine the possibility of it being blocked by the fish fillet, which is: where r is the radius of the needle, and P cover (x i , y j ) is the probability that the needle is blocked by the fish slice; Calculate the total number N of fish slice occlusion needles cover It is: where M and N are the numbers of needles in the length and width directions on the needle carving substrate, and τ * is the threshold of the occlusion probability, and 1[P cover (x i , y j ) > τ * means that if the condition P cover (x i , y j ) > τ * is satisfied, its value is 1, otherwise it is 0; Calculate the number of needles blocked m and n along the length and width directions of the fish fillet respectively, specifically: where projection(x i ) and projection(y i ) map the positions of the needle in the length and width directions, projection(x i ) = x i , projection(y i ) = y i , projection(x i ) ∩ U and projection(y i ) ∩ U represent the intersections of U with the projections projection(x i ) and projection(y i ) of the needle in the length and width directions; According to the total number of needles blocked by the fish fillet and the number of needles blocked in the length and width directions, accuracy verification is carried out as follows: N cover = m × n; Furthermore, the length S and width P of the fish fillet size 2 are obtained by using the distance L2 between adjacent needles, and the final size length X and width Y are obtained by combining the two as follows: In the formula, S is the length of the fish fillet size 2 obtained by using the fine needle carving substrate measurement method, and P is the width of the fish fillet size 2 obtained by using the fine needle carving substrate measurement method; Extract the color features of the fish fillet, use the original RGB color space of the image, and then calculate the average value of its three color channels as the color feature as follows: where a×b is the pixel size of the smallest rectangle enclosing the outermost contour of the fish fillet , P R , P G and P B respectively represent the average values of the red, green, and blue channels, and I i,j represents the pixel at the i-th row and j-th column of the image I; Extract the gloss feature of the fish fillet, where the gloss feature is represented by the average brightness of the image as follows: where G p represents the average luminance of the image, a·b is the pixel size of the smallest rectangle enclosing the outermost contour of the fish fillet, and represents the pixel luminance at the i-th row and j-th column of the image; Extract the texture features of the fish fillet, and the texture features are obtained through contrast, entropy, and homogeneity as follows: where P(i,j) is the gray-level co-occurrence matrix, I gray is the grayscale image, i and j are gray levels, Δx, Δy are relative displacements, D is the contrast, S is the entropy, and T is the homogeneity.

4. The method according to claim 3, wherein In S4, the rate of change verification is performed on the temperature and humidity information of the detected fish fillet, and the impedance amplitude and phase of the fish fillet measured by the four-electrode method are respectively compared with the preset interval to screen the data quality, specifically as follows: For the temperature and humidity information of the detected fish fillet, a method of dynamic parameter adjustment is adopted. Starting from the fourth group of data, compare the change amplitudes of the first three groups of data, dynamically determine the parameters, calculate the average value of the first three groups of data, establish the threshold change range, and compare the currently measured data with the threshold. If the currently measured data is within the range, it indicates that the quality of the temperature and humidity data meets the standard, and the next operation is performed. Otherwise, it is prompted that the quality does not meet the standard and re-collection is performed as follows: Where, T n-1 , T n-2 , T n-3 are three groups of temperature data before the current temperature measurement value T n ; is the average value of three groups of temperature data before the current temperature measurement value T n , k T is the parameter of temperature change, y T is the temperature threshold interval, S n-1 , S n-2 , S n-3 are three groups of humidity data before the current humidity measurement value S n ; is the average value of three groups of humidity data before the current humidity measurement value S n , k S is the parameter of humidity change, y S is the humidity threshold interval; Compare the impedance amplitude and phase of the fish fillet measured by the four-electrode method with the preset intervals respectively, where the preset amplitude interval is [α Ω , β Ω , and the preset phase interval is [α°, β°]. If the currently measured data is within the range, it indicates that the quality of the impedance data meets the standard, and the next step is carried out; otherwise, it is prompted that the quality does not conform, and re-acquisition is carried out.

5. The method according to claim 4, wherein In S5, the transverse and longitudinal resistivity of the fish fillet under standard conditions is calculated according to the resistance, size, temperature, and humidity information of the fish fillet. The collected transverse and longitudinal impedance information is corrected by using the size, temperature, and humidity information of the fish fillet, and the corresponding impedance features are extracted, specifically as follows: At different frequencies, the impedance amplitude and phase of the fish fillet are measured by the four-electrode method, and the rectangular coordinate form of the fish fillet impedance is calculated as follows: Z 横 / 纵 = R f + jx f ; where, Z 横 / 纵 is the impedance measured under the transverse or longitudinal arrangement of the electrodes, |Z 横 / 纵 | is the amplitude of the impedance measured under the transverse or longitudinal arrangement of the electrodes, is the phase difference of the voltage relative to the current, R f横 / 纵 is the real part in the rectangular coordinate form of the impedance of the fish fillet, x f横 / 纵 is the imaginary part in the rectangular coordinate form of the impedance of the fish fillet; The transverse and longitudinal resistivity of the fish fillet under standard conditions is calculated according to the resistance, size, temperature, and humidity information of the fish fillet as follows: Where ρ H / ρ Z is the transverse / longitudinal resistivity of the fish fillet under standard conditions of T0℃ and S0%, R f横 / R f横 is the transverse / longitudinal resistance of the fish fillet under the current detection environment of T℃ and S%, X is the length of the fish fillet, Y is the width of the fish fillet, H is the thickness of the fish fillet, d VV h is the fixed distance between the two voltage electrodes when the electrodes are arranged transversely, d VV z is the fixed distance between the two voltage electrodes when the electrodes are arranged longitudinally, α is the resistance temperature coefficient, β is the resistance humidity coefficient, γ is the non-linear coefficient of humidity and temperature, T0 is the standard temperature, S0 is the standard humidity, T is the temperature during detection, and S is the humidity during detection; According to the temperature, humidity, and size information of the fish fillet, correct the impedance Z 横 , Z 纵 , and the corrected impedance is expressed as: Wherein, α is the temperature coefficient of resistance; γ is the humidity coefficient of resistance, β is the non-linear coefficient of humidity and temperature, T0 is the standard temperature, S0 is the standard humidity, T is the temperature during detection, S is the humidity during detection, X is the length of the fish fillet, Y is the width of the fish fillet, X 标 is the reference standard fish fillet length, Y 标 is the reference standard fish fillet width; Extract the impedance characteristics of fish fillets. The extracted characteristics include the relative peak resistance ratio K and the change rate of impedance modulus Δ|Z f |. The relative peak resistance ratio can analyze the frequency-dependent characteristics in the impedance spectrum and the relative relationship between capacitance and resistance. The change rate of impedance modulus represents the dynamic characteristics of the conductivity and capacitance of fish fillet tissues changing with frequency, which is: Wherein, R flow represents the low-frequency resistance measured for the fish fillet, R fhigh represents the high-frequency resistance measured for the fish fillet, x max represents the peak value of reactance in the graph plotted with the real part of the impedance as the horizontal axis and the imaginary part as the vertical axis, R max represents the maximum value of resistance in the graph plotted with the real part of the impedance as the horizontal axis and the imaginary part as the vertical axis, R f is the real part in the impedance, x f is the imaginary part in the impedance, f a 、f b are two adjacent frequency points; The obtained impedance features are transformed into feature vectors so that they can be fused with visual features in a high-dimensional space as follows: K = ReLU(W1·K + b1); Δ|Z f | = ReLU(W2·Δ|Z f | + b1); Wherein, K, Δ|Z f | is the impedance characteristic after being transformed into a feature vector, ReLU is the activation function, W is the corresponding weight matrix, K, Δ|Z f | is the impedance before transformation, and b is the bias vector.

6. The method according to claim 5, characterized in that, In S6, the obtained color features, gloss features, texture features, and impedance features of the fish fillet are standardized and spliced and fused. According to the use of the fish fillet, the freshness grading standard is determined, and a fish fillet freshness prediction model is constructed. Using the color features, gloss features, texture features, and impedance features of the fish fillet as input values and the fish fillet freshness grading as output, the freshness of the fish fillet is predicted, specifically as follows: Normalize the obtained color features, gloss features, texture features, and impedance features of the fish fillets and perform multimodal feature fusion to establish a fish fillet freshness prediction model. Among them, the input layer of the prediction model receives the fused feature vector F = [P R , P G , P B , G p , D, S, T, ρ H , ρ Z , K, Δ|Z f |]. First, normalize the fused feature vector F to obtain F°. Each neuron in the input layer is fully connected to each neuron in the hidden layer 1 to form a weight matrix W o1 . The number of neurons in the hidden layer 1 is δ1, the dropout rate of the Dropout layer is 50%, the activation function is ReLU. The input feature F° is linearly combined with the weight matrix of the hidden layer 1 through the fully connected layer and added with a bias vector b o1 , that is, Z o1 = W o1 ·F° + b o1 . Further, after being processed by the activation function, H o1 = ReLU(Z o1 ) is obtained and output to the Dropout layer. The Dropout layer randomly discards the output of some neurons, and the output H o1 * of the remaining neurons is passed to the hidden layer 2. The number of neurons in the hidden layer 2 is . The hidden layer 2 receives the output H o1 * from the hidden layer 1 and performs a linear transformation through the fully connected layer to obtain Z o2 = W o2 ·H o1 * + b o2 . The hidden layer 2 introduces a batch normalization layer after the fully connected layer to standardize the output after the linear transformation to obtain BN(Z o2 ). The output after batch normalization passes through the ReLU activation function to introduce non-linear characteristics to obtain H o2 = ReLU(BN(Z o2 )). The output layer receives the output H o2 from the hidden layer 2. The number of neurons in the output layer varies according to the number of levels of the required freshness, so as to predict the freshness of the fish fillets. After passing through the prediction model, the softmax activation function is used to classify the fish fillets. The formula is: wherein, is a four-dimensional vector representing the probabilities of four classifications.

7. The method according to claim 6, characterized in that In S7, an artificial comparison program is added to calculate the error rate of the prediction model and confirm the accuracy of the prediction result, specifically as follows: Randomly select a part of the fish fillet sample X f , manually verify the freshness of the fish fillets using standard detection tools, confirm the accuracy of the prediction results, compare the predicted output of the model with the actual detection results, and count the number of prediction errors X c , if the error rate is lower than the threshold It is considered that the predicted output is accurate and continue the detection. If the error rate is high, prompt for system correction. Among them, the calculation formula for the error rate is: where μ d is the error rate of the model prediction output in the fish fillet sample.

8. A fish fillet freshness detection device integrating vision and electrical impedance, characterized in that, Including: Installation base, industrial camera, top light source board, support column, electrode patch, fine needle carving substrate, control unit, impedance detector, reduction device, lighting device, humidity monitoring device and infrared thermometer. The reduction device, support column and control unit are arranged on the installation base. The fine needle carving substrate is arranged on the installation base through a vertical rod. The lighting device is arranged on the vertical rod. And the fine needle carving substrate is located above the reduction device. The top light source board is arranged corresponding to the fine needle carving substrate on the front side of the support column through an extension plate. The industrial camera is arranged at the bottom of the top light source board. The reduction device is used to restore the fine needle carving substrate. The impedance detector is also arranged on the installation base. The impedance detector is connected with 4 electrode patches for collecting the impedance of the fish fillet. The humidity monitoring device is used to collect the humidity information of the fish fillet. The infrared thermometer is used to collect the temperature information of the fish fillet.

9. The device according to claim 8, characterized in that The reduction device includes a fixed frame, a stepping motor, a stepping motor fixing frame, a coupling, a fixed end, a ball screw, a support column, a slide table and a flat plate. The fixed frame is arranged on the installation base. The stepping motor fixing frame is arranged on the fixed frame. The stepping motor is arranged on the stepping motor fixing frame. The fixed end is arranged in the middle of the fixed frame. The support columns are arranged on both sides of the top of the fixed end. The slide table is slidably connected to the support columns. Both sides of the slide table are connected to the flat plate through connecting rods. The coupling passes through the fixed end to connect the ball screw. The ball screw is slidably connected to the slide table for driving the slide table to slide up and down along the support columns. The flat plate is located directly below the fine needle carving substrate.

10. The device according to claim 8, characterized in that The humidity monitoring device includes a capacitance humidity board, an LED display screen, a wire and a detection patch. The LED display screen is arranged on the capacitance humidity board. The capacitance humidity board is connected to the detection patch through a wire. The humidity of the fish fillet is detected through the detection patch.

Images