An acoustic vibration-based non-destructive detection method and device for the freshness of closed shell oysters

Abstract

This invention belongs to the field of non-destructive testing of the freshness of oysters, and provides a method and apparatus for non-destructive testing of the freshness of oysters based on acoustic vibration. The method includes: collecting acoustic vibration signals and images of oysters; extracting audio and morphological features; constructing fused features; and determining the freshness of oysters using a non-destructive testing model. This invention constructs a model for non-destructive testing of oyster freshness by extracting features and using machine learning from the sound and image information of oysters. This solves the problem of not being able to determine oyster freshness using sound information, improves the speed and accuracy of oyster freshness analysis, and reduces damage to the oysters used for testing.

Description

Technical Field

[0001] This invention relates to the field of non-destructive testing of the freshness of closed oysters, and in particular to a method and apparatus for non-destructive testing of the freshness of closed oysters based on acoustic vibration. Background Technology

[0002] During post-harvest storage, oysters, due to their high moisture content, active endogenous enzymes, and neutral pH, may decompose under the mediation of bacteria and fungi, leading to spoilage, nutrient loss, and even potential threats to human health. The oyster shell plays a crucial role in protecting the oyster meat. During storage, the shell prevents external bacteria, dust, and other contaminants from entering the oyster meat, maintaining its cleanliness. Furthermore, the shell prevents oxygen entry and moisture loss, avoiding oxidation and spoilage. Therefore, compared to shelled oysters, closed oysters better preserve the quality of their meat.

[0003] However, due to the obstruction of the oyster shell, existing non-destructive testing techniques such as hyperspectral, near-infrared reflectance spectroscopy, Raman spectroscopy, and X-rays have difficulty penetrating the shell to probe deep into the oyster's interior. This makes it difficult to accurately detect dynamic changes in the oyster's freshness, posing a challenge to ensuring food safety and quality control. Sound waves are a common type of mechanical wave, carrying important information about the vibration source and its propagation medium. Because acoustic vibration detection technology is non-invasive, penetrating, and non-radiative, it can be applied to develop rapid, non-destructive methods for detecting the freshness of closed-shell oysters. However, current technology lacks a non-destructive method and device for detecting the freshness of closed-shell oysters based on acoustic vibration. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a non-destructive testing method and apparatus for the freshness of closed oysters based on acoustic vibration. By performing feature extraction and machine learning on the sound and image information of closed oysters, a model that can be used for non-destructive testing of the freshness of closed oysters is constructed, solving the problem that sound information cannot be used to determine the freshness of oysters. This improves the speed and accuracy of oyster freshness analysis while reducing damage to the oysters used for testing.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A non-destructive method for detecting the freshness of closed oysters based on acoustic vibration includes:

[0007] Collect the acoustic vibration signals and images of the oyster shells to be tested;

[0008] Feature extraction is performed on the acoustic vibration signal to be detected and the image of the closed-shell oyster to be detected to obtain the audio features and morphological features to be detected.

[0009] The detection fusion feature is constructed using the audio feature to be detected and the morphological feature to be detected;

[0010] The fusion features to be detected are input into a preset non-destructive detection model for the freshness of closed-shell oysters to obtain the target freshness result;

[0011] The construction process of the non-destructive testing model for the freshness of closed oysters includes:

[0012] Collect the acoustic vibration signals, images, and freshness grade data of the pre-selected closed oysters;

[0013] Feature extraction is performed on the acoustic vibration signal to be analyzed and the image of the closed oyster to be analyzed to obtain the audio features and morphological features to be analyzed.

[0014] The fused features to be analyzed are constructed using the audio features to be analyzed and the morphological features to be analyzed.

[0015] Training sample data is constructed based on the fusion features to be analyzed and the freshness level data, and the training sample data is divided into a training set and a test set.

[0016] Based on machine learning algorithms, a two-layer chained multi-model ensemble initial model is constructed using the training set, and the initial model is validated using the test set to obtain a validated non-destructive detection model for the freshness of closed-shell oysters.

[0017] Preferably, the process of collecting the acoustic vibration signal and image of the oyster to be detected includes:

[0018] Issue detection commands to the preset control unit;

[0019] The placement status of the closed-shell oysters to be detected is determined using a preset infrared sensor;

[0020] When the oysters are placed in an oyster-containing state, a preset tapping rod is controlled to tap the closed-shell oysters to be tested.

[0021] The images of the closed-shell oyster to be detected and the acoustic vibration signals to be detected are collected using a pre-set industrial camera and microphone sensor, respectively.

[0022] Preferably, constructing a fusion feature to be analyzed using the audio features to be analyzed and the morphological features to be analyzed includes:

[0023] The acoustic vibration signal to be analyzed is sequentially subjected to signal pre-emphasis, framing and smoothing to obtain a temporal spatial signal;

[0024] Feature extraction is performed on the temporal spatial signal to obtain temporal spatial features; the temporal spatial features are matrices with the total number of rows equal to the total number of frames and 1 column.

[0025] The time-series spatial signal is mapped to a frequency-series spatial signal;

[0026] Feature extraction is performed on the frequency sequence spatial signal to obtain frequency sequence spatial features; the frequency sequence spatial features are matrices with the total number of rows equal to the total number of frames and 1 column.

[0027] Calculate the fused spectral characteristics based on the frequency sequence spatial signal;

[0028] The image of the closed oyster to be analyzed is sequentially converted to grayscale, binarized, and its contour is extracted to obtain a contour image;

[0029] Feature extraction is performed on the contour image to obtain external morphological features; the external morphological features are a matrix with 1 row and 1 column.

[0030] The temporal spatial features, frequency spatial features, fused spectral features, and external morphological features are fused to obtain the fused features to be analyzed.

[0031] Preferably, calculating the fused spectral features based on the frequency sequence spatial signal includes:

[0032] Extract the SC coefficients of the frequency sequence spatial signal to a predetermined order; the SC coefficients are a matrix with the total number of rows equal to the total number of frames and the predetermined order equal to the number of columns.

[0033] The first-order and second-order difference coefficients of the SC coefficients are obtained using the difference coefficient formula; both the first-order and second-order difference coefficients are matrices with the number of rows equal to the total number of frames and the number of columns equal to the preset order.

[0034] The SC coefficients, the first-order difference coefficients, and the second-order difference coefficients are summed in a 1:1:1 ratio to obtain the fused spectral feature; the fused spectral feature is a matrix with the number of rows equal to the total number of frames and the number of columns equal to the preset order.

[0035] Preferably, the temporal spatial features, the frequency spatial features, the fused spectral features, and the external morphological features are fused to obtain the fused features to be analyzed, including:

[0036] The temporal spatial features, the frequency spatial features, and the fused spectral features are horizontally merged to obtain an intermediate feature matrix;

[0037] The mean of the intermediate feature matrix is ​​calculated column by column and then horizontally merged with the external morphological feature to obtain the fusion feature to be analyzed; the fusion feature to be analyzed has 1 row and the number of columns is the sum of the number of columns of the temporal spatial feature, the frequency spatial feature, the external morphological feature and the fusion spectral feature.

[0038] Preferably, training sample data is constructed based on the fusion features to be analyzed and the freshness level data, and the training sample data is divided into a training set and a test set, including:

[0039] Construct a feature set matrix using the fusion features to be analyzed;

[0040] Construct a freshness set matrix using the freshness level data;

[0041] The feature set matrix and the freshness set matrix are merged horizontally to obtain the training sample data;

[0042] Based on the stratified sampling method, the training sample data is divided into the training set and the test set; the sample ratio of the training set and the test set is 2:1.

[0043] Preferably, based on a machine learning algorithm, a two-layer chained multi-model ensemble initial model is constructed using the training set, and the initial model is validated using the test set to obtain a validated non-destructive detection model for the freshness of the closed-shell oyster, including:

[0044] The training set is divided into K mutually exclusive subsets.

[0045] Repeat the selection of one subset as a validation sample K times, and use the remaining K-1 subsets as training samples to obtain K sets of sample combinations of validation samples and training samples;

[0046] The first-level classifier of the first layer of the machine learning algorithm is trained using K combinations of the aforementioned samples to obtain K trained first-level classifiers.

[0047] Using the trained sample and the trainer, classify and predict each feature vector in the validation sample and the test set respectively to obtain K validation prediction values ​​and K test prediction values;

[0048] The K verification prediction values ​​are stacked vertically and integrated with the freshness set matrix to update and replace the sample combination;

[0049] The K predicted test values ​​are summed and averaged, and then integrated with the freshness set matrix to update and replace the test set.

[0050] The updated sample combination is used to train the second-level classifier of the second layer of the machine learning algorithm to obtain the trained non-destructive detection model for the freshness of closed-shell oysters.

[0051] The non-destructive testing model for the freshness of oysters is evaluated using the updated test set. If the evaluation results meet the standards, the non-destructive testing model for the freshness of oysters is output.

[0052] Preferably, a non-destructive testing device for the freshness of closed-shell oysters based on acoustic vibration includes: a noise shielding chamber frame, sound-absorbing cotton, an industrial camera, and a knocking sound vibration signal detection device; the knocking sound vibration signal detection device includes: a testing platform, a microphone bracket, a microphone sensor, an infrared sensor, a control unit, an oyster placement platform, a knocking rod, a spring, a knocking platform, a stepper motor, and a cam.

[0053] The sound-absorbing cotton is disposed on the outside of the noise shielding chamber frame; the industrial camera is disposed on the upper inside of the noise shielding chamber frame; the impact vibration signal detection device is disposed inside the noise shielding chamber frame; the detection platform is fixed on the lower inside of the noise shielding chamber frame; the impact platform is disposed on one side of the detection platform; the microphone bracket is disposed at the corner of the detection platform; the microphone sensor and the infrared sensor are disposed on the microphone bracket; the oyster placement platform is disposed in the center of the detection platform; the stepper motor is disposed on the impact platform; the cam is connected to the stepper motor; the impact rod is connected to the cam; the spring is disposed between the impact platform and the impact rod; the control unit is disposed at the bottom of the detection platform.

[0054] The noise shielding chamber frame provides a basic framework for the industrial camera and the impact vibration signal detection device; the sound-absorbing cotton reduces noise inside the noise shielding chamber frame; the industrial camera acquires images of closed oysters; the detection platform provides a placement platform for the oyster placement table; the microphone bracket secures the microphone sensor and the infrared sensor; the microphone sensor acquires acoustic vibration signals; the infrared sensor detects the oyster placement status on the oyster placement table; the oyster placement table holds the closed oysters to be detected; the striking rod strikes the closed oysters to be detected; the spring absorbs the rebound energy of the striking rod; the stepper motor and the cam both control the striking rod to strike the closed oysters to be detected; the control unit receives preset detection commands from a computer, controls the microphone sensor, the striking rod, the stepper motor, the cam, and the industrial camera to operate according to the detection commands, and sends the acquired images of the closed oysters and the acoustic vibration signals to the computer; the oyster placement status includes: a state with oysters placed and a state without oysters placed.

[0055] Preferably, the formula for calculating the release height of the striking rod is: H = a × sinθ; where H is the release height, a is the length of the striking rod, and θ is the rotation angle of the cam.

[0056] The present invention discloses the following technical effects:

[0057] This invention provides a non-destructive testing method and apparatus for the freshness of closed-shell oysters based on acoustic vibration. By extracting features and using machine learning from the sound and image information of closed-shell oysters, it solves the problem of not being able to use sound information to determine the freshness of oysters and realizes non-destructive testing of oyster freshness. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a flowchart of a non-destructive testing method for the freshness of closed-shell oysters provided in an embodiment of the present invention.

[0060] Figure 2 This is a schematic diagram of the device structure provided in an embodiment of the present invention;

[0061] Figure 3 This is a schematic diagram of the structure of the impact sound vibration signal detection device provided in an embodiment of the present invention;

[0062] Figure 4 This is a schematic diagram of the external appearance of a closed-shell oyster provided in an embodiment of the present invention;

[0063] Figure 5 This is a schematic diagram of acoustic vibration signals provided in an embodiment of the present invention;

[0064] Figure 6 This is a frequency sequence space diagram provided for an embodiment of the present invention;

[0065] Figure 7 This is a schematic diagram of matrix addition provided in an embodiment of the present invention;

[0066] Figure 8 This is a schematic diagram illustrating the classification performance of the model provided in an embodiment of the present invention. Figure 8 (a) is a schematic diagram illustrating the classification performance of the KNN model. Figure 8 (b) is a schematic diagram illustrating the classification performance of the SVM model. Figure 8 (c) is a schematic diagram illustrating the classification performance of the Decision Tree model. Figure 8 (d) is a schematic diagram of the classification performance of the GBDT model. Figure 8 (e) is a schematic diagram illustrating the classification performance of the XGBoost model. Figure 8 (f) is a schematic diagram illustrating the classification performance of the Catboost model. Figure 8 (g) is a schematic diagram illustrating the classification performance of the Stacking model;

[0067] Explanation of reference numerals in the attached figures:

[0068] 1-Noise shielding chamber frame, 2-Audio vibration signal detection equipment, 21-Detection table, 211-Cam, 212-Connecting cable, 213-Computer, 22-Microphone bracket, 23-Microphone sensor, 24-Infrared sensor, 25-Control unit, 26-Oyster stand, 27-Audio stick, 28-Spring, 29-Audio platform, 210-Stepper motor, 3-Sound absorbing cotton, 4-Industrial camera. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] The purpose of this invention is to provide a non-destructive testing method and apparatus for the freshness of closed oysters based on acoustic vibration. By extracting features and using machine learning from the sound and image information of closed oysters, a model that can be used for non-destructive testing of the freshness of closed oysters is constructed, solving the problem that sound information cannot be used to determine the freshness of oysters. This improves the speed and accuracy of oyster freshness analysis while reducing damage to the oysters used for testing.

[0071] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0072] Figure 1 The flowchart for non-destructive testing of the freshness of closed oysters provided in this embodiment of the invention is as follows: Figure 1 As shown, this invention provides a non-destructive method for detecting the freshness of closed oysters based on acoustic vibration, comprising:

[0073] Collect the acoustic vibration signals and images of the oyster shells to be tested;

[0074] Feature extraction is performed on the acoustic vibration signal to be detected and the image of the closed-shell oyster to be detected to obtain the audio features and morphological features to be detected.

[0075] Construct a fused feature for detection using the audio features and morphological features to be detected;

[0076] The fusion features to be detected are input into the preset non-destructive detection model for the freshness of closed-shell oysters to obtain the target freshness result;

[0077] The process of constructing a non-destructive testing model for the freshness of closed oysters includes:

[0078] Collect the acoustic vibration signals, images, and freshness grade data of the pre-selected closed oysters;

[0079] Feature extraction is performed on the acoustic vibration signal to be analyzed and the image of the closed oyster to be analyzed to obtain the audio features and morphological features to be analyzed.

[0080] Construct the fused features to be analyzed using the audio features and morphological features to be analyzed;

[0081] Training sample data is constructed based on the fusion features to be analyzed and the freshness level data, and the training sample data is divided into training set and test set;

[0082] Based on machine learning algorithms, a two-layer chain multi-model ensemble initial model is constructed using the training set, and the initial model is validated using the test set to obtain a validated non-destructive detection model for the freshness of closed-shell oysters.

[0083] Further, the acoustic vibration signals and images of the target oysters are collected, including:

[0084] Issue a detection command to the preset control unit 25;

[0085] The placement status of the closed-shell oyster to be detected is determined using a preset infrared sensor 24.

[0086] When the oysters are in the oyster-containing state, the preset tapping rod 27 is controlled to tap the closed-shell oyster to be tested.

[0087] The industrial camera 4 and microphone sensor 23 are used to collect images of the closed-shell oyster to be detected and acoustic vibration signals to be detected, respectively.

[0088] Specifically, the fused features to be analyzed are constructed using the audio features and morphological features to be analyzed, including:

[0089] The acoustic vibration signal to be analyzed is sequentially pre-emphasized, framed, and smoothed to obtain the temporal spatial signal.

[0090] Feature extraction is performed on the temporal spatial signal to obtain temporal spatial features; the temporal spatial features are matrices with the total number of rows equal to the total number of frames and 1 column.

[0091] Mapping time-series spatial signals to frequency-series spatial signals;

[0092] Feature extraction is performed on the frequency sequence spatial signal to obtain frequency sequence spatial features; the frequency sequence spatial features are matrices with the total number of rows equal to the total number of frames and 1 column.

[0093] Calculate the fused spectral characteristics based on the frequency sequence spatial signal;

[0094] The images of the oyster shell to be analyzed are sequentially converted to grayscale, binarized, and have their contours extracted to obtain contour images;

[0095] Feature extraction is performed on the contour image to obtain external morphological features; the external morphological features are matrices with 1 row and 1 column.

[0096] The temporal spatial features, frequency spatial features, fused spectral features, and external morphological features are fused to obtain the fused features to be analyzed.

[0097] Preferably, calculating the fused spectral characteristics based on the frequency sequence spatial signal includes:

[0098] Extract the SC coefficients of the frequency sequence spatial signal of a set order; the SC coefficients are a matrix with the number of rows equal to the total number of frames and the number of columns equal to the preset order;

[0099] The first-order and second-order difference coefficients of the SC coefficients are obtained using the difference coefficient formula; both the first-order and second-order difference coefficients are matrices with the number of rows equal to the total number of frames and the number of columns equal to the preset order.

[0100] The SC coefficients, first-order difference coefficients, and second-order difference coefficients are summed in a 1:1:1 ratio to obtain the fused spectral features; the fused spectral features are matrices with the number of rows equal to the total number of frames and the number of columns equal to the preset order.

[0101] Specifically, temporal spatial features, frequency spatial features, fused spectral features, and external morphological features are fused to obtain the fused features to be analyzed, including:

[0102] The temporal spatial features, frequency spatial features, and fused spectral features are horizontally merged to obtain an intermediate feature matrix;

[0103] The mean of the intermediate feature matrix is ​​calculated column by column and then horizontally merged with the external morphological features to obtain the fusion features to be analyzed. The fusion features to be analyzed are the sum of the number of columns of time-series spatial features, frequency-series spatial features, external morphological features and fusion spectral features, with 1 row.

[0104] Optionally, training sample data is constructed based on the fusion features to be analyzed and the freshness level data, and the training sample data is divided into a training set and a test set, including:

[0105] Construct a feature set matrix using the features to be analyzed and fused;

[0106] Construct a freshness set matrix using freshness grade data;

[0107] The feature set matrix and the freshness set matrix are merged horizontally to obtain the training sample data;

[0108] Based on the stratified sampling method, the training sample data is divided into a training set and a test set; the ratio of the training set to the test set is 2:1.

[0109] Specifically, based on machine learning algorithms, a two-layer chained multi-model ensemble initial model is constructed using the training set, and the initial model is validated using the test set to obtain a validated non-destructive detection model for the freshness of closed-shell oysters, including:

[0110] The training set is divided into K mutually exclusive subsets.

[0111] Repeat the process K times to select a subset as the validation sample and use the remaining K-1 subsets as the training sample to obtain K sets of validation and training sample combinations.

[0112] The first-level classifier of the first layer of the machine learning algorithm is trained by using K sets of sample combinations to obtain K trained first-level classifiers;

[0113] Using the trained data and the trainer, classify and predict each feature vector in the validation sample and test set respectively, to obtain K validation prediction values ​​and K test prediction values;

[0114] The K validation predictions are stacked vertically and integrated with the freshness set matrix to update and replace the sample combination;

[0115] The K test predictions are summed and averaged, then integrated with the freshness set matrix to update and replace the test set.

[0116] The updated sample combination is used to train the second-level classifier of the machine learning algorithm to obtain the trained non-destructive detection model for the freshness of closed-shell oysters.

[0117] The non-destructive testing model for the freshness of oyster shells is evaluated using the updated test set. If the evaluation results meet the standards, the non-destructive testing model for the freshness of oyster shells is output.

[0118] refer to Figure 2 and Figure 3 A non-destructive testing device for the freshness of closed-shell oysters based on acoustic vibration includes: a noise shielding chamber frame 1, sound-absorbing cotton 3, an industrial camera 4, and a knocking sound vibration signal detection device 2; the knocking sound vibration signal detection device 2 includes: a testing platform 21, a microphone bracket 22, a microphone sensor 23, an infrared sensor 24, a control unit 25, an oyster placement platform 26, a knocking rod 27, a spring 28, a knocking platform 29, a stepper motor 210, and a cam 211;

[0119] Sound-absorbing cotton 3 is placed on the outside of the noise shielding chamber frame 1; industrial camera 4 is placed on the upper inside of the noise shielding chamber frame 1; impact vibration signal detection device 2 is placed inside the noise shielding chamber frame 1; detection platform 21 is fixed on the lower inside of the noise shielding chamber frame 1; impact platform 29 is placed on one side of detection platform 21; microphone bracket 22 is placed at the corner of detection platform 21; microphone sensor 23 and infrared sensor 24 are placed on microphone bracket 22; oyster placement platform 26 is placed in the center of detection platform 21; stepper motor 210 is placed on impact platform 29; cam 211 is connected to stepper motor 210; impact rod 27 is connected to cam 211; spring 28 is placed between impact platform 29 and impact rod 27; control unit 25 is placed at the bottom of detection platform 21.

[0120] The noise shielding chamber frame 1 provides a basic framework for the industrial camera 4 and the impact vibration signal detection device 2; the sound-absorbing cotton 3 reduces noise inside the noise shielding chamber frame 1; the industrial camera 4 is used to acquire images of closed oysters; the detection table 21 provides a platform for the oyster placement table 26; the microphone bracket 22 is used to fix the microphone sensor 23 and the infrared sensor 24; the microphone sensor 23 is used to acquire acoustic vibration signals; the infrared sensor 24 is used to detect the oyster placement status on the oyster placement table 26; the oyster placement table 26 is used to place the closed oysters to be tested; and the striking rod 27 is used for striking. The test involves two components: a closed oyster to be tested; a spring 28 to absorb the rebound energy of the striking rod 27; a stepper motor 210 and a cam 211 to control the striking rod 27 to strike the closed oyster; a control unit 25 to receive a preset detection command from a computer 213, and to control the microphone sensor 23, the striking rod 27, the stepper motor 210, the cam 211, and the industrial camera 4 to work according to the detection command, and to send the collected images of the closed oyster and acoustic vibration signals to the computer 213 through a connecting line 212; the oyster placement status includes: a state with oysters placed and a state without oysters placed.

[0121] Specifically, the formula for calculating the release height of the striking rod is: H = a × sinθ; where H is the release height, a is the length of the striking rod 27, and θ is the rotation angle of the cam 211.

[0122] Specifically, to train a non-destructive testing model for the freshness of oysters, this embodiment selected 150 oysters of the same size grade (weight 210±25g) from the same oyster farm, excluding inferior oysters such as banana-shaped, broken-shell, or irregularly shaped oysters, and transported them via refrigerated road transport. Upon receipt, the oysters were washed with running drinking water to remove dirt and debris, drained, and stored in a 4°C refrigerator. The tested oysters were stored in the 4°C refrigerator for 0, 1, 2, 3, 4, 6, 8, 10, 12, and 14 days. Each day, 15 oysters were removed, and each oyster sample was repeatedly tapped twice using an oyster tapping vibration acoustic signal acquisition system to obtain 300 acoustic vibration signals (150 tested oysters × 2 times). Simultaneously, the freshness index of the tested oysters was measured.

[0123] Furthermore, the noise generated by the operation of various mechanical equipment and the normal communication of personnel in the oyster processing workshop is unavoidable, which will affect the accuracy of the system. The noise shielding chamber in the non-destructive testing device for the freshness of closed oysters can absorb the uncontrollable noise from the oyster processing workshop and the reflected sound waves from knocking vibrations when the oyster activity non-destructive testing system is working.

[0124] Preferably, in the prior art, noise shielding rooms mainly use glass fiber and polyester fiber as sound-absorbing cotton 3 materials to reduce the impact of environmental noise. Glass fiber has good resistance to chemical corrosion; however, it is brittle and easily broken. Broken glass fibers can puncture human skin and cause lung damage if inhaled, making it unsuitable for the food production field. On the other hand, sound-absorbing cotton 3 materials made of polyester fiber have good antibacterial properties and are easy to process, but they have poor mechanical strength and heat resistance, which is not conducive to long-term and safe use by enterprises. The sound-absorbing cotton 3 material used in this embodiment is a composite material sound-absorbing cotton 3 made by mixing biodegradable polyester fiber material with environmentally friendly thermosetting resin and then heating and pressing it. While retaining the original advantages, it overcomes the problems of poor mechanical strength and heat resistance of traditional polyester fiber sound-absorbing cotton 3 materials, and its moisture resistance and durability are enhanced.

[0125] The self-made noise shielding room includes a cabinet-type device with sound insulation and absorption, which is 600mm long and wide and 1200mm high. One side is open, while the other three sides and the top are completely covered with 5cm thick composite material sound-absorbing cotton to reduce noise interference with sound wave acquisition.

[0126] The industrial camera 4 used in this embodiment is a Hikvision 6-megapixel color TBS060-10GC, which is installed at the top center of the noise shielding room, perpendicular to the oyster stand 26; its shutter speed is 1 / 20s, aperture is F5.6, and zoom is 35mm.

[0127] The testing platform 21 in this embodiment consists of an upper platform, a base, and four supports. The base is fixedly connected to the lower surface of the noise shielding chamber with bolts.

[0128] The excitation module in this embodiment consists of a striking rod 27, a spring 28, a stepper motor 210, a cam 211, and a striking platform 29. The oyster placement platform 26 is located at the center of the upper surface of the detection platform 21. The striking rod 27 is located at the centerline of the oyster placement platform 26 and forms a hinge structure with the striking platform 29. The stepper motor 210 is fixed on the striking platform 29 and forms a drive mechanism with the cam 211. The cam 211 contacts the rear end of the striking rod 27. The stepper motor 210 drives the cam 211 to rotate, realizing the striking action of the striking rod 27. The spring 28 is used to buffer the energy rebounded after the striking rod 27 strikes the oyster shell, ensuring that only one excitation is generated. The striking rod 27 is made of solid 40Cr steel, with a length of 15cm and a diameter of 2cm. Due to the high elastic modulus of 40Cr steel, the contact time between the striking rod 27 and the oyster is short, ensuring the consistency of the sound wave resonance frequency.

[0129] The acoustic vibration signal detection and acquisition module in this embodiment consists of a microphone sensor 23, a microphone bracket 22, and an infrared sensor 24. The microphone sensor 23 is an INMP441 omnidirectional MEMS microphone manufactured by InvenSense, USA, with an operating frequency range of 60Hz to 15kHz, a high sensitivity of -26dBFS, and a high signal-to-noise ratio of 61dBA. The infrared sensor 24 is an E18-D80NK model. Both the microphone sensor 23 and the infrared sensor 24 are bolted to the microphone bracket 22 and connected to the control unit 25 via a USB cable. The microphone bracket 22 is fixed at the upper left corner of the detection platform 21, and the infrared sensor 24 and the microphone sensor 23 point towards the center of the oyster placement platform 26.

[0130] The control unit 25 in this embodiment is an ESP32-S3-WROOM microcontroller manufactured by Espressif Systems. It is designed and developed using Arduino software and is connected to the sound excitation module, the sound vibration signal detection and acquisition module, and the industrial camera 4. It includes a main program and six subroutines: an initialization program, an infrared detection program, a microphone driver program, a motor driver program, an image acquisition program, and an interrupt program. It is located directly below the detection platform 21. The control unit 25 realizes the tapping of the closed oyster and the acquisition and transmission of sound vibration signals by calling the subroutines.

[0131] Furthermore, the process of acquiring sound and image information:

[0132] 1) The computer 213 issues a detection command to the control unit 25. The control unit 25 executes the main program, which first runs the initialization program to initialize the system.

[0133] 2) After the system initialization is completed, the main program will run the infrared detection program in a loop. The infrared detection program calls the infrared sensor 24 to detect whether there are oysters on the oyster placement platform 26. When the main program detects that the infrared light is blocked by the oysters through the infrared detection program, it will send feedback to the main program.

[0134] 3) The main program invokes the motor driver program to control the stepper motor 210 to drive the cam 211 to rotate. By setting the rotation angle of the cam 211, the release height of the striking rod 27 is changed, as shown in the following formula:

[0135] H = a × sinθ

[0136] Where H is the release height, a is the length of the striking rod 27, and θ is the rotation angle of the cam 211.

[0137] Each time cam 211 rotates from the horizontal position to a 60-degree angle, and then to a preset angle, the striking rod 27 is released. The striking rod 27 then undergoes free fall, ensuring the same release height H. This ensures the same striking force is obtained, thus striking the peak of the oyster shell (the highest point of the protrusion on the shell, such as...). Figure 4 (As shown) Action: By tapping the same area, the oyster shell is excited to produce free vibration. Simultaneously, the microphone driver calls the microphone sensor 23 to collect an acoustic vibration signal for 1 second at a set sampling frequency of 22.05kHz. Figure 5 As shown; the sampling duration can be adjusted according to the release height H of the striking rod 27. When the release height increases or decreases, the sampling duration is increased or decreased synchronously, which can ensure the integrity of the acquired signal; increasing the sampling frequency will increase the data size. Therefore, the upper limit of the sampling frequency is affected by the data transmission rate between the selected microcontroller and the computer 213 and the storage capacity of the computer 213.

[0138] 4) After the acoustic vibration signal acquisition is completed, the main program takes an image of the closed-shell oyster to be detected through the image acquisition program.

[0139] 5) After image acquisition is completed, the interrupt program converts the acoustic vibration signal data and image into digital signals and sends them in CSV and JPEG formats respectively via USB transmission line and stores them in computer 213 to complete the acquisition of the knocking acoustic vibration signal and image of the oyster to be detected.

[0140] Furthermore, the sound information processing process:

[0141] 1) Acquire the acoustic vibration signal of the oysters to be tested by tapping, and simultaneously obtain the freshness grade of the oysters. In this embodiment, the ATP and its metabolite content of the oysters after tapping are analyzed by high performance liquid chromatography (HPLC) to obtain the freshness K value, and the freshness grade of the oysters is divided into 3 categories according to the K value: K value less than 20% is defined as very fresh (Level 0), K value between 20% and 60% is defined as moderately fresh (Level 1), and K value greater than 60% is defined as spoiled (Level 2). The K value is a biochemical quality indicator that reflects the initial freshness change of aquatic products and is related to quality and flavor. It can reflect the degradation level of ATP in oysters. After analyzing the ATP and its metabolite content by high performance liquid chromatography (HPLC), the freshness K value of 150 tested oysters is calculated. The calculation formula is as follows:

[0142]

[0143] In the formula, CATP, CADP, CAMP, CIMP, CHxR, and CHx represent the contents of adenosine triphosphate, adenosine diphosphate, adenosine monophosphate, inosine monophosphate, hypoxanthine nucleoside, and hypoxanthine in the tested oyster shell, respectively (unit: μmol / g).

[0144] After grading all oyster samples by freshness, 50 acoustic vibration signals were selected from each of the 300 freshness grades, for a total of 150 acoustic vibration signals, as shown in Table 1:

[0145] Table 1

[0146] K≤20 50 Very fresh (Level 0) 20＜K≤60 50 Medium freshness level (Level 1) K＞60 50 Corruption Level (Level 2)

[0147] 2) Preprocess the acquired acoustic vibration signals from the tapping of oyster shells, including signal pre-emphasis and sampling at a set frequency f. s The sampling point number N is used for framing and smoothing to obtain the temporal spatial signal a(n), and the frame length R and total number of frames P of the temporal spatial signal are obtained at the same time; pre-emphasis is used to highlight the energy of the high-frequency part and reduce the attenuation of the high-frequency part, and the calculation formula is as follows:

[0148] y(t) = x(t) - εx(t)

[0149] In the formula, y(t) is the pre-emphasized speech amplitude value at time t, x(t) is the speech amplitude value at time t, and ε is the correction coefficient, which takes the value of 0.8.

[0150] Based on a sampling frequency of 22.05kHz f s The frame is divided into frames using the number of sampling points N (256), and the frame length R for each frame is calculated. The frame shift G is half of the frame length. The frame length calculation formula is as follows:

[0151]

[0152] In this embodiment, R is 11.6ms and G is 5.8ms.

[0153] The formula for calculating the total number of frames P is as follows:

[0154]

[0155] Where T is the duration of the vibratory acoustic signal, and in this embodiment, T is 1000s and P is 171.

[0156] To ensure a smooth transition at the truncation points of each frame of the vibratory acoustic signal, thereby reducing spectral leakage, a smoothing function needs to be multiplied by each frame of the segmented signal to increase the continuity between frames. The smoothing function is as follows:

[0157]

[0158] In the formula, w(n) is the value of the smoothing function, representing the smoothing function coefficient at position n, where n is the index of the current sample point, ranging from 0 to N-1. N is the number of sampling points.

[0159] The vibrational sound waves generated by tapping interact with the internal structure of the oyster shell, thus carrying different internal response signals. As the freshness of the oyster changes dynamically during storage, its textural properties (such as hardness and chewiness) typically alter, leading to changes in the internal response signals. Tapping the shell of a less fresh oyster produces a hollow, low-pitched sound. Therefore, in the temporal space, its zero-crossing rate and root-mean-square energy are more significant, while in the frequency space, its spectral centroid, bandwidth, spectral contrast, spectral flatness, and spectral attenuation are more pronounced. Therefore, fused spectral characteristics (FSC) can be extracted in the temporal-frequency space to reflect the dynamic changes of the acoustic vibration signal.

[0160] 3) Select A eigenvalues ​​from the temporal spatial signal of the preprocessed acoustic vibration signal of the knocked oyster. Each eigenvalue is a (P, 1) matrix. Extract two eigenvalues—zero-crossing rate and root-mean-square energy—from the temporal spatial signal as temporal spatial features. Specifically, the zero-crossing rate refers to the number of times the signal crosses zero in each frame, as shown in the following formula:

[0161]

[0162] In the formula, Z CR The zero-crossing rate is calculated as a matrix of size (171, 1); x(m) and x(m-1) are the positive and negative values ​​of the signal amplitudes of two adjacent sampling points; sgn[] is the sign function, which returns an integer variable indicating the sign of the parameter.

[0163]

[0164] In the formula, x(n) represents the sign of the amplitude of a certain frame signal.

[0165] The root mean square energy (RMS) refers to the square root of the average of the squared amplitude values ​​at each point in frame t. Its calculation formula is as follows:

[0166]

[0167] In the formula, s(n) is the amplitude corresponding to the nth sampling point in a frame of signal, n = 1, 2, 3, ..., N; RMS is a matrix of size (171, 1).

[0168] 4) Convert the input acoustic vibration signal time-series spatial signal a(n) from time-series spatial signal to frequency-series spatial signal X(t,f), the result is as follows. Figure 6As shown, the calculation formula is as follows:

[0169]

[0170] Where X(t,f) is the frequency sequence space transformation result at time t and frequency f.

[0171] 5) Select B frequency-sequence spatial features from the frequency-sequence spatial signal X(k). Each frequency-sequence spatial feature is a matrix of size (P, 1). The frequency-sequence spatial features include five characteristic values: spectral centroid, bandwidth, spectral contrast, spectral flatness, and spectral attenuation. Specifically, the spectral centroid C... t It is one of the important physical parameters describing timbre attributes. It is the centroid of frequency components, and the calculation formula is as follows:

[0172]

[0173] In the formula, F(k) represents the signal frequency at n, and X(k,i) is the frequency amplitude corresponding to the k-th frequency band of the i-th frame of the signal. Finally, C t It is a matrix of size (171, 1).

[0174] 6) Frequency Sequence Spatial Spectrum Contrast (SC) is used to measure the energy difference between different frequency bands in the spectrum. The spectrum is divided into K frequency bands, and the frequency range of each band is [f k ,f k+1 ] Calculate the energy for the frequency components within each frequency band k:

[0175]

[0176] Within each frequency band, the α% high energy points are identified as peak values, and the β% low energy points are identified as valley values. Typically, α and β are set to small values; in this embodiment, they are set to 10%.

[0177] Peak energy: P k (t)=mean(E k,topα％ (t))

[0178] Valley energy: V k (t)=mean(E k,bottomβ％ (t))

[0179] Spectral contrast can be calculated from the difference between peak and trough values:

[0180] SC k (t) = log 10 (P k (t))-log 10 (V k (t))

[0181] In this embodiment, SC is a matrix of size (171, 1).

[0182] 7) Spectral flatness describes the degree of deviation of the energy distribution of the spectrum relative to the center frequency within that range. Its calculation formula is as follows:

[0183]

[0184] In the formula, m is the upper boundary of the spectrum, and m takes the value f s / 2; Flatness is a matrix of size (171, 1).

[0185] 8) Spectral attenuation (Rollof) describes the sum of amplitude values ​​from zero frequency to a certain frequency point, which is 85% of the sum of the entire spectrum. Its calculation formula is as follows:

[0186]

[0187] In the formula, In this embodiment, the spectral energy reduction coefficient is... The value is 0.85; Rollof is a matrix of size (171, 1).

[0188] 9) An improved Mel-frequency cepstral coefficient (FSC) method was extracted from the time-series frequency space. SSC coefficients of a predetermined order L were obtained from the frequency-series signal X(k). The SSC coefficients are a matrix of size (P, L), and the order L is preset to 20. The specific process for obtaining the SSC coefficients is as follows: using filter bank H... m (k) After filtering the spectrum X(k), the mel spectrum is obtained, and the frequency response H of the filter bank is obtained. m The formula for calculating (k) is:

[0189]

[0190] In the formula, f(m) represents the center frequency of the m-th triangular filter. There are M filters in total, where M is 20, and 1 ≤ m ≤ M. The formula for calculating the center frequency f(m) is:

[0191]

[0192] In the formula, B(f) = 1125ln(f / 700+1), f h f represents the highest frequency of the filter. l The lowest frequency of the filter is represented by f, and the highest frequency is the sampling frequency f. h Half of that, with the lowest frequency set at 300Hz.

[0193] The formula for calculating the Mel filter output Mel(m) is:

[0194]

[0195] Performing a logarithmic operation on Mel(m) yields:

[0196]

[0197] After performing a discrete cosine transform on S(m), the SC coefficients c(n) are obtained, and the calculation formula is as follows:

[0198]

[0199] In the formula, L represents the order of the SC coefficients. In this embodiment, L is 20, so the SC coefficients are a matrix of size (171, 20).

[0200] 10) Obtain the first-order and second-order difference coefficients (αSC) of the L-dimensional SC coefficients using the difference coefficient formula. Both αSC and αSC are matrices of size (P, L). Specifically, the first-order difference coefficients (αSC) of SC are obtained using the difference coefficient formula on c(n). The first-order difference coefficients reflect the dynamic changes of the audio signal between frames, and their calculation formula is as follows:

[0201]

[0202] Where, d m and C m Let Q represent the m-th first-order difference and the t-th cepstral coefficient; Q and K are the order of the cepstral coefficient and the time difference of the first derivative, with K set to 1.

[0203] The second-order difference coefficients (βSC) are obtained by applying the difference coefficient formula to αSC. The second-order difference can represent the dynamic relationship between three adjacent frames after the vibratory acoustic signal is framed. The calculation formula is as follows:

[0204]

[0205] z t d represents the t-th second difference; t Q is the t-th first-order difference; t The order of the first-order difference coefficients is represented by K; K is the time difference of the second derivative; K is set to 2. In this embodiment, the first-order and second-order differences are both matrices of size (171, 20).

[0206] 11) Sum the eigenma matrices of L-dimensional SC, L-dimensional αSC, and L-dimensional βSC in a 1:1:1 ratio to obtain FSC, where FSC is a matrix of size (171, 60). The calculation principle is as follows: Figure 7 As shown, the calculation formula is as follows:

[0207] FSC=SC+αSC+βSC

[0208] Specifically, the image information processing procedure:

[0209] 1) The acquired images of the knocked-out oysters are converted to grayscale, binarized, and have their contours extracted to obtain a contour image. Specifically, the RGB images of the knocked-out oysters are converted to grayscale images. Each pixel in the RGB image is represented by the values ​​of the red, green, and blue channels. For each pixel, the value is (R, G, B). Based on its R, G, B values ​​and corresponding weights, the grayscale value is calculated using the following formula:

[0210] Gray=0.279×R+0.568×G+0.128×B

[0211] The calculated value of each pixel in the grayscale image is a single-channel (Gray), and the value of each pixel represents the brightness of that point.

[0212] 2) Convert the grayscale image into a binary image containing only black and white. Set a threshold YR, compare the grayscale value with the threshold, set the pixels greater than the threshold to white (255), and set the pixels less than or equal to the threshold to black (0). The conversion formula is as follows:

[0213]

[0214] In this embodiment, the value of YR is 127.

[0215] If a pixel in the image is black, and all eight pixels adjacent to it are also black, then the target pixel is deleted and converted to white. This process is repeated for each pixel after removing the boundary to obtain a new contour image.

[0216] 3) Obtain C external morphological features from the contour image, each an array of size (1, 1). Extract three feature values ​​from the contour image: roughness, roundness, and elongation, as frequency-sequence spatial features. Specifically, roughness is used to measure the complexity of the oyster contour, and its calculation formula is as follows:

[0217]

[0218] Where: R is roughness, the larger the value, the higher the profile complexity; p is the perimeter; b is the convex perimeter.

[0219] Roundness C is used to assess the degree of deviation of oysters from their roundness, and its calculation formula is as follows:

[0220]

[0221] Where: s is the area of ​​the smallest circumcircle.

[0222] Elongation is used to measure whether an oyster's outline is closer to a rectangle or a square, and its calculation formula is as follows:

[0223]

[0224] Where: E is the elongation rate, the elongation rate of oysters that tend to be round is about 1 or slightly greater than 1, and the elongation rate of oysters that tend to be square is greater than 1; j is the major axis (the straight line with the longest distance between any two points in the contour), and m is the minor axis (the width of the minimum outer matrix).

[0225] 4) Fuse A temporal spatial features, B frequency spatial features, FSC features and C external morphological features to obtain fused feature F.

[0226] Specifically, the data preprocessing process is as follows:

[0227] 1) A single feature cannot comprehensively and comprehensively describe the information contained in a signal. Therefore, it is necessary to extract and fuse the temporal, frequency, and temporal-frequency spatial features of the acoustic vibration signal of the closed-shell oyster. Two temporal spatial features—zero-crossing rate and root-mean-square energy—and five frequency spatial features—spectral centroid, bandwidth, spectral contrast, spectral flatness, and spectral attenuation—are all matrices of size (171, 1), while the FSC feature is a matrix of size (171, 20). These eight matrices are horizontally concatenated to form a matrix of size (171, 27), and the average of each column is taken to obtain a matrix of size (1, 27). This matrix is ​​then horizontally merged with three external morphological feature matrices of size (1, 1) to obtain the fused feature F, a matrix of size (1, 30). This fused feature F enhances and corrects the acoustic vibration features through external morphological features, reduces the feature dimensionality, and lowers computational complexity.

[0228] 2) A training sample dataset was constructed based on the fusion features and the freshness level of the oyster shell, and divided into a training set and a test set. The training sample dataset D was constructed based on 150 acoustic vibration signal samples of tapped oyster shells. The feature of each tested oyster shell can be represented as X. i ={x1,x2,x3,…x S}, 1≤S≤27, where S is the sum of temporal spatial features, frequency spatial features, and external morphological features, and X i Let F be the fusion feature of the i-th sample, and let y be the freshness level corresponding to each tested closed oyster. j j = 1, 2, ..., 150, where y j The three freshness levels are derived from the K-value of the oyster freshness index obtained from actual measurements; therefore, the dataset D of the measured closed oysters can be represented as a matrix (X, Y), where X = (X1, X2, ..., X...). 150 ),Y=(y1,y2,…y150 );

[0229] 3) A stratified sampling method was used to divide two-thirds of the samples in the tested oyster dataset D into a training set DT (a total of 100 oyster samples), while the remaining one-third of the oyster samples were used as a test set DS (a total of 50 oyster samples). This ensured that the training and test sets contained a balanced number of oyster samples of the three freshness levels, which helps ensure that the model has good generalization ability. A two-layer chained multi-model ensemble non-destructive detection model for the freshness of oysters was built based on the training set, and the model was output after validation on the test set.

[0230] Furthermore, the model training process includes:

[0231] 1) Divide the training set DT into equal subsets, with no overlap between subsets, and set K to 3. Repeat this process K times, selecting one sample from each subset as the validation set, and selecting another K-1 subsets as the training set, resulting in K distinct combinations of training and validation sets {(D T,1 D V,1 ),(D T,2 D V,2 ),…,(D T,Q D V,Q )}, where 1≤Q≤K.

[0232] 2) In the first layer, R first-level classifiers are set, and the same process is repeated for each first-level classifier. In this embodiment, five machine learning models, KNN, SVM, Decision Tree, GBDT, and XGBoost, are set as first-level classifiers, so R is 5. First, the training set (D... T,1 D T,2 ,…,D T,Q The first-level classifiers were trained, resulting in K trained first-level classifiers R. 1,1 ,R 1,2 …,R 1,Q Using the trained K first-level classifiers on the validation set (D) V,1 D V,2 ,…,D V,Q ) and each sample (x) in the test set DS i ,y j ) eigenvector X i Perform classification predictions and obtain K classification prediction results. The validation set yields predicted values ​​p1, p2, ..., p. K The predicted values ​​obtained from the test set are B1, B2, ..., B K The predictions from the K validation sets are stacked vertically to form a new feature F. i F iThe number of samples is the same as the training set DT. As a new feature I i I i The number of samples is the same as the test set DS. R first-level classifiers ultimately yield two feature matrices F. i ={F1,F2,…,F R} and I i ={I1,I2,…,I R}. F i and the training set DT's y j Integration as a new training set D T2 ={F1,F2,…,F R ,y j}, and at the same time I i and the test set DS's y j Integrate as a new test set DS2 = {I1, I2, ..., I R ,y j}

[0233] 3) D T2 The training set of Catboost, a secondary classifier for the second layer, was used to train the Modeloyster model, which is a non-destructive detection model for the freshness of closed oysters.

[0234] 4) The freshness grading performance of the Modeloyster model is evaluated using the DS2 test set. Once the freshness grading performance meets the requirements, the Modeloyster non-destructive freshness detection model for closed oysters is output. This embodiment uses accuracy, precision, recall, and the harmonic mean of precision and recall, F1-Score, to evaluate the performance of the Modeloyster non-destructive freshness detection model for closed oysters. The calculation formula is as follows:

[0235]

[0236]

[0237]

[0238]

[0239] Wherein, TP represents the number of samples that were actually positive but were predicted as positive, FP represents the number of samples that were actually negative but were predicted as positive, FN represents the number of samples that were actually positive but were predicted as negative, and TN represents the number of samples that were actually negative.

[0240] Furthermore, since this is a multi-class classification problem, accuracy measures the overall prediction performance of all samples, while Precision, Recall, and F1-Score need to be calculated separately for each class. To evaluate the overall performance of the classification model across all classes, the Macro-average method is used. This method averages the evaluation metrics (Precision / Recall / F1-Score) for each class, assigning equal weight to all classes. The calculation formula is as follows:

[0241]

[0242]

[0243]

[0244] Where k represents the number of oyster freshness grades.

[0245] The performance of the Modeloyster non-destructive testing model for the freshness of oysters was compared in identifying and classifying the freshness of oysters. The results are shown in Table 2. The overall accuracy, precision, recall, and F1-score of the Modeloyster model were 98.1%, 98%, and 98%, respectively, all of which were higher than the non-destructive testing models for the freshness of oysters constructed by KNN, SVM, Decision Tree, GBDT, XGBoost, and Catboost. This indicates that the obtained non-destructive testing model for the freshness of oysters is suitable for classifying the freshness of oysters and shows a significant advantage.

[0246] Table 2

[0247]

[0248] Furthermore, the two-layer chain multi-model ensemble model Modeloyster, a non-destructive detection model for the freshness of oyster shells, and six other existing models were compared in their classification performance for three different freshness levels of oyster shells. Figure 8 (a) to Figure 8As shown in (g). Specifically, apart from the KNN-based non-destructive detection model for the freshness of oysters, the other two models can accurately distinguish rotten oysters, effectively preventing food safety issues caused by rotten oysters entering the market. However, the Modeloyster model achieves precision, recall, and F1-score of 96.3%, 98.0%, and 97.0% for predicting very fresh oysters, and 98.0%, 96.0%, and 97.0% for predicting moderately fresh oysters, respectively. These figures are all higher than the non-destructive detection models for the freshness of oysters built using KNN, SVM, Decision Tree, GBDT, XGBoost, and Catboost. Therefore, this model can accurately predict both very fresh and moderately fresh oysters.

[0249] The beneficial effects of this invention are as follows:

[0250] This invention utilizes feature extraction and machine learning on the sound and image information of oysters to construct a model for non-destructive testing of oyster freshness. This solves the problem of not being able to determine oyster freshness using sound information, improving the speed and accuracy of oyster freshness analysis while minimizing damage to the oysters used for testing.

[0251] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0252] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A non-destructive method for determining the freshness of closed shell oysters based on acoustic vibrations, characterized in that, include: Collect the acoustic vibration signals and images of the oyster shells to be tested; Feature extraction is performed on the acoustic vibration signal to be detected and the image of the closed-shell oyster to be detected to obtain the audio features and morphological features to be detected. The detection fusion feature is constructed using the audio feature to be detected and the morphological feature to be detected; The fusion features to be detected are input into a preset non-destructive detection model for the freshness of closed-shell oysters to obtain the target freshness result; The construction process of the non-destructive testing model for the freshness of closed oysters includes: Collect the acoustic vibration signals, images, and freshness grade data of the pre-selected closed oysters; Feature extraction is performed on the acoustic vibration signal to be analyzed and the image of the closed oyster to be analyzed to obtain the audio features and morphological features to be analyzed. The fused features to be analyzed are constructed using the audio features to be analyzed and the morphological features to be analyzed. Training sample data is constructed based on the fusion features to be analyzed and the freshness level data, and the training sample data is divided into a training set and a test set. Based on machine learning algorithms, a two-layer chain multi-model ensemble initial model is constructed using the training set, and the initial model is validated using the test set to obtain a validated non-destructive detection model for the freshness of closed-shell oysters. Constructing a fusion feature to be analyzed using the audio features to be analyzed and the morphological features to be analyzed includes: The acoustic vibration signal to be analyzed is sequentially subjected to signal pre-emphasis, framing and smoothing to obtain a temporal spatial signal; Feature extraction is performed on the temporal spatial signal to obtain temporal spatial features; the temporal spatial features are matrices with the total number of rows equal to the total number of frames and 1 column. The time-series spatial signal is mapped to a frequency-series spatial signal; Feature extraction is performed on the frequency sequence spatial signal to obtain frequency sequence spatial features; the frequency sequence spatial features are matrices with the total number of rows equal to the total number of frames and 1 column. Calculate the fused spectral characteristics based on the frequency sequence spatial signal; The image of the closed oyster to be analyzed is sequentially converted to grayscale, binarized, and its contour is extracted to obtain a contour image; Feature extraction is performed on the contour image to obtain external morphological features; the external morphological features are a matrix with 1 row and 1 column. The temporal spatial features, frequency spatial features, fused spectral features, and external morphological features are fused to obtain the fused features to be analyzed.

2. The method according to claim 1, wherein, Collect the acoustic vibration signals and images of the target oyster, including: Issue detection commands to the preset control unit; The placement status of the closed-shell oysters to be detected is determined using a preset infrared sensor; When the oysters are placed in an oyster-containing state, a preset tapping rod is controlled to tap the closed-shell oysters to be tested. The images of the closed-shell oyster to be detected and the acoustic vibration signals to be detected are collected using a pre-set industrial camera and microphone sensor, respectively.

3. The method according to claim 1, wherein, Calculating the fused spectral features based on the frequency sequence spatial signal includes: Extract the SC coefficients of the frequency sequence spatial signal to a predetermined order; the SC coefficients are a matrix with the total number of rows equal to the total number of frames and the predetermined order equal to the number of columns. The first-order and second-order difference coefficients of the SC coefficients are obtained using the difference coefficient formula; both the first-order and second-order difference coefficients are matrices with the number of rows equal to the total number of frames and the number of columns equal to the preset order. The SC coefficients, the first-order difference coefficients, and the second-order difference coefficients are summed in a 1:1:1 ratio to obtain the fused spectral feature; the fused spectral feature is a matrix with the number of rows equal to the total number of frames and the number of columns equal to the preset order.

4. The method according to claim 1, wherein, The temporal spatial features, frequency spatial features, fused spectral features, and external morphological features are fused to obtain the fused features to be analyzed, including: The temporal spatial features, the frequency spatial features, and the fused spectral features are horizontally merged to obtain an intermediate feature matrix; The mean of the intermediate feature matrix is ​​calculated column by column and then horizontally merged with the external morphological feature to obtain the fusion feature to be analyzed; the fusion feature to be analyzed has 1 row and the number of columns is the sum of the number of columns of the temporal spatial feature, the frequency spatial feature, the external morphological feature and the fusion spectral feature.

5. The non-destructive method for freshness detection of closed shell oysters based on acoustic vibration according to claim 4, characterized in that, Training sample data is constructed based on the fusion features to be analyzed and the freshness level data, and the training sample data is divided into a training set and a test set, including: Construct a feature set matrix using the fusion features to be analyzed; Construct a freshness set matrix using the freshness level data; The feature set matrix and the freshness set matrix are merged horizontally to obtain the training sample data; Based on the stratified sampling method, the training sample data is divided into the training set and the test set; the sample ratio of the training set and the test set is 2:

1.

6. The non-destructive method for freshness detection of closed shell oysters based on acoustic vibration according to claim 5, characterized in that, Based on machine learning algorithms, a two-layer chained multi-model ensemble initial model is constructed using the training set, and the initial model is validated using the test set to obtain a validated non-destructive detection model for the freshness of the closed-shell oyster, including: The training set is divided into K mutually exclusive subsets. Repeat the selection of one subset as a validation sample K times, and use the remaining K-1 subsets as training samples to obtain K sets of sample combinations of validation samples and training samples; The first-level classifier of the first layer of the machine learning algorithm is trained using K sets of sample combinations to obtain K trained first-level classifiers; Using the trained sample and the trainer, classify and predict each feature vector in the validation sample and the test set respectively to obtain K validation prediction values ​​and K test prediction values; The K verification prediction values ​​are stacked vertically and integrated with the freshness set matrix to update and replace the sample combination; The K predicted test values ​​are summed and averaged, and then integrated with the freshness set matrix to update and replace the test set. The updated sample combination is used to train the second-level classifier of the second layer of the machine learning algorithm to obtain the trained non-destructive detection model for the freshness of closed-shell oysters. The non-destructive testing model for the freshness of oysters is evaluated using the updated test set. If the evaluation results meet the standards, the non-destructive testing model for the freshness of oysters is output.

7. An acoustic vibration-based non-destructive freshness detection device for closed shell oysters, characterized in that, The method for non-destructive testing of the freshness of closed-shell oysters based on acoustic vibration as described in claim 1 includes: a noise shielding chamber frame, sound-absorbing cotton, an industrial camera, and a knocking sound vibration signal detection device; the knocking sound vibration signal detection device includes: a testing platform, a microphone bracket, a microphone sensor, an infrared sensor, a control unit, an oyster placement platform, a knocking rod, a spring, a knocking platform, a stepper motor, and a cam. The sound-absorbing cotton is disposed on the outside of the noise shielding chamber frame; the industrial camera is disposed on the upper inside of the noise shielding chamber frame; the impact vibration signal detection device is disposed inside the noise shielding chamber frame; the detection platform is fixed on the lower inside of the noise shielding chamber frame; the impact platform is disposed on one side of the detection platform; the microphone bracket is disposed at the corner of the detection platform; the microphone sensor and the infrared sensor are disposed on the microphone bracket; the oyster placement platform is disposed in the center of the detection platform; the stepper motor is disposed on the impact platform; the cam is connected to the stepper motor; the impact rod is connected to the cam; the spring is disposed between the impact platform and the impact rod; the control unit is disposed at the bottom of the detection platform. The noise shielding chamber frame provides a basic framework for the industrial camera and the impact vibration signal detection device; the sound-absorbing cotton reduces noise inside the noise shielding chamber frame; the industrial camera acquires images of closed oysters; the detection platform provides a placement platform for the oyster placement table; the microphone bracket secures the microphone sensor and the infrared sensor; the microphone sensor acquires acoustic vibration signals; the infrared sensor detects the oyster placement status on the oyster placement table; the oyster placement table holds the closed oysters to be detected; the striking rod strikes the closed oysters to be detected; the spring absorbs the rebound energy of the striking rod; the stepper motor and the cam both control the striking rod to strike the closed oysters to be detected; the control unit receives preset detection commands from a computer, controls the microphone sensor, the striking rod, the stepper motor, the cam, and the industrial camera to operate according to the detection commands, and sends the acquired images of the closed oysters and the acoustic vibration signals to the computer; the oyster placement status includes: a state with oysters placed and a state without oysters placed.

8. The non-destructive freshness detection device of the closed shell oyster based on acoustic vibration according to claim 7, characterized in that, The calculation formula of the release height of the knocking rod is: ; is the release height, is the length of the knocking rod, is the rotation angle of the cam.

Images