Object number judgment method and system based on audio signal and storage medium

Abstract

The invention provides an object quantity judgment method and system based on an audio signal and a storage medium, and the method comprises the steps: firstly, guaranteeing the comprehensiveness and representativeness of the audio signal through a training data collection step; in the training data processing step, fine preprocessing is carried out on audio signals, and the signal quality is effectively improved. A double-branch convolutional neural network structure is adopted in the model training step, and the structure can capture local details and global contexts of audio signals in parallel; the steps of deploying the model and judging the number of the objects to be detected reflect the practicability and high efficiency of the method, real-time detection can be completed through simple audio collection, dependence on a visual sensor or complex hardware configuration is not needed, and the deployment threshold and cost are remarkably reduced. The method breaks through the limitation of a traditional number detection method in the aspects of adaptability, precision and cost, is particularly suitable for industrial scenes with vision limitation, serious shielding or changeable environments, and provides a brand new technical path for object number judgment.

Description

Technical Field

[0001] This invention belongs to the field of object quantity detection technology, specifically relating to a method, system, and storage medium for judging the quantity of objects based on audio signals. Background Technology

[0002] In the process of production and resource management, accurate quantity determination of small objects (such as screws, seeds, pills, etc.) is a core link in ensuring product quality, optimizing production processes and controlling costs. Quantity accuracy directly affects the reliability of assembly processes, the efficiency of inventory management and the synergy of the supply chain, and is a necessary prerequisite for ensuring that production activities comply with design specifications and industry standards.

[0003] Currently, mainstream counting methods mainly rely on manual operation, weighing, or model analysis based on visual images. Manual counting is achieved by manually counting objects; weighing is based on the assumption that "the weight of a single object is constant," and the quantity is calculated after weighing multiple objects; while visual image methods acquire images of objects and use models to identify the quantity. These methods can complete the counting task in specific scenarios, but they have obvious limitations in application.

[0004] Manual counting is inefficient and susceptible to human error, with risks of omissions and errors. Weighing counting is based on the assumption that "the weight of a single object is constant," but in actual production, unavoidable differences in the size and material density of individual objects can lead to weight deviations. In large-scale production, the cumulative error can cause the counting results to be distorted. Visual image judgment is subject to strict requirements on environmental conditions such as light intensity and shooting angle, and is prone to misjudgment when objects are stacked, occluded, or have similar surface features. In addition, visual methods usually require additional imaging equipment and processing of complex image data, increasing system complexity and cost. Therefore, existing object counting methods are difficult to meet the requirements of high-precision and high-efficiency real-time counting. Summary of the Invention

[0005] To address the shortcomings of existing methods for determining the quantity of small objects, such as low efficiency and poor accuracy of manual counting, cumulative errors leading to distorted results in large-scale production, and the fact that commonly used visual image-based model analysis is subject to stringent requirements for environmental conditions such as light intensity and shooting angle and is prone to misjudgment, this invention provides a method, system, and storage medium for determining the quantity of objects based on audio signals.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for determining the number of objects based on audio signals, comprising:

[0008] S1: Training data acquisition: Select the type of object to be tested, collect audio signals generated when different numbers of objects to be tested move inside the container, and construct an audio signal set;

[0009] S2: Training data processing: Preprocess multiple audio signals in the audio signal set to obtain the training dataset;

[0010] S3: Model Training: Construct a dual-branch convolutional neural network model to be trained, use the training dataset to train the dual-branch convolutional neural network model to be trained, and obtain a trained branch convolutional neural network model.

[0011] S4: Deploy the model and determine the number of objects to be tested: Deploy the trained branch convolutional neural network model, collect the audio signals corresponding to the unknown number of objects to be tested moving in the container, and use the trained branch convolutional neural network model to determine the number of objects to be tested.

[0012] Optionally, in step S1, collecting the audio signals generated when different numbers of test objects move inside the container is done by placing different numbers of test objects into the same container, shaking them for more than 5 minutes, and recording the corresponding audio signals.

[0013] Optionally, step S2 specifically includes:

[0014] S2.1: Noise Reduction Processing: Perform noise reduction processing on multiple audio signals in the audio signal set to obtain multiple time-domain waveform tensors of fixed duration;

[0015] S2.2: Feature transformation: Perform feature transformation on the multiple time-domain waveform tensors with fixed durations obtained in step S2.1 to obtain multiple standardized Mel spectrograms;

[0016] S2.3: Data Augmentation: Perform data augmentation on the multiple standardized Mel spectrograms obtained in step S2.2 to obtain multiple augmented Mel spectrograms, and construct a training dataset from the multiple augmented Mel spectrograms.

[0017] Optionally, step S2.1 specifically includes:

[0018] S2.1.1: Raw audio loading: Analyze multiple audio signals in the audio signal set to obtain multiple corresponding digitized waveform signals, extract the original sampling rate of multiple audio signals from them, and output tuple data of multiple audio signals;

[0019] S2.1.2: Sampling Rate Resampling: Set a target sampling rate, use an interpolation algorithm to adjust the tuple data of each audio signal so that its sampling rate is adjusted to be consistent with the target sampling rate, and obtain multiple resampled time-domain waveform tensors;

[0020] S2.1.3: Audio length uniformity: Adjust the duration of the multiple resampled time-domain waveform tensors obtained in step S2.1.2 to obtain multiple time-domain waveform tensors with fixed durations.

[0021] Optionally, step S2.2 specifically includes:

[0022] S2.2.1: Mel spectrogram conversion: The multiple time-domain waveform tensors with fixed durations obtained in step S2.1 are divided into frames, and each frame is transformed by FFT to convert it into a spectrum. The spectrum is then converted into multiple Mel spectrograms through a Mel filter bank.

[0023] S2.2.2: Decibels conversion: Perform decibel conversion on the multiple Mel spectrograms output in step S2.2.1 to obtain Mel spectrograms in multiple decibel units;

[0024] S2.2.3: Standardization: Based on the Mel spectrograms of multiple decibel units, calculate their global mean and standard deviation, and perform standardization to obtain multiple standardized Mel spectrograms.

[0025] Optionally, the dual-branch convolutional neural network model to be trained constructed in step S3 includes a multi-scale feature extraction module with a dual-branch network structure, used to simultaneously capture local detail features and global context features of the audio signal.

[0026] Optionally, step S4 specifically includes:

[0027] S4.1: Deploy the trained branch convolutional neural network model;

[0028] S4.2: Collect the audio signals to be tested corresponding to the movement of an unknown number of objects to be tested in the container, preprocess the audio signals to be tested, and obtain the Mel spectrum of the test object;

[0029] S4.3: Input the Mel spectrum of the object to be tested into the trained branch convolutional neural network model to determine the number of objects to be tested.

[0030] Optionally, step S4.2 specifically includes:

[0031] S4.2.1: Using a single-channel microphone, collect audio signals of an unknown number of test objects falling into a container within a certain time period;

[0032] S4.2.2: Refer to steps S2.1 to S2.3 to preprocess the audio signal to be tested and obtain the Mel spectrum diagram to be tested.

[0033] Secondly, the present invention provides an object quantity determination system based on audio signals, comprising:

[0034] The training data acquisition unit is used to collect audio signals generated by different numbers of test objects moving inside the container, based on the selected types of test objects, and to construct an audio signal set.

[0035] The training data processing unit is used to preprocess multiple audio signals in the audio signal set to obtain the training dataset;

[0036] The model training unit is used to construct a dual-branch convolutional neural network model to be trained, and to train the dual-branch convolutional neural network model to be trained using the training dataset to obtain a trained branch convolutional neural network model.

[0037] The judgment unit is used to deploy a pre-trained branch convolutional neural network model, collect the audio signals corresponding to the unknown number of test objects moving in the container, and use the pre-trained branch convolutional neural network model to judge the number of test objects.

[0038] Thirdly, the present invention provides a storage medium storing instructions generated based on the above-described method for determining the number of objects based on audio signals.

[0039] The beneficial effects of this invention are:

[0040] This invention provides a method for determining the number of objects based on audio signals. Through a systematic approach, it revolutionizes object quantity detection. First, the training data acquisition step ensures the comprehensiveness and representativeness of the audio signals, laying a solid foundation for subsequent model training. The training data processing step performs refined preprocessing on the audio signals, effectively improving signal quality and eliminating interference from environmental noise and signal variations, enabling the model to extract stable and accurate features from the original audio. The model training step employs a dual-branch convolutional neural network structure, which can capture local details and global context of the audio signal in parallel, achieving high-precision quantity mapping in complex scenes and avoiding misjudgments caused by insufficient feature extraction in traditional methods. The model deployment and object quantity determination step demonstrates the method's practicality and efficiency. Real-time detection can be completed through simple audio acquisition, without relying on visual sensors or complex hardware configurations, significantly reducing deployment barriers and costs. This invention overcomes the limitations of traditional quantity detection methods in terms of adaptability, accuracy, and cost, and is particularly suitable for industrial scenarios with limited vision, severe occlusion, or variable environments, providing a completely new technical path for object quantity determination. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the object quantity determination method based on audio signals in this invention. Detailed Implementation

[0042] This invention provides a method, system, and storage medium for determining the number of objects based on audio signals, which will be further described below with reference to embodiments.

[0043] Example 1

[0044] See Figure 1 The diagram illustrates a method for determining the number of objects based on audio signals according to the present invention, including:

[0045] S1: Training data acquisition: Select the type of object to be tested, collect audio signals generated when different numbers of objects to be tested move inside the container, and construct an audio signal set;

[0046] S2: Training data processing: Preprocess multiple audio signals in the audio signal set to obtain the training dataset;

[0047] S3: Model Training: Construct a dual-branch convolutional neural network model to be trained, use the training dataset to train the dual-branch convolutional neural network model to be trained, and obtain a trained branch convolutional neural network model.

[0048] S4: Deploy the model and determine the number of objects to be tested: Deploy the trained branch convolutional neural network model, collect the audio signals corresponding to the unknown number of objects to be tested moving in the container, and use the trained branch convolutional neural network model to determine the number of objects to be tested.

[0049] Optionally, in step S1 of the present invention, collecting audio signals generated when different numbers of test objects move inside the container is done by placing different numbers of test objects into the same container, shaking them for more than 5 minutes, and recording the corresponding audio signals.

[0050] This embodiment provides a method for determining the number of objects based on audio signals. Through a systematic approach, it revolutionizes object quantity detection. First, the training data acquisition step ensures the comprehensiveness and representativeness of the audio signals, laying a solid foundation for subsequent model training. The training data processing step performs refined preprocessing on the audio signals, effectively improving signal quality and eliminating interference from environmental noise and signal variations, enabling the model to extract stable and accurate features from the original audio. The model training step employs a dual-branch convolutional neural network structure, which can capture local details and global context of the audio signals in parallel, achieving high-precision quantity mapping in complex scenarios and avoiding misjudgments caused by insufficient feature extraction in traditional methods. The model deployment and object quantity determination step demonstrates the method's practicality and efficiency. Real-time detection can be completed through simple audio acquisition, without relying on visual sensors or complex hardware configurations, significantly reducing deployment barriers and costs. This invention overcomes the limitations of traditional quantity detection methods in terms of adaptability, accuracy, and cost, and is particularly suitable for industrial scenarios with limited vision, severe occlusion, or variable environments, providing a completely new technical path for object quantity determination.

[0051] Optionally, step S2 in this invention specifically includes:

[0052] S2.1: Noise Reduction Processing: Perform noise reduction processing on multiple audio signals in the audio signal set to obtain multiple time-domain waveform tensors of fixed duration;

[0053] S2.2: Feature transformation: Perform feature transformation on the multiple time-domain waveform tensors with fixed durations obtained in step S2.1 to obtain multiple standardized Mel spectrograms;

[0054] S2.3: Data Augmentation: Perform data augmentation on the multiple standardized Mel spectrograms obtained in step S2.2 to obtain multiple augmented Mel spectrograms, and construct a training dataset from the multiple augmented Mel spectrograms.

[0055] In this embodiment, the audio signal is first purified using a noise reduction step, effectively separating environmental noise from effective sound features and improving the signal-to-noise ratio. This allows the model to focus on key audio patterns generated by object collisions during training, avoiding feature confusion or mislearning caused by noise. The feature transformation step converts the time-domain waveform into a Mel spectrogram. This transformation simulates the human auditory perception mechanism, converting the audio signal into a frequency domain representation that is easier for the model to process, thereby enhancing the separability and robustness of features. Especially for superimposed signals from multiple object collisions, the Mel spectrogram can clearly present the harmonic structure and time-frequency relationship, providing a basis for accurate quantity judgment. The data augmentation step artificially expands the diversity of training data by randomly transforming the Mel spectrogram, such as by masking or adding noise, effectively preventing model overfitting and improving its adaptability to unknown scenarios. These three steps are interconnected and jointly optimize the quality of the training dataset, enabling the dual-branch convolutional neural network model to learn more stable and generalized feature maps from the preprocessed data. In practical applications, this method significantly improves the accuracy and reliability of quantity judgment, and is particularly suitable for high-noise industrial environments or scenarios with variable object surface conditions, providing solid technical support for automated detection.

[0056] Optionally, step S2.1 in this invention specifically includes:

[0057] S2.1.1: Raw audio loading: Analyze multiple audio signals in the audio signal set to obtain multiple corresponding digitized waveform signals, extract the original sampling rate of multiple audio signals from them, and output tuple data of multiple audio signals;

[0058] S2.1.2: Sampling Rate Resampling: Set a target sampling rate, use an interpolation algorithm to adjust the tuple data of each audio signal so that its sampling rate is adjusted to be consistent with the target sampling rate, and obtain multiple resampled time-domain waveform tensors;

[0059] S2.1.3: Audio length uniformity: Adjust the duration of the multiple resampled time-domain waveform tensors obtained in step S2.1.2 to obtain multiple time-domain waveform tensors with fixed durations.

[0060] In this embodiment, the original audio loading step ensures the integrity of the original audio signal. Through parsing and digitization, the high-fidelity characteristics of the sound waveform are preserved, providing reliable input for subsequent processing. The sampling rate resampling step uses an interpolation algorithm to adjust the sampling rate to the target value, unifying the temporal resolution of the audio data and avoiding feature inconsistencies caused by sampling rate differences. This allows the model to handle signals from different acquisition devices, enhancing the compatibility and standardization of the method. The audio length unification step ensures that all training samples have a fixed length by adjusting the duration of the waveform tensor, eliminating the interference of variable duration on the model input, simplifying the network structure design, and improving training efficiency and stability. These three sub-steps together construct an efficient data preprocessing pipeline, which not only improves signal quality but also optimizes the utilization of computing resources and reduces processing latency.

[0061] Optionally, step S2.2 in this invention specifically includes:

[0062] S2.2.1: Mel spectrogram conversion: The multiple time-domain waveform tensors with fixed durations obtained in step S2.1 are divided into frames, and each frame is transformed by FFT to convert it into a spectrum. The spectrum is then converted into multiple Mel spectrograms through a Mel filter bank.

[0063] S2.2.2: Decibels conversion: Perform decibel conversion on the multiple Mel spectrograms output in step S2.2.1 to obtain Mel spectrograms in multiple decibel units;

[0064] S2.2.3: Standardization: Based on the Mel spectrograms of multiple decibel units, calculate their global mean and standard deviation, and perform standardization to obtain multiple standardized Mel spectrograms.

[0065] In this embodiment, the Mel spectrogram conversion step frames the time-domain waveform and converts it into a frequency-domain representation. Utilizing a Mel filter bank to simulate human hearing characteristics, it makes the audio features more consistent with perceptual patterns, thereby enhancing the model's ability to capture acoustic patterns related to the number of objects. This is particularly effective in multi-object collision scenarios, clearly resolving frequency components and temporal relationships. The decibel conversion step converts linear amplitude to logarithmic decibel values, compressing the signal's dynamic range, highlighting the energy distribution of key features, reducing the impact of volume fluctuations on detection results, and improving the method's robustness in varying volume environments. The standardization process normalizes the data distribution through mean and standard deviation calculations, eliminating scale differences between samples, accelerating model convergence, and improving generalization performance. The combination of these three operations ensures high consistency and comparability of the Mel spectrograms in the training dataset, providing optimized input features for the dual-branch convolutional neural network model. Its beneficial effects also include reduced model training complexity, as preprocessed features are easier for the network to learn, reducing the required training epochs and computational resources. In practical applications, this method can adapt to various object types and container materials without adjusting parameters for different scenarios, achieving a "plug-and-play" detection experience and bringing greater flexibility and efficiency to industrial automation.

[0066] Optionally, the dual-branch convolutional neural network model to be trained constructed in step S3 of the present invention includes a multi-scale feature extraction module with a dual-branch network structure, which is used to simultaneously capture local detail features and global context features of the audio signal.

[0067] In this embodiment, the dual-branch structure can process local detail features and global contextual features of audio signals in parallel. The local branch focuses on subtle hourly frequency variations using small convolutional kernels to accurately capture transient patterns of single-object collisions, while the global branch utilizes large convolutional kernels with wide receptive fields to capture broadband correlations of multi-object interactions. This multi-scale fusion mechanism effectively solves the problem of insufficient feature extraction in dense or occluded scenes in traditional single-branch networks. The multi-scale feature extraction module enables the model to adapt to sound signals from objects of different orders of magnitude, achieving stable mapping from clear collisions of a few objects to overlapping noise of many objects, avoiding performance degradation caused by scale variations.

[0068] Furthermore, this design enhances the model's robustness. The complementarity of the two branches ensures the integrity of the feature representation and reduces the risk of misjudgment when faced with environmental noise or signal attenuation. In practical deployments, this method significantly improves the accuracy and speed of quantity determination, making it particularly suitable for industrial scenarios with high real-time requirements, such as online counting on production lines, providing a smarter and more reliable solution for object detection.

[0069] Optionally, step S4 in this invention specifically includes:

[0070] S4.1: Deploy the trained branch convolutional neural network model;

[0071] S4.2: Collect the audio signals to be tested corresponding to the movement of an unknown number of objects to be tested in the container, preprocess the audio signals to be tested, and obtain the Mel spectrum of the test object;

[0072] S4.3: Input the Mel spectrum of the object to be tested into the trained branch convolutional neural network model to determine the number of objects to be tested.

[0073] Optionally, step S4.2 in this invention specifically includes:

[0074] S4.2.1: Using a single-channel microphone, collect audio signals of an unknown number of test objects falling into a container within a certain time period;

[0075] S4.2.2: Refer to steps S2.1 to S2.3 to preprocess the audio signal to be tested and obtain the Mel spectrum diagram to be tested.

[0076] Example 2

[0077] Secondly, the present invention provides an object quantity determination system based on audio signals, comprising:

[0078] The training data acquisition unit is used to collect audio signals generated by different numbers of test objects moving inside the container, based on the selected types of test objects, and to construct an audio signal set.

[0079] The training data processing unit is used to preprocess multiple audio signals in the audio signal set to obtain the training dataset;

[0080] The model training unit is used to construct a dual-branch convolutional neural network model to be trained, and to train the dual-branch convolutional neural network model to be trained using the training dataset to obtain a trained branch convolutional neural network model.

[0081] The judgment unit is used to deploy a pre-trained branch convolutional neural network model, collect the audio signals corresponding to the unknown number of test objects moving in the container, and use the pre-trained branch convolutional neural network model to judge the number of test objects.

[0082] In this embodiment, the training data acquisition unit ensures high-quality acquisition of audio signals, the training data processing unit optimizes signal quality through automated preprocessing, the model training unit utilizes a dual-branch convolutional neural network for efficient learning, and the judgment unit completes real-time inference deployment. This modular design enhances the system's maintainability and scalability, allowing users to flexibly configure unit components according to their needs and supporting customized applications. Its beneficial effects include reduced manual intervention, as the system can automatically complete the entire process from data acquisition to result output, improving detection efficiency and consistency.

[0083] Meanwhile, the system supports multi-user parallel operation, facilitating integration into large industrial networks for data sharing and remote monitoring. From a technical perspective, the system is robust, with collaborative work between units ensuring detection accuracy and stability, making it particularly suitable for large-scale or continuous production environments. Furthermore, it promotes technology adoption because the system is simple to deploy and requires no specialized knowledge to operate, providing SMEs with a low-cost path to intelligent transformation and driving industrial upgrading.

[0084] Example 3

[0085] Thirdly, the present invention provides a storage medium storing instructions that are generated based on the object quantity determination method based on audio signals in Embodiment 1.

[0086] Specifically, the storage medium described in this embodiment may include random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, or CD-ROM. It should be noted that those skilled in the art can select the form and type of storage medium according to actual production and usage needs; this embodiment does not impose further limitations.

[0087] It should be noted that those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and storage media described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0088] Example 4

[0089] To further illustrate the method in Embodiment 1, examples are used in this embodiment for explanation.

[0090] Specifically, this example is used to determine the number of seeds.

[0091] S1: Training data acquisition: Using a single-channel microphone (sampling rate 16000Hz) fixed near the plastic container containing the seeds, record the sound of different numbers of seeds (1-20 seeds) moving in the plastic container. 600 segments are collected for each number of samples, and each segment is 5 seconds long (including the complete process of the seed falling).

[0092] Furthermore, this example includes samples of different seed varieties (such as corn, soybeans, and wheat), different drop heights (20cm / 40cm / 60cm), and different background noises (such as laboratory environment and slight ventilation noise) to ensure data diversity and construct an audio signal set.

[0093] S2: Training Data Processing: Label the number of seeds corresponding to each audio signal in the audio signal set; perform preprocessing, specifically including:

[0094] S2.1: Noise reduction processing: An improved spectral subtraction method is used to remove environmental noise (such as ventilation noise) while retaining the high-frequency characteristics of seed impact (2000-8000Hz);

[0095] S2.1.1: Raw audio loading: This step parses multiple audio signals from the audio signal set to obtain multiple corresponding digitized waveform signals, extracts the original sampling rates of the multiple audio signals, and outputs tuple data of the multiple audio signals. This step only relies on the native attributes of the audio file itself, such as the sampling rate and bit depth, and does not require manual configuration of additional parameters.

[0096] Specifically, audio files can be parsed using audio parsing algorithms. Examples can be implemented based on the torchaudio.load() or ibrosa.load() algorithms, which convert analog audio signals into digital waveform tensors, read the original sampling rate (sr) of the audio file, and output (waveform, sr) tuple data (waveform tensor and original sampling rate).

[0097] S2.1.2: Sampling Rate Resampling: Set the target sampling rate to 16000Hz, and use an interpolation algorithm to adjust the tuple data of each audio signal. Specifically, this can be implemented based on torchaudio.functional.resample() or torchaudio.transforms.Resample algorithm to adjust its sampling rate to be consistent with the target sampling rate, and obtain multiple resampled time-domain waveform tensors. In this step, the sampling frequency of all audio signals is unified to ensure that the audio data format of the input model is consistent.

[0098] S2.1.3: Audio Length Unification: Adjust the duration of the multiple resampled time-domain waveform tensors obtained in step S2.1.2. Specifically, a sliding window clipping algorithm can be used, based on the librosa audio slicing logic, to uniformly adjust them to 5 seconds, obtaining multiple time-domain waveform tensors with fixed durations. In this step, a standard duration is set for all audio signals to ensure the uniformity of the dimensions of the time-domain waveform tensors.

[0099] S2.2: Feature transformation: Convert the audio to a Mel spectrogram (parameters: N_MELS=64, N_FFT=1024, HOP_LENGTH=512), with dimensions [1×157×64];

[0100] S2.2.1: Mel spectrogram conversion: The multiple time-domain waveform tensors with fixed durations obtained in step S2.1 are divided into frames, and each frame is transformed by FFT to convert it into a spectrum. The spectrum is then converted into multiple Mel spectrograms through a Mel filter bank.

[0101] Specifically, based on the framing algorithm (based on HOP_LENGTH and N_FFT), the time-domain waveform tensor of fixed duration is divided into continuous overlapping frames according to N_FFT (window size) and HOP_LENGTH (frame shift); the fast Fourier transform (FFT) is used to perform FFT on the time-domain signal of each frame to convert the time domain into a linear spectrum (obtaining amplitude spectrum / power spectrum); the linear spectrum is filtered through Mel filter bank mapping (based on torchaudio.transforms.MelSpectrogram) to preserve the Mel frequency characteristics that are sensitive to the human ear, and the output is a Mel spectrogram tensor with shape (1, N_MELS, number of time frames).

[0102] Specifically, the values ​​are: N_MELS = 64, N_FFT = 1024, HOP_LENGTH = 512. The ratio of N_FFT (1024) to HOP_LENGTH (512) is 2:1, which means that adjacent frames overlap by 50%, ensuring temporal resolution while avoiding inter-frame information redundancy. N_MELS = 64 is an empirical value that balances feature dimension and computational efficiency, and can cover the key frequency range of human speech and common environmental sounds at a sampling rate of 16000Hz.

[0103] S2.2.2: Decibels conversion: Perform decibel conversion on the multiple Mel spectrograms output in step S2.2.1 to obtain Mel spectrograms in multiple decibel units.

[0104] Specifically, based on the amplitude-to-decibel (Amplitude-to-DB) conversion algorithm, specifically the torchaudio.transforms.AmplitudeToDB() algorithm, the amplitude value (linear scale) of the Mel spectrum is converted into a decibel value (logarithmic scale). The formula logic is dB = 10 × log10 (amplitude / reference amplitude), and the default reference amplitude is 1.

[0105] S2.2.3: Standardization: Based on the Mel spectrograms of multiple decibel units, calculate their global mean and standard deviation, and perform standardization to obtain multiple standardized Mel spectrograms.

[0106] Specifically, the Z-Score normalization algorithm (based on (x-μ) / σ) is used to eliminate differences in data dimensions, so that the pixel values ​​of the spectrogram are distributed around [-1,1], which accelerates model convergence and avoids excessive local feature proportions.

[0107] S2.3: Data Augmentation: Randomly crop the time axis of the spectrogram (extracting a valid segment of 2 to 4 seconds) and perturb the volume (±10% amplitude) to simulate the sound changes at different falling speeds and distances, thus obtaining a training dataset.

[0108] S3: Model Training

[0109] Construct a two-branch convolutional neural network model to be trained and optimize it for the seed counting scenario:

[0110] Output layer dimension: Since the count range is 1-20, set num_classes=20;

[0111] Feature branch enhancement: Add one 5×5 convolution layer (channel number 128→256) to the global branch to enhance the capture of broadband features of continuous impact sound of multiple seeds.

[0112] To train the dual-branch convolutional neural network model, the training dataset is divided into a training set (8400 segments) and a test set (3600 segments) in a 7:3 ratio, and the batch_size is set to 32 using DataLoader.

[0113] Training parameters: The optimizer is Adam (initial learning rate 0.001), cosine annealing learning rate scheduling is used (T_max = 100), and the training epochs are 100.

[0114] Regularization: Keep the Dropout rate at 0.3 and combine it with an early stopping mechanism (patience=15) to prevent overfitting, and obtain a well-trained branch convolutional neural network model.

[0115] S4: Deploy the model and determine the number of objects to be tested: Deploy a microphone next to the drop channel of the seed screening production line to collect the sound of seeds hitting plastic containers in real time. Generate a test audio signal every 5 seconds, input it into the trained branch convolutional neural network model, and output the quantity prediction result corresponding to the test audio signal.

[0116] S4.1: Deploy the trained branch convolutional neural network model.

[0117] Specifically, the model is instantiated according to the network structure during training, the pre-trained optimal weight parameters are loaded, the training effect is reproduced, and Dropout and BatchNorm updates during the training phase are disabled to ensure inference stability.

[0118] S4.2: Collect the audio signals corresponding to the movement of an unknown number of test objects in the container, preprocess the audio signals to obtain the Mel spectrum of the test.

[0119] Specifically, S4.2 includes:

[0120] S4.2.1: Use a single-channel microphone to collect audio signals of an unknown number of test objects falling into a container within a certain time period.

[0121] Specifically, the audio signal generated by the movement of an object is captured by a single-channel microphone, and the audio signal is converted into a digital WAV file to ensure that the acquired signal contains key features such as object collision and friction.

[0122] S4.2.2: Refer to steps S2.1 to S2.3 to preprocess the audio signal to be tested and obtain the Mel spectrum diagram to be tested.

[0123] S4.3: Input the Mel spectrum of the object to be tested into the trained branch convolutional neural network model to determine the number of objects to be tested.

[0124] Furthermore, step S4 follows the principle of deployment consistency: the core logic of steps S4.1 to S4.3 is to "reuse the parameters and algorithms of the training phase" to ensure that the format of the entire process from preprocessing to model inference is consistent. For example, the preprocessing parameters (sampling rate, Mel spectrum configuration) must be completely consistent with those during training, otherwise it will lead to feature distribution shift and decreased prediction accuracy.

[0125] At the same time, the compatibility of preprocessing also needs to be considered: such as S4.2.2 referring to steps S2.1 to S2.3, not only is the algorithm reused, but the "data format verification" logic is also inherited, such as automatically converting multi-channel audio to single-channel, automatically calibrating the sampling rate, and uniformly trimming the duration, to ensure that even if the acquired audio has format differences, it can be converted into an input acceptable to the model through preprocessing.

[0126] Unless explicitly stated otherwise, the actions or steps of the methods and procedures described in the embodiments of the present invention do not necessarily have to be performed in a specific order and can still achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0127] This specification describes various embodiments of the present invention. However, for the sake of brevity, the descriptions of the embodiments are not exhaustive, and identical or similar features or parts between the embodiments may be omitted. In this document, "one embodiment," "some embodiments," "example," "specific example," or "some examples" refers to embodiments applicable to at least one, but not all, of the present invention. The above terms do not necessarily refer to the same embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples.

Claims

1. A method for determining the number of objects based on audio signals, characterized in that, include: S1: Training data acquisition: Select the type of object to be tested, collect audio signals generated when different numbers of objects to be tested move inside the container, and construct an audio signal set; S2: Training data processing: Preprocess multiple audio signals in the audio signal set to obtain the training dataset; S3: Model Training: Construct a dual-branch convolutional neural network model to be trained, use the training dataset to train the dual-branch convolutional neural network model to be trained, and obtain a trained branch convolutional neural network model. S4: Deploy the model and determine the number of objects to be tested: Deploy the trained branch convolutional neural network model, collect the audio signals corresponding to the unknown number of objects to be tested moving in the container, and use the trained branch convolutional neural network model to determine the number of objects to be tested.

2. The method for determining the number of objects based on audio signals according to claim 1, characterized in that, In step S1, collecting audio signals generated when different numbers of test objects move inside the container involves placing different numbers of test objects into the same container, shaking them for more than 5 minutes, and recording the corresponding audio signals.

3. The method for determining the number of objects based on audio signals according to claim 2, characterized in that, Step S2 specifically includes: S2.1: Noise Reduction Processing: Perform noise reduction processing on multiple audio signals in the audio signal set to obtain multiple time-domain waveform tensors of fixed duration; S2.2: Feature transformation: Perform feature transformation on the multiple time-domain waveform tensors with fixed durations obtained in step S2.1 to obtain multiple standardized Mel spectrograms; S2.3: Data Augmentation: Perform data augmentation on the multiple standardized Mel spectrograms obtained in step S2.2 to obtain multiple augmented Mel spectrograms, and construct a training dataset from the multiple augmented Mel spectrograms.

4. The method for determining the number of objects based on audio signals according to claim 3, characterized in that, Step S2.1 specifically includes: S2.1.1: Raw audio loading: Analyze multiple audio signals in the audio signal set to obtain multiple corresponding digitized waveform signals, extract the original sampling rate of multiple audio signals from them, and output tuple data of multiple audio signals; S2.1.2: Sampling Rate Resampling: Set a target sampling rate, use an interpolation algorithm to adjust the tuple data of each audio signal so that its sampling rate is adjusted to be consistent with the target sampling rate, and obtain multiple resampled time-domain waveform tensors; S2.1.3: Audio length uniformity: Adjust the duration of the multiple resampled time-domain waveform tensors obtained in step S2.1.2 to obtain multiple time-domain waveform tensors with fixed durations.

5. The method for determining the number of objects based on audio signals according to claim 4, characterized in that, Step S2.2 specifically includes: S2.2.1: Mel spectrogram conversion: The multiple time-domain waveform tensors with fixed durations obtained in step S2.1 are divided into frames, and each frame is transformed by FFT to convert it into a spectrum. The spectrum is then converted into multiple Mel spectrograms through a Mel filter bank. S2.2.2: Decibels conversion: Convert the multiple Mel spectrograms output in step S2.2.1 to decibels to obtain Mel spectrograms in multiple decibel units; S2.2.3: Standardization: Based on the Mel spectrograms of multiple decibel units, calculate their global mean and standard deviation, and perform standardization to obtain multiple standardized Mel spectrograms.

6. The method for determining the number of objects based on audio signals according to claim 5, characterized in that, The dual-branch convolutional neural network model to be trained constructed in step S3 includes a multi-scale feature extraction module with a dual-branch network structure, which is used to simultaneously capture local detail features and global context features of the audio signal.

7. The method for determining the number of objects based on audio signals according to claim 5, characterized in that, Step S4 specifically includes: S4.1: Deploy the trained branch convolutional neural network model; S4.2: Collect the audio signals to be tested corresponding to the movement of an unknown number of objects to be tested in the container, preprocess the audio signals to be tested, and obtain the Mel spectrum of the test object; S4.3: Input the Mel spectrum of the object to be tested into the trained branch convolutional neural network model to determine the number of objects to be tested.

8. The method for determining the number of objects based on audio signals according to claim 7, characterized in that, Step S4.2 specifically includes: S4.2.1: Using a single-channel microphone, collect audio signals of an unknown number of test objects falling into a container within a certain time period; S4.2.2: Refer to steps S2.1 to S2.3 to preprocess the audio signal to be tested and obtain the Mel spectrum diagram to be tested.

9. A system for determining the number of objects based on audio signals, characterized in that, include: The training data acquisition unit is used to collect audio signals generated by different numbers of test objects moving inside the container, based on the selected types of test objects, and to construct an audio signal set. The training data processing unit is used to preprocess multiple audio signals in the audio signal set to obtain the training dataset; The model training unit is used to construct a dual-branch convolutional neural network model to be trained, and to train the dual-branch convolutional neural network model to be trained using the training dataset to obtain a trained branch convolutional neural network model. The judgment unit is used to deploy a pre-trained branch convolutional neural network model, collect the audio signals corresponding to the unknown number of test objects moving in the container, and use the pre-trained branch convolutional neural network model to judge the number of test objects.

10. A storage medium, characterized in that, The storage medium stores instructions that are generated based on the method for determining the number of objects based on audio signals, as described in any one of claims 1 to 8.

Images