Cow estrus detection method and device based on Transformer spectrogram feature learning

Abstract

The present invention discloses a cow estrus detection method and device based on Transformer spectrogram feature learning, the method comprising: dividing the collected cow sound data into sound segments of equal length and performing noise reduction; annotating the noise-reduced sound segments; converting all the noise-reduced sound segments into Mel spectrograms and performing data enhancement; constructing a Transformer-based spectrogram feature learning model, using a multi-head self-attention mechanism to learn the Mel spectrogram after data enhancement, and obtaining the deep features of the Mel spectrogram; constructing a cow estrus sound event detection model based on a channel attention mechanism, using the deep features of the Mel spectrogram as input, classifying and connecting the time domain clues in the deep features through deep learning, realizing cow estrus sound event detection, and finally outputting the time and probability of the estrus event. The present invention uses the attention mechanism to identify and locate the sound events of the cow, and realizes the detection of cow estrus events based on sound signals.

Description

Technical Field

[0001] The present invention relates to a cow estrus detection technology, and in particular to a cow estrus detection method and device based on Transformer spectrogram feature learning. Background Art

[0002] With the development of the information technology era, more and more artificial intelligence technologies are being applied to the development of smart agriculture. Cattle farming is an important part of my country's animal husbandry, and the demand for beef in my country has increased year by year. Beef cattle farmers use artificial insemination to expand their breeding scale. In order to make fertilization more efficient, cattle farmers need to judge the estrus cycle of beef cattle. The detection method of cow estrus is mainly based on observation, such as judging the hormone level of cows, real-time observation and genital palpation. (See Yu Huifang's "A Brief Analysis of Several Key Points in Observing the Estrus Period of Cows", "Animal Husbandry and Veterinary Science", 2018, Issue 6). Although these methods are highly accurate, they require the real-time presence of animal husbandry personnel, consume a lot of manpower and material resources, and cannot achieve remote real-time monitoring. The front-line production experience of major cattle-raising enterprises shows that cows have typical auditory characteristics when they are in estrus, such as frequent mooing, and the call contains information to attract bulls to mate.

[0003] Machine learning is a category of artificial intelligence, and deep learning is a data representation learning method based on neural networks in machine learning. Sound event detection is a specific field of audio pattern recognition, which aims to detect whether a target audio event appears in an audio segment and mark its start and end time. These technologies provide a solid foundation for using deep learning methods to detect sound events of cow estrus. However, the lack of labeled data is a major problem in building deep learning models based on supervised learning; and due to the different complexities of different breeding environments, the accuracy of detection will also be greatly affected. Summary of the invention

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and to provide a method and device for detecting cow estrus based on Transformer spectrogram feature learning, which utilizes the attention mechanism to identify and locate the sound events of cows, and thereby realizes the detection of cow estrus events based on sound signals.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] On the one hand, the present invention provides a cow estrus detection method based on Transformer spectrogram feature learning, comprising the following steps:

[0007] Collect the sound data of the cow, divide the sound data into sound segments of equal length, and perform noise reduction on all the sound segments;

[0008] Label the denoised sound clips to form a standard labeled data set;

[0009] Convert the sound clips of the standard dataset into Mel spectrograms and perform data augmentation on the Mel spectrograms;

[0010] Constructing a Transformer-based spectrogram feature learning model, wherein the Transformer-based spectrogram feature learning model uses a multi-head self-attention mechanism to learn the data-enhanced Mel spectrogram to obtain deep features of the Mel spectrogram;

[0011] A cow estrus sound event detection model based on the channel attention mechanism is constructed. The cow estrus sound event detection model takes the deep features of the Mel spectrogram as input, classifies and concatenates the time domain clues in the deep features through deep learning, realizes cow estrus sound event detection, and finally outputs the time and probability of the estrus event.

[0012] As a preferred technical solution, the noise reduction process for all sound clips is specifically as follows:

[0013] A noise fragment sample is input, the spectrum of the noise fragment is obtained by calculation, and then the characteristic frequency component corresponding to the spectrum of the noise fragment is subtracted from the collected cow sound data to complete the noise reduction process.

[0014] As a preferred technical solution, the marking is to mark the sound event category and start and end time in the sound clip.

[0015] As a preferred technical solution, the data enhancement of the Mel-spectrogram includes:

[0016] First, the mel spectrogram is processed by the confusion method, and then the frequency domain and time domain masking methods are used to process the mel spectrogram. The specific process is as follows:

[0017] Confusion method: The spectrograms converted from the two sound clips are superimposed in proportion, and the matrices corresponding to the two sound clips are also superimposed in the same proportion to expand the number of training samples;

[0018] Frequency domain and time domain masking method: By masking the frequency domain channel and the time domain channel, the local information of the Mel spectrogram is removed to increase the complexity of the data.

[0019] As a preferred technical solution, the Transformer-based spectrogram feature learning model uses a multi-head self-attention mechanism to learn the Mel spectrogram after data enhancement to obtain the deep features of the Mel spectrogram, specifically:

[0020] First, the Mel-size spectrogram is divided into spectrogram blocks of a preset size in a non-overlapping manner. Each spectrogram block is linearly transformed into a one-dimensional vector of D dimension. There are a total of U one-dimensional vectors of D dimension. The U one-dimensional vectors of D dimension are concatenated to obtain an input matrix of dimension [U, D].

[0021] Subsequently, the input matrix is ​​added to the matrix recording the corresponding position information of each one-dimensional vector in the spectrogram to realize the position encoding of the one-dimensional vector in the input matrix;

[0022] Secondly, the position-encoded matrix is ​​input into multiple multi-head self-attention mechanism modules for calculation. The calculation formula of the self-attention mechanism is:

[0023]

[0024] Where Q, K, and V represent the query vector, key vector, and value vector in the self-attention mechanism, respectively, which are obtained by passing the input matrix through a fully connected layer. Attention(Q, K, V) is the output result of the self-attention mechanism. k is the dimension of K, K T represents the transpose of the key vector K, and softmax is the activation function;

[0025] Finally, the output of the self-attention mechanism is a matrix of size [U, D]. The multi-head self-attention module concatenates the output matrices of multiple self-attention mechanisms, and finally restores the concatenated matrices to a matrix of size [U, D] through linear transformation to obtain deep features that describe global correlation.

[0026] As a preferred technical solution, the calculation process of the cow estrus sound event detection model is as follows:

[0027] First, the deep features with dimension [U, D] are reshaped to [H′, W′, D], so that the time domain cues and frequency domain cues correspond to each other on a two-dimensional level, where H′ represents the number of frequency domain cues and W′ represents the number of time domain cues.

[0028] Then, the reshaped deep features are input into the channel attention module for operation to obtain a 1×1×D matrix, which represents the weight value of the importance of each channel of the deep feature. The channel corresponds to the dimension of the one-dimensional vector of the input matrix. The corresponding channel is multiplied by the corresponding weight value to obtain the channel-enhanced deep feature;

[0029] Finally, the frequency domain dimensions of the channel-enhanced deep features are fused through average pooling to obtain a matrix of dimension [W′,D] that only represents the time domain cues. The time domain cues are classified to obtain a probability matrix of dimension [W′,C] that represents each event category of the time domain cues, which is the sound event detection result, where C is the number of categories finally classified.

[0030] As a preferred technical solution, the binary cross entropy loss function during the training of the Transformer-based spectrogram feature learning model and the cow estrus sound event detection model is:

[0031]

[0032] Among them, H p represents the loss function, y i Represents the label vector. If the true label is marked as a certain event category, then the category is in y i The corresponding position in the vector is marked as 1, otherwise it is marked as 0. i ) represents the probability that the predicted temporal clue is of the i-th category.

[0033] On the other hand, the present invention provides a cow estrus detection system based on Transformer spectrogram feature learning, which is applied to the cow estrus detection method based on Transformer spectrogram feature learning, including a data acquisition module, a data annotation module, a data enhancement module, a learning model construction module and a detection model construction module;

[0034] The data collection module is used to collect cow sound data, divide the sound data into sound segments of equal length, and perform noise reduction processing on all sound segments;

[0035] The data annotation module is used to annotate the noise-reduced sound segments to form a labeled standard data set;

[0036] The data enhancement module is used to convert all the noise-reduced sound clips into Mel spectrograms and perform data enhancement on the Mel spectrograms;

[0037] The learning model building module is used to build a Transformer-based spectrogram feature learning model, and the Transformer-based spectrogram feature learning model uses a multi-head self-attention mechanism to learn the Mel spectrogram after data enhancement to obtain the deep features of the Mel spectrogram;

[0038] The detection model construction module is used to construct a cow estrus sound event detection model based on the channel attention mechanism. The cow estrus sound event detection model takes the deep features of the cow's sound as input, classifies and connects the time domain clues in the deep features through deep learning, realizes the cow estrus sound event detection, and finally outputs the time and probability of the estrus event.

[0039] Another aspect of the present invention provides an electronic device, the electronic device comprising:

[0040] at least one processor; and,

[0041] a memory communicatively connected to the at least one processor; wherein,

[0042] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the cow estrus detection method based on Transformer spectrogram feature learning.

[0043] On the other hand, the present invention provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the cow estrus detection method based on Transformer spectrogram feature learning.

[0044] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0045] 1. The present invention proposes a cow estrus detection method based on Transformer spectrogram feature learning. Compared with the manual cow estrus detection method, it saves a lot of analysis time and reduces labor costs; compared with other sound event detection methods based on deep learning, it has higher accuracy.

[0046] 2. The present invention performs preprocessing through segmentation, noise reduction, Mel-spectrogram conversion and data enhancement operations, which reduce the impact of environmental noise on event detection, alleviate the problem of insufficient cow sound data under realistic conditions, and realize the conversion of sound signals in the frequency domain.

[0047] 3. The present invention constructs a Transformer-based spectrogram feature learning model and uses a multi-head self-attention mechanism to learn Mel-level spectrograms. It can simultaneously calculate the correlation between clues in the time domain and the frequency domain and perform modeling to obtain the global information of the spectrogram, and the extracted features are more robust.

[0048] 4. The present invention constructs a sound event detection model based on the channel attention mechanism, and uses the channel attention mechanism to enhance the channel level of deep features, so that the model focuses on important features and suppresses unimportant features, thereby improving the classification effect when classifying time domain clues in the subsequent time domain. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 This is a flow chart of a cow estrus detection method based on Transformer spectrogram feature learning according to an embodiment of the present invention;

[0051] Figure 2 is a schematic diagram of extracting spectrogram features based on a feature extraction model of Transformer in an embodiment of the present invention;

[0052] Figure 3 is a schematic diagram of an embodiment of the present invention, in which a sound event detection model based on a channel attention mechanism performs event detection when a cow is in estrus;

[0053] Figure 4 It is the calculation process of the channel attention mechanism of the embodiment of the present invention;

[0054] Figure 5 It is a block diagram of a cow estrus detection system based on Transformer spectrogram feature learning according to an embodiment of the present invention;

[0055] Figure 6 4 is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0057] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0058] The difficulty of using cow sound data for estrus detection lies in building a Transformer-based spectrogram feature learning model, and using the multi-head self-attention mechanism to learn the enhanced Mel spectrogram to obtain the deep features of the Mel spectrogram; based on the channel attention mechanism, a cow estrus sound event detection model is built. This model takes the deep features of the Mel spectrogram as input and outputs the time and probability of the estrus event. The time domain clues in the deep features are classified and connected in series through deep learning to realize cow estrus sound event detection.

[0059] In this embodiment, the cow estrus detection method based on Transformer spectrogram feature learning includes the following steps:

[0060] (1) The sound data of cows are collected and divided into sound segments of equal length. Then, spectral subtraction is used to perform noise reduction on all sound segments.

[0061] Furthermore, a noise segment sample is input, the spectrum of the noise segment is obtained by calculation, and then the characteristic frequency component corresponding to the spectrum of the noise segment is subtracted from the sound data to be processed, so as to achieve the purpose of reducing noise. Since the noise in the cattle farm mainly comes from exhaust fans or agricultural machinery, the noise is stable, changes slowly and is additive, so the spectral subtraction method can also achieve good results.

[0062] It is understandable that the use of spectral subtraction to reduce the noise of a sound segment does not belong to the present invention itself and may be replaced by other audio noise reduction methods.

[0063] (2) Annotate the noise-reduced sound clips;

[0064] Furthermore, the start and end times of the cow estrus sound events in the sound clips are annotated to form a labeled standard data set.

[0065] (3) Convert all sound clips into Mel spectrograms;

[0066] It can be understood that the size of the Mel spectrogram is H×W, where H represents the dimension of the frequency domain and W represents the dimension of the time domain.

[0067] (4) Perform data enhancement on the Mel-level spectrogram;

[0068] Furthermore, the specific method for data enhancement is:

[0069] (4-1) First, the spectrogram is processed by the confusion method, which is to superimpose the spectrograms converted from the two sound clips in proportion, and at the same time, the matrices corresponding to the two sound clips are also superimposed in the same proportion. In this way, the purpose of expanding the training samples is achieved.

[0070] (4-2) Then, the frequency domain and time domain masking methods are used to process the spectrogram: the local information of the spectrogram is removed by masking the frequency domain channels and masking the time domain channels, thereby increasing the complexity of the data and training a more robust model.

[0071] (4-3) Since data enhancement can alleviate the problem of insufficient cow sound data under real conditions and increase the diversity of training data, the generalization ability of the model can be improved in the subsequent construction process of feature extraction model and sound event detection model.

[0072] (5) constructing a Transformer-based spectrogram feature learning model, wherein the Transformer-based spectrogram feature learning model uses a multi-head self-attention mechanism to learn the data-enhanced Mel spectrogram to obtain deep features of the Mel spectrogram;

[0073] It is understandable that due to the problem of insufficient cow sound data, this embodiment uses a Transformer pre-trained on a large-scale audio dataset to build a spectrogram feature learning model, and uses the cow sound spectrogram for training, such as Figure 2 As shown, the calculation process of the Transformer-based spectrogram feature learning model is:

[0074] (5-1) The Mel spectrogram is divided into 16×16 spectrogram blocks in a non-overlapping manner. Each spectrogram block is linearly transformed into a one-dimensional vector of dimension D. There are a total of One-dimensional vectors are concatenated to obtain an input matrix of dimension [U, D];

[0075] (5-2) The input matrix is ​​added to the matrix recording the corresponding position information of each one-dimensional vector in the spectrogram to realize the position encoding of the one-dimensional vector in the input matrix;

[0076] (5-3) The position-encoded matrix is ​​input into multiple multi-head self-attention mechanism modules for calculation. The calculation formula of the self-attention mechanism is:

[0077]

[0078] (5-4) Where Q, K, and V represent the query vector, key vector, and value vector in the self-attention mechanism, respectively, which are obtained by passing the input matrix through a fully connected layer. Attention(Q,K,V) is the output result of the self-attention mechanism, d k is the dimension of K, K T represents the transpose of the health vector K, and softmax is the activation function;

[0079] (5-5) The output of the self-attention mechanism is a matrix of size [U, D]. The multi-head self-attention module concatenates the output matrices of multiple self-attention mechanisms, and finally restores the concatenated matrices to a matrix of size [U, D] through linear transformation to obtain deep features that describe global correlations;

[0080] (5-6) Transformers pre-trained on large-scale audio datasets have a good ability to extract audio features through spectrograms, so they can quickly converge to a good level when trained on the cow sound dataset. Secondly, due to the self-attention mechanism used by Transformers, global connections of spectrograms can be established.

[0081] (6) Based on the channel attention mechanism, a cow estrus sound event detection model is constructed. The model takes deep features as input and outputs the time and probability of estrus events. The time domain clues in the deep features are classified and connected in series through deep learning to realize cow estrus sound event detection.

[0082] It is understandable that if Figure 3 As shown in the figure, the sound event detection model based on the channel attention mechanism is used to detect cow estrus events, specifically:

[0083] (6-1) The deep features of dimension [U, D] are reshaped to [H′, W′, D], so that the time domain clues and frequency domain clues can correspond to each other on a two-dimensional level. The size of H′ is represents the number of frequency domain cues. The size of W′ is Represents the number of time domain cues.

[0084] (6-2) Input the reshaped deep features into the channel attention module for operation, and obtain a 1×1×D matrix, which represents the weight value of the importance of each channel of the deep feature. Here, the channel corresponds to the dimension of the one-dimensional vector of the input matrix. The corresponding channel is multiplied by the corresponding weight value to obtain the channel-enhanced deep feature. Figure 4 shows this process.

[0085] (6-3) The frequency domain dimension of the channel-enhanced deep features is fused through average pooling to obtain a matrix of dimension [W′, D] that only represents the time domain clues;

[0086] (6-4) The time-domain clues are classified to obtain a probability matrix of dimension [W′, C] representing each event category of the time-domain clues, which is the sound event detection result. The size of C is the number of categories finally classified.

[0087] (6-5) The binary cross entropy loss function during the training of the cow estrus sound event detection model is:

[0088]

[0089] (6-6) Among them, H p represents the loss function, y i Represents the label vector. If the true label is marked as a certain event category, then the category is in y i The corresponding position in the vector is marked as 1, otherwise it is marked as 0. i ) represents the probability that the predicted temporal clue is of the i-th category.

[0090] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously. With the same idea as the cow estrus detection method based on Transformer spectrogram feature learning, the present invention also provides a cow estrus detection system based on Transformer spectrogram feature learning, which can be used to execute the above-mentioned cow estrus detection method based on Transformer spectrogram feature learning. For the sake of convenience, the structural schematic diagram of the embodiment of the cow estrus detection system based on Transformer spectrogram feature learning only shows the parts related to the embodiment of the present invention. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0091] See also Figure 5 In another embodiment of the present application, a cow estrus detection system 100 based on Transformer spectrogram feature learning is provided, the system comprising a data acquisition module 101, a data annotation module 102, a data enhancement module 103, a learning model construction module 104 and a detection model construction module 105;

[0092] The data collection module 101 is used to collect cow sound data, divide the sound data into sound segments of equal length, and perform noise reduction processing on all sound segments;

[0093] The data annotation module 102 is used to annotate the noise-reduced sound clips, and annotate the sound event categories and start and end times in the sound clips;

[0094] The data enhancement module 103 is used to convert all the noise-reduced sound clips into mel spectrograms and perform data enhancement on the mel spectrograms;

[0095] The learning model building module 104 is used to build a Transformer-based spectrogram feature learning model, wherein the Transformer-based spectrogram feature learning model uses a multi-head self-attention mechanism to learn the Mel spectrogram after data enhancement to obtain the deep features of the Mel spectrogram;

[0096] The detection model construction module 105 is used to construct a cow estrus sound event detection model based on the channel attention mechanism. The cow estrus sound event detection model takes the deep features of the Mel spectrogram as input, classifies and connects the time domain clues in the deep features through deep learning, realizes cow estrus sound event detection, and finally outputs the time and probability of the estrus event.

[0097] It should be noted that the cow estrus detection system based on Transformer spectrogram feature learning of the present invention corresponds one to one with the cow estrus detection method based on Transformer spectrogram feature learning of the present invention. The technical features and beneficial effects described in the embodiment of the cow estrus detection method based on Transformer spectrogram feature learning are applicable to the embodiment of the cow estrus detection based on Transformer spectrogram feature learning. For specific contents, please refer to the description in the embodiment of the method of the present invention, which will not be repeated here. It is hereby declared.

[0098] In addition, in the implementation of the cow estrus detection system based on Transformer spectrogram feature learning in the above embodiment, the logical division of each program module is only an example. In actual applications, the above functions can be assigned to different program modules as needed, for example, for the convenience of corresponding hardware configuration requirements or software implementation. That is, the internal structure of the cow estrus detection system based on Transformer spectrogram feature learning is divided into different program modules to complete all or part of the functions described above.

[0099] See also Figure 6In one embodiment, an electronic device for implementing a cow estrus detection method based on Transformer spectrogram feature learning is provided. The electronic device 200 may include a first processor 201, a first memory 202 and a bus, and may also include a computer program stored in the first memory 202 and executable on the first processor 201, such as cow estrus detection based on Transformer spectrogram feature learning 203.

[0100] The first memory 202 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the first memory 202 can be an internal storage unit of the electronic device 200, such as a mobile hard disk of the electronic device 200. In other embodiments, the first memory 202 can also be an external storage device of the electronic device 200, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 200. Further, the first memory 202 can also include both an internal storage unit of the electronic device 200 and an external storage device. The first memory 202 can not only be used to store application software and various types of data installed in the electronic device 200, such as the code of the cow estrus detection 203 based on Transformer spectrogram feature learning, but also can be used to temporarily store data that has been output or is to be output.

[0101] In some embodiments, the first processor 201 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The first processor 201 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes various functions and processes data of the electronic device 200 by running or executing programs or modules stored in the first memory 202, and calling data stored in the first memory 202.

[0102] Figure 6 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 6The structure shown does not constitute a limitation on the electronic device 200 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0103] The cow estrus detection 203 based on Transformer spectrogram feature learning stored in the first memory 202 of the electronic device 200 is a combination of multiple instructions. When running in the first processor 201, it can achieve:

[0104] Collect the sound data of the cow, divide the sound data into sound segments of equal length, and perform noise reduction on all the sound segments;

[0105] Mark the sound event categories and start and end times in the noise-reduced sound clips;

[0106] Convert all denoised sound clips into Mel spectrograms and perform data augmentation on the Mel spectrograms;

[0107] Constructing a Transformer-based spectrogram feature learning model, wherein the Transformer-based spectrogram feature learning model uses a multi-head self-attention mechanism to learn the data-enhanced Mel spectrogram to obtain deep features of the Mel spectrogram;

[0108] A cow estrus sound event detection model is constructed based on the channel attention mechanism. The cow estrus sound event detection model takes the deep features of the Mel spectrogram as input, classifies and concatenates the time domain clues in the deep features through deep learning, realizes cow estrus sound event detection, and finally outputs the time and probability of the estrus event.

[0109] Furthermore, if the module / unit integrated in the electronic device 200 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0110] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0111] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A cow estrus detection method based on Transformer spectrogram feature learning, characterized in that: The steps include: Collect the sound data of the cow, divide the sound data into sound segments of equal length, and perform noise reduction on all the sound segments; Label the denoised sound clips to form a standard labeled data set; Convert the sound clips of the standard dataset into Mel spectrograms and perform data augmentation on the Mel spectrograms; Constructing a Transformer-based spectrogram feature learning model, wherein the Transformer-based spectrogram feature learning model uses a multi-head self-attention mechanism to learn the data-enhanced Mel spectrogram to obtain deep features of the Mel spectrogram; A cow estrus sound event detection model based on a channel attention mechanism is constructed. The cow estrus sound event detection model takes the deep features of the mel spectrogram as input, classifies and concatenates the time domain clues in the deep features through deep learning, realizes cow estrus sound event detection, and finally outputs the time and probability of the estrus event. The Transformer-based spectrogram feature learning model uses a multi-head self-attention mechanism to learn the data-enhanced Mel spectrogram to obtain the deep features of the Mel spectrogram, specifically: First, the Mel-size spectrogram is divided into spectrogram blocks of a preset size in a non-overlapping manner. Each spectrogram block is linearly transformed into a one-dimensional vector of D dimension. There are a total of U one-dimensional vectors of D dimension. The U one-dimensional vectors of D dimension are concatenated to obtain an input matrix of dimension [U, D]. Subsequently, the input matrix is ​​added to the matrix recording the corresponding position information of each one-dimensional vector in the spectrogram to realize the position encoding of the one-dimensional vector in the input matrix; Secondly, the position-encoded matrix is ​​input into multiple multi-head self-attention mechanism modules for calculation. The calculation formula of the self-attention mechanism is: Where Q, K, and V represent the query vector, key vector, and value vector in the self-attention mechanism, respectively, which are obtained by passing the input matrix through a fully connected layer. Attention(Q, K, V) is the output result of the self-attention mechanism. k is the dimension of K, K T represents the transpose of the key vector K, and softmax is the activation function; Finally, the output of the self-attention mechanism is a matrix of size [U, D]. The multi-head self-attention module concatenates the output matrices of multiple self-attention mechanisms, and finally restores the concatenated matrices to a matrix of size [U, D] through linear transformation to obtain deep features that describe global correlations. The calculation process of the cow estrus sound event detection model is as follows: First, the deep features with dimension [U, D] are reshaped to [H′, W′, D], so that the time domain cues and frequency domain cues correspond to each other on a two-dimensional level, where H′ represents the number of frequency domain cues and W′ represents the number of time domain cues. Then, the reshaped deep features are input into the channel attention module for operation to obtain a 1×1×D matrix, which represents the weight value of the importance of each channel of the deep feature. The channel corresponds to the dimension of the one-dimensional vector of the input matrix. The corresponding channel is multiplied by the corresponding weight value to obtain the channel-enhanced deep feature; Finally, the frequency domain dimension of the channel-enhanced deep features is fused through average pooling to obtain a matrix of dimension [W′, D] that only represents the time domain clues. The time domain clues are classified to obtain a probability matrix of dimension [W′, C] that represents each event category of the time domain clues, which is the sound event detection result, where C is the number of categories finally classified. The binary cross entropy loss function during the training of the Transformer-based spectrogram feature learning model and the cow estrus sound event detection model is: Among them, H p represents the loss function, y i Represents the label vector. If the true label is marked as a certain event category, then the category is in y i The corresponding position in the vector is marked as 1, otherwise it is marked as 0. i ) represents the probability that the predicted temporal clue is of the i-th category.

2. The cow estrus detection method based on Transformer spectrogram feature learning according to claim 1 is characterized in that: The noise reduction process for all sound clips is specifically as follows: A noise fragment sample is input, the spectrum of the noise fragment is obtained by calculation, and then the characteristic frequency component corresponding to the spectrum of the noise fragment is subtracted from the collected cow sound data to complete the noise reduction process.

3. The cow estrus detection method based on Transformer spectrogram feature learning according to claim 1 is characterized in that: The annotation is to mark the sound event category and start and end time in the sound clip for subsequent model construction and training.

4. The cow estrus detection method based on Transformer spectrogram feature learning according to claim 1 is characterized in that: The data enhancement of the Mel spectrogram includes: First, the mel spectrogram is processed by the confusion method, and then the frequency domain and time domain masking methods are used to process the mel spectrogram. The specific process is as follows: Confusion method: The spectrograms converted from the two sound clips are superimposed in proportion, and the matrices corresponding to the two sound clips are also superimposed in the same proportion to expand the number of training samples; Frequency domain and time domain masking method: By masking the frequency domain channel and the time domain channel, the local information of the Mel spectrogram is removed to increase the complexity of the data.

5. A cow estrus detection system based on Transformer spectrogram feature learning, characterized in that: A cow estrus detection method based on Transformer spectrogram feature learning applied to any one of claims 1-4, comprising a data acquisition module, a data annotation module, a data enhancement module, a learning model construction module and a detection model construction module; The data collection module is used to collect cow sound data, divide the sound data into sound segments of equal length, and perform noise reduction processing on all sound segments; The data annotation module is used to annotate the noise-reduced sound segments to form a labeled standard data set; The data enhancement module is used to convert all the noise-reduced sound clips into Mel spectrograms and perform data enhancement on the Mel spectrograms; The learning model building module is used to build a Transformer-based spectrogram feature learning model, and the Transformer-based spectrogram feature learning model uses a multi-head self-attention mechanism to learn the Mel spectrogram after data enhancement to obtain the deep features of the Mel spectrogram; The detection model construction module is used to construct a cow estrus sound event detection model based on the channel attention mechanism. The cow estrus sound event detection model takes the deep features of the cow's sound as input, classifies and connects the time domain clues in the deep features through deep learning, realizes the cow estrus sound event detection, and finally outputs the time and probability of the estrus event.

6. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the cow estrus detection method based on Transformer spectrogram feature learning as described in any one of claims 1-4.

7. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the cow estrus detection method based on Transformer spectrogram feature learning as described in any one of claims 1 to 4 is implemented.

Images