A method, apparatus and terminal device for monitoring biodiversity

Abstract

The application provides a kind of biodiversity monitoring method, device and terminal equipment, suitable for data processing technical field, the method comprises: obtaining biological image information and biological sound information;The biological sound information is extracted, and a plurality of biological audio feature information is obtained;The biological image information is extracted, and a plurality of biological image feature information is obtained;The biological audio feature information and the biological image feature information are aligned, and biological audio alignment features and biological image alignment features are obtained;The biological audio alignment features and biological image alignment features are respectively strengthened, and biological audio monitoring features and biological image monitoring features are obtained;According to the biological audio monitoring features and biological image monitoring features, biological monitoring is carried out.The application combines audio information and image information, thereby improving the accuracy and comprehensiveness of real-time monitoring of biodiversity.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to methods, devices and terminal equipment for biodiversity monitoring. Background Technology

[0002] In diverse ecological environments, different organisms are distributed and interdependent, maintaining the dynamic balance of the entire ecosystem. To prevent this dynamic balance from being suddenly disrupted, leading to biodiversity loss or even species extinction, routine biodiversity monitoring is essential. In particular, monitoring the specific behaviors of organisms during specific growth periods or seasons provides reliable reference data for studying the growth and reproduction of species populations, as well as the development trends and protection measures of the ecological environment. This is beneficial for maintaining ecosystem stability and promoting sustainable development.

[0003] Current methods for biodiversity monitoring typically employ traditional acoustic analysis techniques. These methods involve statistically analyzing various biological audio recordings collected in the ecological environment to identify the calls of different species within the audio recordings, and then further analyzing these calls. Alternatively, image processing methods can be used to classify images taken in the wild and identify the external morphology or behavioral characteristics of different species within the images.

[0004] However, when using traditional acoustic analysis methods or image processing to monitor biodiversity, existing technologies are prone to missing or inaccurate identification of biological behaviors, and there is little room for improvement in identification accuracy, which increases the difficulty of accurately monitoring biological behaviors in research work. Summary of the Invention

[0005] In view of this, embodiments of this application provide a biodiversity monitoring method, apparatus, and terminal equipment, aiming to solve the problems of missed monitoring or inaccurate monitoring in the process of monitoring biological behavior in the prior art.

[0006] The first aspect of this application provides a biodiversity monitoring method, including:

[0007] Acquire biological image information and biological sound information;

[0008] Based on multiple preset biological sound feature extraction windows, preset filtering windows, and preset window processing functions, the biological sound information is feature extracted to obtain multiple biological audio feature information.

[0009] Based on multiple preset biological image feature extraction matrices, feature extraction is performed on the biological image information to obtain multiple biological image feature information;

[0010] Based on preset basis vectors, preset encoding vectors, and preset spatial projection functions, the biological audio feature information and the biological image feature information are aligned to obtain biological audio alignment features and biological image alignment features.

[0011] The biological audio alignment features and biological image alignment features are enhanced respectively to obtain biological audio monitoring features and biological image monitoring features;

[0012] The organism is monitored based on the biological audio monitoring characteristics and biological image monitoring characteristics.

[0013] A second aspect of this application provides a biodiversity monitoring device, comprising:

[0014] The information acquisition module is used to acquire biological image information and biological sound information;

[0015] The biological audio feature information generation module is used to extract features from the biological sound information based on multiple preset biological sound feature extraction windows, preset filtering windows, and preset window processing functions to obtain multiple biological audio feature information.

[0016] The biological image feature information generation module is used to extract features from the biological image information based on multiple preset biological image feature extraction matrices to obtain multiple biological image feature information.

[0017] The biological audio alignment feature and biological image alignment feature generation module is used to perform alignment processing on the biological audio feature information and the biological image feature information based on preset basis vectors, preset encoding vectors and preset spatial projection functions to obtain biological audio alignment features and biological image alignment features.

[0018] The biological audio monitoring feature and biological image monitoring feature generation module is used to enhance the biological audio alignment feature and biological image alignment feature respectively to obtain the biological audio monitoring feature and biological image monitoring feature.

[0019] The biological monitoring module is used to monitor organisms based on the biological audio monitoring characteristics and biological image monitoring characteristics.

[0020] A third aspect of this application provides a terminal device, which includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the biodiversity monitoring method described in the first aspect above.

[0021] A fourth aspect of this application provides a computer-readable storage medium, comprising: a computer program stored thereon, which, when executed by a processor, implements the steps of the biodiversity monitoring method as described in the first aspect above.

[0022] The beneficial effects of this application embodiment compared with the prior art are as follows: This application extracts and aligns features from image and sound information obtained in the ecological environment, and enhances the aligned features for biodiversity monitoring. This enables the combination of audio and image information, using audio information to compensate for the shortcomings of image information in biodiversity identification, and using image information to compensate for the limitations of audio information in biodiversity monitoring. It also enhances the expressive power and reliability of monitoring data, thereby effectively improving the accuracy and comprehensiveness of real-time biodiversity monitoring, and enabling the selection of biodiversity information that is more important for research work for researchers to conduct ecological protection research. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram illustrating the implementation process of the biodiversity monitoring method provided in Embodiment 1 of this application;

[0025] Figure 2 This is a schematic diagram illustrating the implementation process of the biodiversity monitoring method provided in Embodiment 2 of this application;

[0026] Figure 3 This is a schematic diagram illustrating the implementation process of the biodiversity monitoring method provided in Embodiment 3 of this application;

[0027] Figure 4 This is a schematic diagram illustrating the implementation process of the biodiversity monitoring method provided in Embodiment 4 of this application;

[0028] Figure 5 This is a schematic diagram illustrating the implementation process of the biodiversity monitoring method provided in Embodiment 5 of this application;

[0029] Figure 6 This is a schematic diagram illustrating the implementation process of the biodiversity monitoring method provided in Embodiment Six of this application;

[0030] Figure 7 This is a schematic diagram illustrating the implementation process of the biodiversity monitoring method provided in Embodiment 7 of this application;

[0031] Figure 8 This is a schematic diagram of the biodiversity monitoring device provided in the embodiments of this application;

[0032] Figure 9 This is a schematic diagram of the terminal device provided in the embodiments of this application. Detailed Implementation

[0033] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0034] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0035] Figure 1 A flowchart illustrating the implementation of the biodiversity monitoring method provided in Embodiment 1 of this application is shown, and is described in detail below:

[0036] Step S101: Obtain biological image information and biological sound information.

[0037] In this embodiment, biological image information can be acquired omnidirectionally using camera equipment positioned within the ecological environment. Biological sound information can be acquired omnidirectionally using an array of microphones distributed throughout the ecological environment.

[0038] Step S102: Based on multiple preset biological sound feature extraction windows, preset filtering windows, and preset window processing functions, feature extraction is performed on the biological sound information to obtain multiple biological audio feature information.

[0039] In this embodiment, the preset biological sound feature extraction window can be a matrix with different dimensions or scales introduced into the analysis and calculation process. This matrix is ​​used to extract features of biological sound information through continuously changing dimensions or scales, thereby obtaining biological audio feature information of different dimensions or scales. The preset biological sound feature extraction window can also be an interval window used to sample biological sound information, truncating the sound information to achieve sampling. The preset filtering window can be used to filter the sampled sound information to suppress high-frequency harmonics. The preset window processing function can be set based on the Hamming window function to smooth the sound information and reduce spectral leakage. Both the preset biological sound feature extraction window and the preset filtering window can be manually set.

[0040] Step S103: Based on multiple preset biological image feature extraction matrices, feature extraction is performed on the biological image information to obtain multiple biological image feature information.

[0041] In this embodiment, the preset biological image feature extraction window refers to introducing matrices of different dimensions or scales into the analysis and calculation process. These matrices are used to extract features from biological image information through continuously changing dimensions or scales, thereby obtaining biological image feature information of different dimensions or scales. The preset biological image features can be manually set. They can be generated by performing multiple convolution calculations between the biological image feature extraction matrix and the biological image information, and then generating the biological image feature information based on the calculation results. It is understood that the biological image feature information is presented in a matrix format.

[0042] Step S104: Based on preset basis vectors, preset encoding vectors, and preset spatial projection functions, the biological audio feature information and the biological image feature information are aligned to obtain biological audio alignment features and biological image alignment features.

[0043] In this embodiment, due to the significant differences in visualization, semantics, and information density between audio and text data, misalignment issues arise between feature data. Basis vectors and encoding vectors are needed to extract and transform the semantic information of both types of data. Then, a spatial projection function projects the semantic information of both types of data into the same feature space to simultaneously generate aligned feature vectors. This ensures semantic consistency in the feature representations of the two types of data, facilitating subsequent processing using consistent feature data and improving the accuracy and reliability of biodiversity monitoring. The aligned audio and image features are thus called biological audio aligned features and biological image aligned features. The preset basis vectors, preset encoding vectors, and preset spatial projection functions can be manually set.

[0044] Step S105: Enhance the biological audio alignment features and biological image alignment features respectively to obtain biological audio monitoring features and biological image monitoring features.

[0045] In this embodiment, biological audio alignment features and biological image alignment features can be further extracted using preset audio masks and preset image masks. Then, the extracted features are used to construct a spatial structure. This spatial structure can be based on the propagation mechanism of nodes and connecting arcs. By calculating and adjusting the correlation between nodes and connecting arcs in the spatial structure, the local features of audio and images are enhanced at the global level. The enhanced features are the biological audio monitoring features and biological image monitoring features, which are used to improve the global feature representation of audio and image features, thereby reducing the interference of features with low importance on the biological monitoring process, and improving the comprehensiveness and reliability of biodiversity monitoring.

[0046] Step S106: Monitor the organism based on the biological audio monitoring characteristics and biological image monitoring characteristics.

[0047] In this embodiment, biological behavior can be monitored by matching biological audio monitoring features and biological image monitoring features with preset biological behavior sample features. The preset biological behavior sample features can be extracted and summarized based on biological behavior sound data and biological behavior image data collected in previous research work, and are used to characterize different behaviors of organisms in the ecological environment, such as courtship, mating, nest building, egg laying, and brooding.

[0048] Understandably, in existing technologies, processing only audio information to monitor biological calls cannot effectively identify the behavior of organisms in static situations, and effective behavioral information cannot be obtained when the microphone is damaged. On the other hand, processing only the captured images cannot be used to track the behavior of organisms in the images when special circumstances such as the camera being blocked, the equipment being damaged, or blind spots appearing in the monitoring range occur. This fails to fill the gaps in biodiversity monitoring.

[0049] The biodiversity monitoring method provided in this application extracts and aligns features from image and sound information obtained from the ecological environment. The aligned features are then enhanced for use in biodiversity monitoring. This method combines audio and image information, using audio to compensate for the limitations of image information in biodiversity identification and vice versa. It also enhances the expressive power and reliability of monitoring data, thereby effectively improving the accuracy and comprehensiveness of real-time biodiversity monitoring. This allows researchers to identify biodiversity information that is more important for their ecological conservation studies.

[0050] Figure 2The flowchart illustrating the implementation of the biodiversity monitoring method provided in Embodiment 2 of this application is shown. The difference between this method and Embodiment 1 is that step S102 specifically includes:

[0051] Step S201: Perform analog-to-digital conversion on the biological sound information to obtain biological sound digital information.

[0052] In this embodiment, the biological sound information can be sampled first, that is, sampled values ​​are taken at discrete time points for continuous biological sound information in the time domain, where the interval between discrete time points can be set manually. Then, the sampled biological sound information is quantized, that is, the continuous values ​​of biological sound information in discrete time are converted into discrete values ​​of biological sound information in discrete time, where each discrete value can be selected from a finite set of continuous biological sound information values. Finally, the quantized biological sound information is encoded, that is, the discrete values ​​of biological sound information in discrete time are converted into a multi-bit binary sequence, where the number of bits can be set manually.

[0053] Step S202: According to the preset biological sound feature extraction window, the biological sound digital information is extracted to obtain multiple biological audio frame information.

[0054] In this embodiment, the size of the preset biological sound feature extraction window in the time domain can be manually set. Through the biological sound feature extraction window, the biological sound digital information is truncated into multiple short segments, each segment becoming a frame, or a biological audio frame. It is understood that the truncating does not have to be back-to-back, but rather involves overlapping portions in the time domain. This is to prevent the ends of the biological audio frame from being severely weakened and causing local feature loss during subsequent processing using a preset window function. The time difference between the starting positions of two adjacent biological audio frames is called frame shift.

[0055] Step S203: Use the multiple biological audio frame information as independent variables of a preset window processing function to calculate the biological audio frame variable information.

[0056] In this embodiment, the preset window processing function can be based on a Hamming window function, a zero-power window function in the time domain, or a Gaussian window function. When the window processing function is based on a Hamming window function, and the biological audio frame information is used as the independent variable of the window processing function, the biological audio frame information is multiplied by the Hamming window function to avoid the Gibbs effect, and the calculated value is the biological audio frame variable information.

[0057] Step S204: Perform Fourier transform on the biological audio frame variable information to obtain the biological audio frame frequency domain information.

[0058] In this embodiment, a Fourier transform is performed on all the obtained biological audio frame variable information to convert the biological audio frame variable information from the time domain to the frequency domain, and the calculated result is the biological audio frame frequency domain information.

[0059] Step S205: According to the preset filtering window, the frequency domain information of the biological audio frame is filtered to obtain the spectrum information of the biological audio frame.

[0060] In this embodiment, the preset filtering window can be manually set, or it can be a scale window of a Mel filter. The filtering operation on the frequency domain information of biological audio frames can be performed by first stacking multiple biological audio frame frequency domain information along one dimension to obtain a stacked biological audio frame spectrum, and then filtering the biological audio frame spectrum through multiple scale windows of a Mel filter. The filtered information is the biological audio frame spectrum information, thus realizing the filtering operation on the frequency domain information of biological audio frames.

[0061] Step S206: Perform frequency domain analysis and normalization on the spectral information of the biological audio frame to obtain multiple biological audio feature information.

[0062] In this embodiment, frequency domain analysis calculation can be performed on the Mel frequency cepstral coefficients of the biological audio frame spectrum information. The Mel frequency cepstral coefficients after analysis and calculation are normalized to eliminate the influence of factors such as different recording devices on audio characteristics, and at the same time to avoid the values ​​exceeding the dimensional range of the values ​​that the computer can calculate.

[0063] The biodiversity monitoring method provided in this application can adapt to biological sound information with different signal-to-noise ratios by using different biological sound feature extraction windows and window processing functions. It converts biological sound information with different signal-to-noise ratios into spectral characteristics in the frequency domain and filters the spectral characteristics through a filtering window, effectively reducing the interference of noise in biological sound information on important biological sound features, thereby ensuring the accuracy and robustness of sound feature extraction from biological sound information.

[0064] Figure 3 The flowchart illustrating the implementation of the biodiversity monitoring method provided in Embodiment 3 of this application is shown. The difference between this method and Embodiment 1 is that step S103 specifically includes:

[0065] Step S301: Randomly generate multiple strings of numbers that conform to a Gaussian distribution.

[0066] In this embodiment, the Box-Muller method can be used to generate Gaussian distributed random numbers, i.e., a string of numbers that conforms to a Gaussian distribution, by utilizing the relationship between two independent uniformly distributed random variables.

[0067] Step S302: Convert the format of the multiple number strings respectively to obtain multiple Gaussian numerical matrices.

[0068] In this embodiment, the number of rows and columns of the matrix can be set, and then the number string can be arranged according to the number of rows and columns to generate a Gaussian numerical matrix.

[0069] Step S303: Perform multiple convolution calculations between multiple preset biological image feature extraction matrices and the biological image information to obtain multiple initial biological image feature maps.

[0070] In this embodiment, the preset biological image feature extraction matrix can be manually set. The number of convolution calculations can be 8 or 10. The result of multiple convolution calculations is the initial biological image feature map, which is used for subsequent deep feature extraction.

[0071] Step S304: Perform inner product operations on the multiple initial biological image feature maps and the multiple Gaussian numerical matrices respectively to obtain multiple intermediate biological image feature maps.

[0072] In this embodiment, the initial biological image feature map is inner-matrixed with a Gaussian numerical matrix. The result of this inner-matrix operation is an intermediate biological image feature map, presented in matrix form. This matrix is ​​used to distribute the weights of all values ​​in the initial biological image feature map using a normal distribution, thereby highlighting the important features in the initial biological image feature map and facilitating subsequent consideration of these important features. It is understood that a normal distribution is a bell-shaped curve; the closer to the center, the larger the value. Therefore, it can simultaneously smooth the values ​​in the middle rows and columns and the edge rows and columns of the initial biological image feature map, thereby achieving noise reduction and reducing the impact of irrelevant feature values ​​in the initial biological image feature map on the overall biological image features.

[0073] Step S305: Calculate the element average of the multiple intermediate biological image feature maps to generate multiple target biological image feature maps.

[0074] In this embodiment, by extracting all elements from each intermediate biological image feature map and then calculating the average value of all elements in each intermediate biological image feature map, multiple one-dimensional vectors, namely target biological image feature maps, are generated. These vectors are used to extract more abstract and stable feature representation elements while retaining the original biological image features, thereby reducing the dimensionality of the intermediate biological image feature maps, reducing the complexity of subsequent analysis and calculation processes, and alleviating the computational burden on the processor.

[0075] Step S306: Normalize the multiple target biological image feature maps to generate biological image feature information.

[0076] In this embodiment, the normalization technique can be min-max normalization, Z-score normalization, maximum absolute value scaling (scaling the data to the [-1, 1] interval), or Robust Scaling (scaling using the median and quartile ranges). The result of normalizing the target biological image feature map is the biological image feature information, used to avoid numerical results generated during the calculation process exceeding the numerical dimension range that the computer can process, ensuring the stability and reliability of the calculation process.

[0077] The biodiversity monitoring method provided in this application generates a matrix with Gaussian distribution characteristics to process biological image features. This process effectively smooths the values ​​in biological image features while preserving the main image features, reduces the negative impact of high-frequency noise on the calculation of biological image features, and thus improves the effectiveness and accuracy of feature extraction from biological image information, thereby enhancing the accuracy and reliability of biological monitoring.

[0078] Figure 4 The flowchart of the implementation of the biodiversity monitoring method provided in Embodiment 4 of this application is shown. The difference between this method and Embodiment 1 above is that the preset encoding vector includes a preset feature alignment encoding vector and a preset feature projection encoding vector.

[0079] Step S104 specifically includes:

[0080] Step S401: Based on the biological audio feature information, biological image feature information, and preset basis vectors, obtain the initial biological audio alignment feature and the initial biological image alignment feature.

[0081] In this embodiment, the preset basis vectors can be manually set, and there can be multiple vectors. These vectors can be used to represent different scales to perform alignment processing on biological audio features and biological image features at different scales. Multiple basis vectors can be convolved with biological audio features and biological image features respectively to obtain biological audio features and biological image features at multiple scales, i.e., initial biological audio alignment features and initial biological image alignment features. This allows for deeper extraction of biological audio features and biological image features, while avoiding feature alignment failures in subsequent calculations, thus improving the matching effectiveness and reliability of biological audio and biological image information.

[0082] Step S402: Based on the initial biological audio alignment features, the initial biological image alignment features, and the preset feature alignment encoding vector, the first intermediate biological audio alignment features and the first intermediate biological image alignment features are obtained.

[0083] In this embodiment, the preset feature alignment encoding vector can be manually set. It can be achieved by performing inner product operations between the feature alignment encoding vector and the initial biological audio alignment feature and the initial biological image alignment feature, respectively, to further extract features from the initial biological audio alignment feature and the initial biological image alignment feature. This achieves deep extraction of alignment features for biological audio features and biological image features while increasing the differences between multiple biological audio alignment features and multiple biological image alignment features, thus amplifying the details in the biological sound information and biological image information. The calculation results are the first intermediate biological audio alignment feature and the first intermediate biological image alignment feature.

[0084] Step S403: Based on the biological audio feature information, biological image feature information, and preset feature projection encoding vector, obtain the second intermediate biological audio alignment feature and the second intermediate biological audio alignment feature.

[0085] In this embodiment, the preset feature projection encoding vector can be manually set. It can be obtained by convolving the feature projection encoding vector with biological audio features and biological image features, respectively, to quantify the values ​​after projecting the biological audio features and biological image features into a nonlinear space. The calculated second intermediate biological audio alignment feature will then be used as a variable in subsequent calculations within the nonlinear space.

[0086] Step S404: Based on the first intermediate biological audio alignment feature, the second intermediate biological audio alignment feature, the first intermediate image audio alignment feature, the second intermediate biological image alignment feature, and a preset spatial projection function, obtain the biological audio alignment feature and the biological image alignment feature.

[0087] In this embodiment, the preset spatial projection function can be the Tanh function, the ReLU function, or the APReLU function, which is used to project biological audio features and biological image features into a nonlinear space for further analysis and calculation. The results of the further analysis and calculation are the biological audio alignment features and the biological image alignment features.

[0088] The biodiversity monitoring method provided in this application further extracts biological audio and image features at a deeper level, aligning them at different granularities to effectively avoid feature alignment failures during computation. The aligned biological audio and image features are then spatially projected and quantified to obtain spatially projected feature values. These quantized values, projected into a nonlinear space, are then used for further analysis and calculation of the biological audio and image features, enhancing their feature representation and preventing the omission of detailed features in biological sound and image information. This improves the accuracy and reliability of timely biological monitoring.

[0089] Figure 5 The flowchart illustrating the implementation of the biodiversity monitoring method provided in Embodiment 5 of this application is shown. The difference between this method and Embodiment 4 above is that step S404 specifically includes:

[0090] Step S501: Perform an inner product operation on the first intermediate biological audio alignment feature and the second intermediate biological audio alignment feature to obtain the third intermediate biological audio alignment feature.

[0091] In this embodiment, the first intermediate biological audio alignment feature and the second intermediate biological audio alignment feature are subjected to an inner product operation. This is used to quantify the displacement of the first intermediate biological audio alignment feature in the nonlinear space through the second intermediate biological audio alignment feature, so as to further increase the distance between each biological audio feature and thus highlight the feature representation of each biological audio feature in the nonlinear space.

[0092] Step S502: Perform an inner product operation on the first intermediate biological image alignment feature and the second intermediate biological image alignment feature to obtain the third intermediate biological image alignment feature.

[0093] In this embodiment, the first intermediate biological image alignment feature and the second intermediate biological image alignment feature are subjected to an inner product operation. This is used to quantify the displacement of the first intermediate biological image alignment feature in the nonlinear space through the second intermediate biological image alignment feature, so as to further increase the distance between each biological image feature and thus highlight the feature representation of each biological image feature in the nonlinear space.

[0094] Step S503: Based on the third intermediate biological audio alignment feature, the third intermediate biological image alignment feature, and the preset spatial projection function, obtain the biological audio feature projection variable and the biological image feature projection variable.

[0095] In this embodiment, the third intermediate biological audio alignment feature and the third intermediate biological image alignment feature can be used as independent variables of the spatial projection function, and the calculated function values ​​are the biological audio feature projection variable and the biological image feature projection variable.

[0096] Step S504: The biological audio feature projection variables and biological image feature projection variables are weighted and summed to obtain biological audio alignment features and biological image alignment features.

[0097] In this embodiment, the biological audio feature projection variables and the biological image feature projection variables are weighted and summed respectively to perform fusion processing on the biological audio features and biological image features. The result of the fusion processing is the biological audio alignment feature and the biological image alignment feature.

[0098] The biodiversity monitoring method provided in this application increases the distance between various biological sound features and biological image features by performing depth calculations on the aligned features. This enhances the feature representation of biological sound features and biological image features in nonlinear space, effectively preventing the omission of important details in biological sound information and biological image information, thereby improving the accuracy and sensitivity of biodiversity monitoring. By weighted summing of feature variables in nonlinear space, the extracted features are summarized. This ensures that effective features are fully considered during the calculation process while saving computer storage space, facilitating the analysis and calculation of more biological sound information and biological image information.

[0099] Figure 6 The flowchart illustrating the implementation of the biodiversity monitoring method provided in Embodiment Six of this application is shown. The difference between this method and Embodiment One described above is that step S105 specifically includes:

[0100] Step S601: Based on the preset audio feature mask information, the biological audio alignment features are numerically filtered to obtain multiple biological behavioral sound features; the biological behavioral sound features include biological behavioral sound feature nodes and biological behavioral sound feature connecting arcs.

[0101] In this embodiment, the preset audio feature mask information can be set by the person, or it can be obtained by deep feature extraction and spatial transformation of audio information of various types of biological behaviors extracted from past studies. It is used to characterize the audio features of biological behaviors that are of interest to researchers in biodiversity samples. The biological audio alignment features calculated from the acquired biological sound information are filtered using the audio feature mask information. This is used to select the biological audio features corresponding to the biological behaviors that researchers are interested in from the biological audio alignment features. The biological audio features are then constructed into a spatial structure based on the principle of spatial correlation. This can be based on the numerical values ​​in the biological audio features to construct nodes of the spatial structure, i.e., biological behavior sound feature nodes, and based on the temporal sequence of the numerical values ​​in the biological audio features to construct the connecting arcs of the nodes of the spatial structure, i.e., biological behavior sound feature connecting arcs.

[0102] Step S602: Based on the preset image feature mask information, the biological image alignment features are numerically filtered to obtain multiple biological behavior image features; the biological behavior image features include biological behavior image feature nodes and biological behavior image feature connecting arcs.

[0103] In this embodiment, the preset image feature mask information can be set by the person, or it can be obtained by deep feature extraction and spatial transformation of image information of various types of biological behaviors extracted from past studies. It is used to characterize the image features of biological behaviors that are of great interest to researchers in biodiversity samples. The biological image alignment features calculated from the acquired biological image information are filtered using the image feature mask information. This is used to select the biological image features corresponding to the biological behaviors of interest to researchers from the biological image alignment features. The biological image features are then constructed into a spatial structure based on the principle of spatial correlation. This can be based on the numerical values ​​in the biological image features to construct the nodes of the spatial structure, i.e., biological behavior image feature nodes, and based on the temporal sequence of the numerical values ​​in the biological image features to construct the connecting arcs of the nodes of the spatial structure, i.e., biological behavior image feature connecting arcs.

[0104] Step S603: Calculate the Euclidean distance between each of the biological behavior sound feature nodes and the multiple biological behavior sound feature connection arcs to obtain the first biological behavior sound feature correlation degree.

[0105] In this embodiment, considering the different correlations between various auditory information related to biological behaviors—for example, the calls made by birds during courtship are significantly different from their calls during juvenile stage—there is a high correlation between the courtship calls of different birds. This correlation can be used to filter out the auditory features of bird courtship calls from a large number of biological behavioral auditory features. Therefore, it is necessary to calculate the Euclidean distance between each auditory feature node and the arc connecting the auditory features to quantify the correlation between the various auditory features. This facilitates the subsequent filtering of highly correlated auditory features for effective monitoring of specific biological behaviors.

[0106] Step S604: Calculate the connection arcs of each of the biological behavior sound features and the Euclidean distances of the multiple biological behavior sound feature nodes to obtain the second biological behavior sound feature correlation degree.

[0107] In this embodiment, considering the spatiotemporal correlation between different sounds of various biological behaviors, this spatiotemporal correlation is represented by the connecting arcs of biological behavior sound features. It is understood that biological sound information acquired in the same spatiotemporal space has spatiotemporal consistency; therefore, the correlation between the connecting arcs of biological behavior sound features is strong. The spatiotemporal correlation between various biological behavior sound features can be quantified by calculating the connecting arcs of biological behavior sound features and the Euclidean distance between multiple biological behavior sound feature nodes. This is used to identify the spatiotemporal occurrence of special biological behaviors, thereby enabling effective monitoring of these behaviors.

[0108] Step S605: Normalize and weighted summate the correlation degree of the first biological behavior sound feature and the correlation degree of the second biological behavior sound feature respectively to obtain the dimension conversion feature of the first biological behavior sound and the dimension conversion feature of the second biological behavior sound.

[0109] In this embodiment, the correlation degree of the first biological behavior sound feature and the correlation degree of the second biological behavior sound feature can be normalized respectively. The normalized values ​​are used as the weight values ​​of the two sets of biological behavior sound features. Based on the weight values, the biological behavior sound features are weighted and summed to obtain the first biological behavior sound dimension conversion feature and the second biological behavior sound dimension conversion feature.

[0110] Step S606: Calculate the Euclidean distance between each of the biological behavior image feature nodes and the multiple biological behavior image feature connection arcs to obtain the first biological behavior image feature correlation degree.

[0111] In this embodiment, considering the different correlations between different images of biological behavior—for example, the behavior of birds during courtship is significantly different from their behavior when they are young and hungry—different birds exhibit highly correlated behavior during courtship. This correlation allows for the selection of biological behavior image features representing bird courtship behaviors from a large pool of biological behavior image features. Therefore, it is necessary to calculate the Euclidean distance between each biological behavior image feature node and the connecting arc of the biological behavior image features to quantify the correlation between the various biological behavior image features. This facilitates the subsequent selection of highly correlated biological behavior image features for effective monitoring of specific biological behaviors. Specifically, one biological behavior image feature node corresponds to multiple first biological behavior image feature correlations, and one biological behavior image feature connecting arc corresponds to one first biological behavior image feature correlation.

[0112] Step S607: Calculate the Euclidean distance between each of the biological behavior image feature connection arcs and the multiple biological behavior image feature nodes to obtain the second biological behavior image feature correlation degree.

[0113] In this embodiment, considering the different correlations between various biological behavior image information, such as the significantly different behaviors exhibited by birds during courtship compared to their behavior when juveniles are hungry, different bird behaviors during courtship exhibit high correlations. Therefore, biological behavior image features exhibiting bird courtship behaviors can be selected from numerous biological behavior image features by calculating these correlations. Thus, it is necessary to calculate the Euclidean distance between each biological behavior image feature node and the connecting arc of the biological behavior image feature to quantify the correlation between various biological behavior image features. This facilitates the subsequent selection of highly correlated biological behavior image features for effective monitoring of specific biological behaviors. Specifically, one biological behavior image feature connecting arc corresponds to multiple second biological behavior image feature correlations, and one biological behavior image feature node corresponds to one second biological behavior image feature correlation.

[0114] Step S608: Normalize and perform weighted summation on the correlation degree of the first biological behavior image features and the correlation degree of the second biological behavior image features respectively to obtain the dimension transformation features of the first biological behavior image and the dimension transformation features of the second biological behavior image.

[0115] In this embodiment, the correlation degree of the first biological behavior image feature and the correlation degree of the second biological behavior image feature can be normalized respectively. The normalized values ​​are used as the weight values ​​of the two sets of biological behavior image features. Based on the weight values, the biological behavior image features are weighted and summed to obtain the first biological behavior image dimension transformation feature and the second biological behavior image dimension transformation feature.

[0116] Step S609: Based on the first biological behavior sound dimension conversion feature, the second biological behavior sound dimension conversion feature, the first biological behavior image dimension conversion feature, the second biological behavior image dimension conversion feature, the preset biological audio feature weight matrix, the preset biological image feature weight matrix, the preset biological audio feature mapping function, and the preset biological image feature mapping function, the biological audio monitoring feature and the biological image monitoring feature are obtained.

[0117] In this embodiment, the first biological behavior sound dimension conversion feature and the second biological behavior sound dimension conversion feature can be fused to obtain the fused biological behavior sound dimension conversion feature. Then, the biological behavior sound dimension conversion feature is multiplied by a preset biological audio feature weight matrix to quantify the importance of each value in the biological behavior sound dimension conversion feature. The multiplication result is then mapped to a multidimensional space through a preset biological audio feature mapping function for further processing to enhance the feature representation of each value in the biological behavior sound dimension conversion feature, widen the gap between each feature value, and enable even the smallest biological behavior features reflected in the sound information to be detected. The process involves fusing the first and second biological behavior image dimensional transformation features to obtain a fused biological behavior image dimensional transformation feature. This fused feature is then multiplied by a preset biological image feature weight matrix to quantify the importance of each value within the dimensional transformation feature. The multiplication result is then mapped to a multidimensional space using a preset biological image feature mapping function to further enhance the feature representation of each value within the dimensional transformation feature, widening the differences between feature values, and enabling the detection of even the most minute biological behavior features in the image information. The preset biological audio feature weight matrix, the preset biological image feature weight matrix, the preset biological audio feature mapping function, and the preset biological image feature mapping function can all be manually defined.

[0118] The biodiversity monitoring method provided in this application further extracts biological audio alignment features and biological image alignment features through audio feature masking information and image feature masking information, respectively. This effectively filters out biological behavioral information irrelevant to biodiversity research and highlights information on specific biological behaviors required for research for analysis and processing. By constructing a spatial structure based on nodes and connecting arcs, the method calculates the content correlation and spatiotemporal correlation between various biological sound features and biological image features, thereby further filtering the biological sound features and biological image features. The deeply filtered biological sound features and biological image features are then mapped to a multidimensional space for analysis and processing to enhance the differences between sound features and image features of different biological behaviors. This achieves feature enhancement of biological audio alignment features and biological image alignment features, improving the accuracy and effectiveness of monitoring specific biological behaviors through sound and images.

[0119] Figure 7 The flowchart illustrating the implementation of the biodiversity monitoring method provided in Embodiment Seven of this application is shown. The difference between this method and Embodiment Six is ​​that step S609 specifically includes:

[0120] Step S701: The first biological behavior sound dimension conversion feature and the second biological behavior sound dimension conversion feature are spliced ​​together to obtain the initial biological audio monitoring feature.

[0121] In this embodiment, the first biological behavior sound dimension conversion feature and the second biological behavior sound dimension conversion feature can be horizontally spliced ​​or vertically spliced, and the resulting matrix is ​​the initial biological audio monitoring feature.

[0122] Step S702: Perform an inner product operation on the initial biological audio monitoring features and the preset first biological audio feature weight matrix to obtain the first intermediate biological audio monitoring features.

[0123] In this embodiment, the preset first biological audio feature weight matrix can be manually set and is used to quantify the importance of each value in the initial biological audio monitoring features, thereby characterizing the importance of the biological audio feature values. The initial biological audio monitoring features are multiplied by the first biological audio feature weight matrix, and the result is used as the first intermediate biological audio monitoring feature, reflecting the importance of each value in the initial biological audio monitoring features.

[0124] Step S703: Perform an inner product operation on the first intermediate biological audio monitoring feature and the preset second biological audio feature weight matrix to obtain the second intermediate biological audio monitoring feature.

[0125] In this embodiment, the preset second biological audio feature weight matrix can be artificially set to quantify the offset of each biological audio monitoring feature in multidimensional space, thereby increasing the difference between each biological audio monitoring feature in multidimensional space and enhancing the feature representation of the biological audio monitoring features. The first intermediate biological audio monitoring feature and the second biological audio feature weight matrix are subjected to an inner product operation, and the result is used as the second intermediate biological audio monitoring feature, facilitating further in-depth analysis and calculation in multidimensional space.

[0126] Step S704: The first biological behavior image dimension conversion feature and the second biological behavior image dimension conversion feature are stitched together to obtain the initial biological image monitoring features.

[0127] In this embodiment, the first biological behavior image dimension conversion feature and the second biological behavior image dimension conversion feature can be horizontally stitched together or vertically stitched together. The resulting matrix is ​​the initial biological image monitoring feature.

[0128] Step S705: Perform an inner product operation on the initial biological image monitoring features and the preset first biological image feature weight matrix to obtain the first intermediate biological image monitoring features.

[0129] In this embodiment, the preset first biological image feature weight matrix can be manually set and is used to quantify the importance of each value in the initial biological image monitoring features, thereby characterizing the importance of the biological image feature values. The initial biological image monitoring features are multiplied by the first biological image feature weight matrix, and the result is used as the first intermediate biological image monitoring feature, reflecting the importance of each value in the initial biological image monitoring features.

[0130] Step S706: Perform an inner product operation on the first intermediate biological image monitoring feature and the preset second biological image feature weight matrix to obtain the second intermediate biological image monitoring feature.

[0131] In this embodiment, the preset second biological image feature weight matrix can be manually set to quantify the offset of each biological image monitoring feature in multidimensional space, thereby increasing the difference between each biological image monitoring feature in multidimensional space and enhancing the feature representation of the biological image monitoring features. The first intermediate biological image monitoring feature and the second biological image feature weight matrix are subjected to an inner product operation, and the result is used as the second intermediate biological image monitoring feature, facilitating further in-depth analysis calculations in multidimensional space.

[0132] Step S707: Based on the second intermediate biological audio monitoring feature, the second intermediate biological image monitoring feature, the preset biological audio feature mapping function, and the preset biological image feature mapping function, the biological audio monitoring feature and the biological image monitoring feature are obtained.

[0133] In this embodiment, the preset biological audio feature mapping function and the preset biological image feature mapping function can be the Sigmoid function, used to map biological audio features and biological image features into a multi-dimensional space, improving the nonlinear analysis effect of biological audio features and biological image features. Alternatively, a second intermediate biological audio monitoring feature can be used as the independent variable of the biological audio feature mapping function, and the calculated function value can be used as the biological audio monitoring feature. Similarly, a second intermediate biological image monitoring feature can be used as the independent variable of the biological image feature mapping function, and the calculated function value can be used as the biological image monitoring feature.

[0134] In this embodiment, the biological audio monitoring features and biological image monitoring features can be further fused to obtain biological monitoring features. Feature fusion can be achieved by stitching together the biological audio monitoring features and biological image monitoring features, or by calculating the average values ​​of the biological audio monitoring features and biological image monitoring features, and using the matrix formed by these average values ​​as the biological monitoring features. This combines biological sound information and biological image information, enabling the integration of audio and image information. Audio information can compensate for the shortcomings of image information in biodiversity identification, and image information can compensate for the limitations of audio information in biodiversity monitoring, thereby enhancing the expressive power and reliability of the monitoring data.

[0135] The biodiversity monitoring method provided in this application maps biological sound features and biological image features into a multi-dimensional space for analysis and processing. This enhances the differences between sound and image features of different biological behaviors, thereby achieving feature enhancement processing of biological audio-visual alignment features and biological image alignment features. The alignment-processed audio and image information improves the reliability of biodiversity monitoring. At the same time, audio information compensates for the deficiencies of image information in biodiversity identification, and image information compensates for the limitations of audio information in biodiversity monitoring. This improves the accuracy and effectiveness of monitoring specific biological behaviors simultaneously through sound and images.

[0136] Corresponding to the method in the above embodiments, Figure 8 A structural block diagram of the biodiversity monitoring device provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown. Figure 8 The example biodiversity monitoring device can be the implementing entity of the biodiversity monitoring method provided in the aforementioned embodiment one.

[0137] Reference Figure 8 The biodiversity monitoring device includes:

[0138] The information acquisition module 810 is used to acquire biological image information and biological sound information;

[0139] The biological audio feature information generation module 820 is used to extract features from the biological sound information based on multiple preset biological sound feature extraction windows, preset filtering windows, and preset window processing functions to obtain multiple biological audio feature information.

[0140] The biological image feature information generation module 830 is used to extract features from the biological image information according to multiple preset biological image feature extraction matrices to obtain multiple biological image feature information.

[0141] The biological audio alignment feature and biological image alignment feature generation module 840 is used to perform alignment processing on the biological audio feature information and the biological image feature information based on preset basis vectors, preset encoding vectors and preset spatial projection functions to obtain biological audio alignment features and biological image alignment features.

[0142] The biological audio monitoring feature and biological image monitoring feature generation module 850 is used to enhance the biological audio alignment feature and biological image alignment feature respectively to obtain the biological audio monitoring feature and biological image monitoring feature.

[0143] The biological monitoring module 860 is used to monitor organisms based on the biological audio monitoring characteristics and biological image monitoring characteristics.

[0144] The process by which each module in the biodiversity monitoring device provided in this application implements its respective function can be specifically referred to the foregoing. Figure 1 The description of Embodiment 1 shown will not be repeated here.

[0145] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0146] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0147] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0148] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0149] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance. It should also be understood that although the terms "first," "second," etc., are used in the text to describe various elements in some embodiments of this application, these elements should not be limited by these terms. These terms are merely used to distinguish one element from another. For example, a first table may be named a second table, and similarly, a second table may be named a first table, without departing from the scope of the various described embodiments. Both the first table and the second table are tables, but they are not the same table.

[0150] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0151] The biodiversity monitoring method provided in this application can be applied to terminal devices such as mobile phones, tablets, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of terminal device.

[0152] For example, the terminal device may be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, an in-vehicle device, a vehicle networking terminal, a computer, a laptop computer, a handheld communication device, a handheld computing device, a satellite wireless device, a wireless modem card, a set-top box (STB), customer premises equipment (CPE), and / or other devices used for communication over a wireless system, as well as next-generation communication systems, such as mobile terminals in 5G networks or mobile terminals in future evolved Public Land Mobile Network (PLMN) networks.

[0153] As an example and not a limitation, when the terminal device is a wearable device, the term "wearable device" can also refer to any device that utilizes wearable technology to intelligently design and develop everyday wearables, such as glasses, gloves, watches, clothing, and shoes. Wearable devices are portable devices worn directly on the body or integrated into a user's clothing or accessories. Wearable devices are not merely hardware devices; they achieve powerful functions through software support, data interaction, and cloud interaction. Broadly defined, wearable smart devices include those with comprehensive functions, large sizes, and the ability to perform complete or partial functions without relying on a smartphone, such as smartwatches or smart glasses, as well as those focused on a specific application function that require interaction with other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0154] Figure 9This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. For example... Figure 9 As shown, the terminal device 9 of this embodiment includes: at least one processor 90 ( Figure 9 (Only one is shown in the image), a memory 91, which stores a computer program 92 that can run on the processor 90. When the processor 90 executes the computer program 92, it implements the steps in the various biodiversity monitoring method embodiments described above, for example... Figure 1 Steps S101 to S106 are shown. Alternatively, when the processor 90 executes the computer program 92, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 8 The functions of modules 810 to 860 are shown.

[0155] The terminal device 9 can be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor 90 and a memory 91. Those skilled in the art will understand that... Figure 9 This is merely an example of terminal device 9 and does not constitute a limitation on terminal device 9. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal device may also include input transmission devices, network access devices, buses, etc.

[0156] The processor 90 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0157] In some embodiments, the memory 91 may be an internal storage unit of the terminal device 9, such as a hard disk or memory of the terminal device 9. The memory 91 may also be an external storage device of the terminal device 9, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device 9. Furthermore, the memory 91 may include both internal and external storage units of the terminal device 9. The memory 91 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 91 can also be used to temporarily store data that has been sent or will be sent.

[0158] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0159] This application also provides a terminal device, which includes at least one memory, at least one processor, and a computer program stored in the at least one memory and executable on the at least one processor. When the processor executes the computer program, it causes the terminal device to implement the steps in any of the above method embodiments.

[0160] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0161] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0162] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0163] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0164] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0165] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0166] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method of biodiversity monitoring, characterized in that, The method comprises the following steps: acquiring biological image information and biological sound information; extracting features from the biological sound information according to a plurality of preset biological sound feature extraction windows, a preset filter window, and a preset window processing function, to obtain a plurality of biological audio feature information; extracting features from the biological image information according to a plurality of preset biological image feature extraction matrices, to obtain a plurality of biological image feature information; aligning the biological audio feature information and the biological image feature information based on a preset basis vector, a preset encoding vector, and a preset spatial projection function, to obtain biological audio alignment features and biological image alignment features; respectively strengthening the biological audio alignment features and the biological image alignment features, to obtain biological audio monitoring features and biological image monitoring features; monitoring a living being according to the biological audio monitoring features and the biological image monitoring features; the preset encoding vector comprises a preset feature alignment encoding vector and a preset feature projection encoding vector; the step of aligning the biological audio feature information and the biological image feature information based on the preset basis vector, the preset encoding vector, and the preset spatial projection function, to obtain the biological audio alignment features and the biological image alignment features, specifically comprises the following steps: obtaining initial biological audio alignment features and initial biological image alignment features according to the biological audio feature information, the biological image feature information, and the preset basis vector; obtaining first intermediate biological audio alignment features and first intermediate biological image alignment features according to the initial biological audio alignment features, the initial biological image alignment features, and the preset feature alignment encoding vector; obtaining second intermediate biological audio alignment features and second intermediate biological image alignment features according to the biological audio feature information, the biological image feature information, and the preset feature projection encoding vector; obtaining the biological audio alignment features and the biological image alignment features according to the first intermediate biological audio alignment features, the second intermediate biological audio alignment features, the first intermediate biological image alignment features, the second intermediate biological image alignment features, and the preset spatial projection function.

2. The biodiversity monitoring method of claim 1, wherein, the step of extracting features from the biological sound information according to the plurality of preset biological sound feature extraction windows, the preset filter window, and the preset window processing function, to obtain the plurality of biological audio feature information, specifically comprises the following steps: performing analog-to-digital conversion on the biological sound information, to obtain biological sound digital information; cutting the biological sound digital information according to a preset biological sound feature extraction window, to obtain a plurality of biological audio frame information; taking the plurality of biological audio frame information as an independent variable of a preset window processing function, to calculate biological audio frame variable information; performing Fourier transform on the biological audio frame variable information, to obtain biological audio frame frequency domain information; performing filtering operation on the biological audio frame frequency domain information according to a preset filter window, to obtain biological audio frame spectrum information; performing frequency domain analysis calculation and normalization processing on the biological audio frame spectrum information, to obtain the plurality of biological audio feature information.

3. The biodiversity monitoring method of claim 1, wherein, The step of extracting features from the biological image information according to a plurality of preset biological image feature extraction matrices to obtain a plurality of biological image feature information specifically includes: Randomly generating a plurality of digital strings conforming to Gaussian distribution; Respectively converting the formats of the plurality of digital strings to obtain a plurality of Gaussian numerical matrices; Respectively performing inner product operations on the plurality of initial biological image feature maps and the plurality of Gaussian numerical matrices to obtain a plurality of intermediate biological image feature maps; Respectively calculating the element average values of the plurality of intermediate biological image feature maps to generate a plurality of target biological image feature maps; Respectively performing normalization processing on the plurality of target biological image feature maps to generate biological image feature information. The step of obtaining biological audio alignment features and biological image alignment features according to the first intermediate biological audio alignment feature, the second intermediate biological audio alignment feature, the first intermediate image audio alignment feature, the second intermediate biological image alignment feature, and a preset spatial projection function specifically includes:

4. The biodiversity monitoring method of claim 1, wherein, Respectively performing inner product operations on the first intermediate biological audio alignment feature and the second intermediate biological audio alignment feature to obtain a third intermediate biological audio alignment feature; Respectively performing inner product operations on the first intermediate biological image alignment feature and the second intermediate biological image alignment feature to obtain a third intermediate biological image alignment feature; According to the third intermediate biological audio alignment feature, the third intermediate biological image alignment feature, and the preset spatial projection function, biological audio feature projection variables and biological image feature projection variables are obtained; Respectively performing weighted summation on the biological audio feature projection variables and the biological image feature projection variables to obtain biological audio alignment features and biological image alignment features. The step of respectively performing reinforcement processing on the biological audio alignment features and the biological image alignment features to obtain biological audio monitoring features and biological image monitoring features specifically includes:

5. The biodiversity monitoring method of claim 1, wherein, According to a preset audio feature mask information, the biological audio alignment features are subjected to numerical screening to obtain a plurality of biological behavior sound features; The biological behavior sound features include biological behavior sound feature nodes and biological behavior sound feature connection arcs; According to a preset image feature mask information, the biological image alignment features are subjected to numerical screening to obtain a plurality of biological behavior image features; The biological behavior image features include biological behavior image feature nodes and biological behavior image feature connection arcs; The Euclidean distances of each biological behavior sound feature node and a plurality of biological behavior sound feature connection arcs are calculated to obtain a first biological behavior sound feature correlation degree; The Euclidean distances of each biological behavior sound feature connection arc and a plurality of biological behavior sound feature nodes are respectively calculated to obtain a second biological behavior sound feature correlation degree; The Euclidean distances of each biological behavior sound feature connection arc and a plurality of biological behavior sound feature nodes are respectively calculated to obtain a second biological behavior sound feature correlation degree; The first biological behavior sound feature correlation degree and the second biological behavior sound feature correlation degree are normalized and weighted sum calculated respectively to obtain first biological behavior sound dimension conversion features and second biological behavior sound dimension conversion features; Euclidean distances of each biological behavior image feature node and a plurality of biological behavior image feature connection arcs are calculated to obtain first biological behavior image feature correlation degrees; Euclidean distances of each biological behavior image feature connection arc and a plurality of biological behavior image feature nodes are calculated respectively to obtain second biological behavior image feature correlation degrees; The first biological behavior image feature correlation degree and the second biological behavior image feature correlation degree are normalized and weighted sum calculated respectively to obtain first biological behavior image dimension conversion features and second biological behavior image dimension conversion features; According to the first biological behavior sound dimension conversion features, the second biological behavior sound dimension conversion features, the first biological behavior image dimension conversion features, the second biological behavior image dimension conversion features, a preset biological audio feature weight matrix, a preset biological image feature weight matrix, a preset biological audio feature mapping function and a preset biological image feature mapping function, biological audio monitoring features and biological image monitoring features are obtained.

6. The biodiversity monitoring method of claim 5, wherein, The step of obtaining biological audio monitoring features and biological image monitoring features according to the first biological behavior sound dimension conversion features, the second biological behavior sound dimension conversion features, the first biological behavior image dimension conversion features, the second biological behavior image dimension conversion features, a preset biological audio feature weight matrix, a preset biological image feature weight matrix, a preset biological audio feature mapping function and a preset biological image feature mapping function specifically includes: The first biological behavior sound dimension conversion features and the second biological behavior sound dimension conversion features are spliced to obtain initial biological audio monitoring features; The initial biological audio monitoring features and a preset first biological audio feature weight matrix are subjected to inner product operation to obtain first intermediate biological audio monitoring features; The first intermediate biological audio monitoring features and a preset second biological audio feature weight matrix are subjected to inner product operation to obtain second intermediate biological audio monitoring features; The first biological behavior image dimension conversion features and the second biological behavior image dimension conversion features are spliced to obtain initial biological image monitoring features; The initial biological image monitoring features and a preset first biological image feature weight matrix are subjected to inner product operation to obtain first intermediate biological image monitoring features; The first intermediate biological image monitoring features and a preset second biological image feature weight matrix are subjected to inner product operation to obtain second intermediate biological image monitoring features; According to the second intermediate biological audio monitoring features, the second intermediate biological image monitoring features, a preset biological audio feature mapping function and a preset biological image feature mapping function, biological audio monitoring features and biological image monitoring features are obtained.

7. A biodiversity monitoring device, characterized in that, It includes: An information acquisition module is configured to acquire biological image information and biological sound information; The biological audio feature information generation module is configured to perform feature extraction on the biological sound information according to a plurality of preset biological sound feature extraction windows, a preset filter window, and a preset window processing function, to obtain a plurality of biological audio feature information. The biological image feature information generation module is configured to perform feature extraction on the biological image information according to a plurality of preset biological image feature extraction matrices, to obtain a plurality of biological image feature information. The biological audio alignment feature and biological image alignment feature generation module is configured to perform alignment processing on the biological audio feature information and the biological image feature information based on a preset basis vector, a preset encoding vector, and a preset space projection function, to obtain biological audio alignment features and biological image alignment features. The biological audio monitoring feature and biological image monitoring feature generation module is configured to respectively perform strengthening processing on the biological audio alignment features and the biological image alignment features, to obtain biological audio monitoring features and biological image monitoring features. The biological monitoring module is configured to monitor a biological object based on the biological audio monitoring features and the biological image monitoring features. The preset encoding vector includes a preset feature alignment encoding vector and a preset feature projection encoding vector. The step of performing alignment processing on the biological audio feature information and the biological image feature information based on the preset basis vector, the preset encoding vector, and the preset space projection function to obtain biological audio alignment features and biological image alignment features specifically includes: obtaining initial biological audio alignment features and initial biological image alignment features based on the biological audio feature information, the biological image feature information, and the preset basis vector; obtaining first intermediate biological audio alignment features and first intermediate biological image alignment features based on the initial biological audio alignment features, the initial biological image alignment features, and the preset feature alignment encoding vector; obtaining second intermediate biological audio alignment features and second intermediate biological image alignment features based on the biological audio feature information, the biological image feature information, and the preset feature projection encoding vector; obtaining biological audio alignment features and biological image alignment features based on the first intermediate biological audio alignment features, the second intermediate biological audio alignment features, the first intermediate biological image alignment features, the second intermediate biological image alignment features, and the preset space projection function.

8. A terminal device, comprising: The terminal device includes a memory and a processor, the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 6.

Images