Biodiversity monitoring method and device and terminal equipment
By combining biological images and sound information in the ecological environment for feature extraction and alignment processing, the problems of missed identification and inaccuracy in biodiversity monitoring in existing technologies are solved, higher monitoring accuracy and comprehensiveness are achieved, and ecological protection research is supported.
Patent Information
- Application Number
- CN202511133808.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-14
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2045-08-14
AI Technical Summary
Existing biodiversity monitoring methods are prone to missed or inaccurate identification, and there is little room for improvement in identification accuracy, which increases the difficulty of biological behavior monitoring in research work.
By acquiring biological image and sound information in the ecological environment, feature extraction is performed using preset feature extraction windows, filter windows, and window processing functions, and feature alignment is performed in combination with basis vectors, coding vectors, and spatial projection functions. The aligned features are then enhanced, and finally monitoring is performed based on biological audio and image monitoring features.
It improves the accuracy and comprehensiveness of biodiversity monitoring, makes up for the shortcomings of image and audio information in monitoring, enhances the expressiveness and reliability of monitoring data, and provides more important biodiversity information for researchers in ecological protection research.
Smart Images

Figure CN120727018A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of data processing technology, and in particular relates to a biodiversity monitoring method, apparatus, and terminal device. Background Art
[0002] Across numerous ecological environments, diverse organisms are distributed and interdependent, maintaining a dynamic balance within the ecosystem. To prevent this dynamic balance from being disrupted, leading to biodiversity decline or even species extinction, routine biodiversity monitoring is essential. In particular, monitoring of specific behaviors during specific growth periods or seasons can provide reliable reference data for studying the growth and reproduction of species populations, as well as ecological trends and conservation measures. This helps maintain ecosystem stability and contributes to its sustainable development.
[0003] Current methods for biodiversity monitoring typically employ traditional acoustic analysis methods to statistically analyze various types of biological audio collected in the ecological environment, identify the calls of different species of organisms contained in the audio, and then further identify and analyze the calls. Alternatively, image processing methods are used to classify images taken in the wild and identify the external morphology or behavioral manifestations of different species of organisms contained in the images.
[0004] However, when using traditional acoustic analysis methods or image processing methods to monitor biodiversity in existing technologies, it is easy to miss or inaccurately identify biological behavior, and there is little room for improvement in identification accuracy, which increases the difficulty of accurately monitoring biological behavior in research work. Summary of the Invention
[0005] In view of this, embodiments of the present application provide a biodiversity monitoring method, apparatus, and terminal device, aiming to solve the problems of missed monitoring or inaccurate monitoring in the process of monitoring biological behavior in the prior art.
[0006] A first aspect of the embodiments of the present application provides a biodiversity monitoring method, comprising: Acquiring biological image information and biological sound information; Extracting features from the bio-sound information based on a plurality of preset bio-sound feature extraction windows, a preset filtering window, and a preset window processing function to obtain a plurality of bio-audio feature information; Extracting features from the bio-image information according to a plurality of preset bio-image feature extraction matrices to obtain a plurality of bio-image feature information; Based on a preset basis vector, a preset encoding vector, and a preset spatial projection function, the bio-audio feature information and the bio-image feature information are aligned to obtain a bio-audio alignment feature and a bio-image alignment feature; Respectively performing enhancement processing on the bio-audio alignment feature and the bio-image alignment feature to obtain a bio-audio monitoring feature and a bio-image monitoring feature; The living being is monitored based on the biological audio monitoring features and the biological image monitoring features.
[0007] A second aspect of an embodiment of the present application provides a biodiversity monitoring device, comprising: An information acquisition module, used to acquire biological image information and biological sound information; a bio-audio feature information generation module, configured to extract features from the bio-sound information based on a plurality of preset bio-sound feature extraction windows, a preset filtering window, and a preset window processing function, to obtain a plurality of bio-audio feature information; A biological image feature information generation module, configured to extract features from the biological image information according to a plurality of preset biological image feature extraction matrices to obtain a plurality of biological image feature information; A bio-audio alignment feature and a bio-image alignment feature generation module, configured to align the bio-audio feature information and the bio-image feature information based on a preset basis vector, a preset encoding vector, and a preset spatial projection function to obtain a bio-audio alignment feature and a bio-image alignment feature; a bio-audio monitoring feature and a bio-image monitoring feature generation module, configured to enhance the bio-audio alignment feature and the bio-image alignment feature to obtain a bio-audio monitoring feature and a bio-image monitoring feature; The biological monitoring module is used to monitor the biological being according to the biological audio monitoring characteristics and the biological image monitoring characteristics.
[0008] A third aspect of an embodiment of the present application provides a terminal device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, and when the processor executes the computer program, the steps of the biodiversity monitoring method described in the first aspect above are implemented.
[0009] A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, comprising: storing a computer program, which, when executed by a processor, implements the steps of the biodiversity monitoring method described in the first aspect above.
[0010] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the present application extracts and aligns features of image information and sound information obtained in the ecological environment, and enhances the aligned features for biodiversity monitoring, thereby combining audio information with image information, compensating for the shortcomings of image information in biodiversity identification through audio information, compensating for the limitations of audio information in biodiversity monitoring through image information, and enhancing the expressiveness and reliability of monitoring data, thereby effectively improving the accuracy and comprehensiveness of real-time monitoring of biodiversity, and screening out biodiversity conditions that are more important to research work for researchers to conduct ecological protection research. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0012] Figure 1 This is a schematic diagram of the implementation process of the biodiversity monitoring method provided in Example 1 of the present application; Figure 2 This is a schematic diagram of the implementation process of the biodiversity monitoring method provided in Example 2 of the present application; Figure 3 This is a schematic diagram of the implementation process of the biodiversity monitoring method provided in Example 3 of the present application; Figure 4 This is a schematic diagram of the implementation process of the biodiversity monitoring method provided in Example 4 of the present application; Figure 5 This is a schematic diagram of the implementation process of the biodiversity monitoring method provided in Example 5 of the present application; Figure 6 This is a schematic diagram of the implementation process of the biodiversity monitoring method provided in Example 6 of the present application; Figure 7 This is a schematic diagram of the implementation process of the biodiversity monitoring method provided in Example 7 of the present application; Figure 8 is a schematic structural diagram of a biodiversity monitoring device provided in an embodiment of the present application; Figure 9 It is a schematic diagram of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0013] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.
[0014] In order to illustrate the technical solution described in this application, specific embodiments are provided below.
[0015] Figure 1 The following is a flowchart of the implementation of the biodiversity monitoring method provided in Example 1 of the present application, which is detailed as follows: Step S101: Acquire biological image information and biological sound information.
[0016] In this embodiment, the biological image information can be collected omnidirectionally by a camera device set in the ecological environment, and the biological sound information can be collected omnidirectionally by an array of microphones set in the ecological environment.
[0017] Step S102 , performing feature extraction on the bio-sound information according to a plurality of preset bio-sound feature extraction windows, a preset filtering window, and a preset window processing function, to obtain a plurality of bio-audio feature information.
[0018] In this embodiment, the preset biological sound feature extraction window can be a matrix of different dimensions or scales introduced into the analysis and calculation process, which is used to extract the characteristics of the biological sound information through continuously changing dimensions or scales, thereby obtaining biological audio feature information of different dimensions or scales. The preset biological sound feature extraction window can also be an interval window for sampling the biological sound information, which is used to intercept the biological sound information to achieve the sampling of the sound information. The preset filtering window can be used to filter the sampled sound information to suppress the high-frequency harmonics in the sound information. The preset window processing function can be set based on the Hamming window function to smooth the sound information and reduce spectrum leakage. Among them, the preset biological sound feature extraction window and the preset filtering window can both be set manually.
[0019] Step S103 : performing feature extraction on the biological image information according to a plurality of preset biological image feature extraction matrices to obtain a plurality of biological image feature information.
[0020] In this embodiment, the preset biometric image feature extraction window refers to a matrix of varying dimensions or scales introduced into the analysis and calculation process. This matrix is used to extract features of the biometric image information through continuously varying dimensions or scales, thereby obtaining biometric image feature information of varying dimensions or scales. The preset biometric image features can be manually set. The biometric image feature information can be generated based on the results of multiple convolution calculations performed on the biometric image feature extraction matrix and the biometric image information. It is understood that the biometric image feature information is presented in a matrix format.
[0021] Step S104 , aligning the bio-audio feature information and the bio-image feature information based on a preset basis vector, a preset encoding vector, and a preset spatial projection function to obtain a bio-audio alignment feature and a bio-image alignment feature.
[0022] In this embodiment, due to the huge differences in visualization, semantics, information density, etc. between sound and text data, there will be a problem of misalignment between feature data. Basis vectors and coding vectors are required to extract and transform the semantic information of the two data, and then project the semantic information of the two data into the same feature space through a spatial projection function, so as to simultaneously generate aligned feature vectors, so that the feature representations of the two data can be semantically consistent, which is convenient for subsequent processing through feature data of the same caliber to improve the accuracy and reliability of biodiversity monitoring. The audio features and image features after alignment are biological audio alignment features and biological image alignment features. Among them, the preset basis vectors, the preset coding vectors and the preset spatial projection function can be set manually.
[0023] Step S105 , performing enhancement processing on the bio-audio alignment feature and the bio-image alignment feature to obtain a bio-audio monitoring feature and a bio-image monitoring feature.
[0024] In this embodiment, the biological audio alignment features and the biological image alignment features can be further extracted through a preset audio mask and a preset image mask, and then the spatial structure of the further extracted features can be constructed. The spatial structure can be a spatial structure set based on the propagation mechanism of nodes and connection arcs. By calculating and adjusting the correlation between the nodes and connection arcs in the spatial structure, the local features of the audio and image are enhanced at the global level. The enhanced features, namely the biological audio monitoring features and the biological image monitoring features, are used to enhance the global feature representation of the audio features and the image features, thereby reducing the interference of less important features on the biological monitoring process, so as to improve the comprehensiveness and reliability of biodiversity monitoring.
[0025] Step S106: monitoring the living being according to the biological audio monitoring characteristics and the biological image monitoring characteristics.
[0026] In this embodiment, biological behavior monitoring can be achieved by matching biological audio monitoring features and biological image monitoring features with preset biological behavior sample features. The preset biological behavior sample features can be extracted and aggregated based on biological behavior sound data and biological behavior image data collected in previous research work, and are used to characterize different biological behaviors in the ecological environment, such as courtship, mating, nesting, egg laying, and chick rearing.
[0027] It is understandable that in the existing technology, when only audio information is processed to achieve biological call monitoring, the behavior of organisms in static conditions cannot be effectively identified, and when the microphone is damaged, effective behavioral information cannot be obtained; and when only the captured images are processed, when there are special circumstances such as the camera being blocked, the equipment being damaged, and blind spots appearing within the monitoring range, it is impossible to introduce information from outside the image to track the behavior of organisms in the image, and thus it is impossible to make up for the loopholes in biodiversity monitoring.
[0028] The biodiversity monitoring method provided in the embodiments of the present application extracts and aligns features of image information and sound information obtained in the ecological environment, and enhances the aligned features for use in biodiversity monitoring. This method can combine audio information with image information, compensate for the shortcomings of image information in biodiversity identification through audio information, compensate for the limitations of audio information in biodiversity monitoring through image information, and enhance the expressiveness and reliability of monitoring data, thereby effectively improving the accuracy and comprehensiveness of real-time monitoring of biodiversity, and screening out biodiversity conditions that are more important to research work for researchers to conduct ecological protection research.
[0029] Figure 2 The flowchart of the biodiversity monitoring method provided in the second embodiment of the present application is shown. The difference between the second embodiment and the first embodiment is that step S102 specifically includes: Step S201 , performing analog-to-digital conversion on the biological sound information to obtain biological sound digital information.
[0030] In this embodiment, the biosound information may be sampled first. Specifically, samples of the continuous biosound information in the time domain are taken at discrete time points, where the intervals between discrete time points can be manually set. The sampled biosound information is then quantized. Specifically, the continuous values of the biosound information at discrete times are converted into discrete values of the biosound information at discrete times. Each discrete value of the biosound information can be selected from a finite set of continuous values of the biosound information. The quantized biosound information is then encoded. Specifically, the discrete values of the biosound information at discrete times are converted into a multi-bit binary representation, where the number of bits can be manually set.
[0031] Step S202 : intercepting the biological sound digital information according to a preset biological sound feature extraction window to obtain a plurality of biological audio frame information.
[0032] In this embodiment, the size of the preset bio-sound feature extraction window in the time domain can be manually set. Through the bio-sound feature extraction window, the bio-sound digital information is cut into multiple short segments, each of which becomes a frame, that is, a bio-audio frame information. It can be understood that the cut-offs may not be back-to-back cuts, but rather cuts with overlapping parts in the time domain, which is used to prevent the ends of the bio-audio frame information from being severely weakened and causing local feature loss through the subsequent preset window processing function. The time difference between the starting positions of two adjacent bio-audio frames is called frame shift.
[0033] Step S203 , using the plurality of biological audio frame information as independent variables of a preset window processing function to calculate biological audio frame variable information.
[0034] In this embodiment, the preset window processing function can be set based on a Hamming window function, a zero-power window function in the time domain, or a Gaussian window function. When the window processing function is set based on the Hamming window function and the biological audio frame information is used as the independent variable of the window processing function, the biological audio frame information is multiplied by the Hamming window function to avoid the generation of the Gibbs effect, and the calculated value is the biological audio frame variable information.
[0035] Step S204: Perform Fourier transform on the biological audio frame variable information to obtain biological audio frame frequency domain information.
[0036] In this embodiment, Fourier transform is performed on all obtained biological audio frame variable information to convert the biological audio frame variable information from the time domain to the frequency domain, and the calculation result is the biological audio frame frequency domain information.
[0037] Step S205 : performing a filtering operation on the biological audio frame frequency domain information according to a preset filtering window to obtain biological audio frame spectrum information.
[0038] In this embodiment, the preset filtering window can be manually set, such as a scale window of a Mel filter. The frequency domain filtering operation performed on the biological audio frame frequency domain information can include first stacking multiple biological audio frame frequency domain information along a dimension to obtain a stacked biological audio frame spectrogram, and then filtering the biological audio frame spectrogram through multiple scale windows of a Mel filter. The filtered information is the biological audio frame spectrum information, thereby achieving the filtering operation on the biological audio frame frequency domain information.
[0039] Step S206 , performing frequency domain analysis and normalization processing on the biological audio frame spectrum information to obtain a plurality of biological audio feature information.
[0040] In this embodiment, the frequency domain analysis calculation can be to calculate the Mel-frequency cepstral coefficients of the biological audio frame spectrum information, and normalize the Mel-frequency cepstral coefficients after analysis and calculation to eliminate the influence of factors such as different recording devices on the audio characteristics, and at the same time to avoid the numerical value exceeding the dimensional range of the numerical value that can be calculated by the computer.
[0041] The biodiversity monitoring method provided in the embodiment of the present application can adapt to biosound information with different signal-to-noise ratios through different biosound feature extraction windows and window processing functions, convert biosound information with different signal-to-noise ratios into spectral characteristics obtained in the frequency domain, and filter the spectral characteristics through a filtering window, effectively reducing the interference of noise in the biosound information on important biosound features, thereby ensuring the accuracy and robustness of sound feature extraction for biosound information.
[0042] Figure 3 The flowchart of the biodiversity monitoring method provided in the third embodiment of the present application is shown. The difference between the third embodiment and the first embodiment is that step S103 specifically includes: Step S301: randomly generate multiple digital strings that conform to Gaussian distribution.
[0043] In this embodiment, the Box-Muller method may be used to utilize the relationship between two independent uniformly distributed random variables to generate a Gaussian distributed random number, that is, a number string that conforms to the Gaussian distribution.
[0044] Step S302 : convert the formats of the plurality of digital strings respectively to obtain a plurality of Gaussian numerical matrices.
[0045] In this embodiment, the number of rows and columns of the matrix may be set, and then the digital strings may be arranged according to the number of rows and columns to generate a Gaussian numerical matrix.
[0046] Step S303 , performing multiple convolution calculations on multiple preset bio-image feature extraction matrices and the bio-image information to obtain multiple initial bio-image feature maps.
[0047] In this embodiment, the preset bio-image feature extraction matrix can be manually set. The number of convolution calculations can be 8 or 10. The result of multiple convolution calculations is the initial bio-image feature map, which is used for subsequent depth feature extraction.
[0048] Step S304 , performing inner product operations on the multiple initial biological image feature maps and the multiple Gaussian numerical matrices to obtain multiple intermediate biological image feature maps.
[0049] In this embodiment, an inner product operation is performed on the initial biometric image feature map and a Gaussian numerical matrix. The result of the inner product operation is an intermediate biometric image feature map, which is presented in the form of a matrix. This inner product operation is used to assign weights to all values in the initial biometric image feature map using a normal distribution, thereby highlighting important features in the initial biometric image feature map and facilitating subsequent consideration of these important features. It is understood that a normal distribution is a bell-shaped curve, with values increasing as they approach the center. Therefore, it is possible to simultaneously smooth values in the middle and edge rows of the initial biometric image feature map, thereby achieving noise reduction for the initial biometric image feature map and reducing the impact of insignificant feature values in the initial biometric image feature map on the overall biometric image characteristics.
[0050] Step S305 , respectively calculating the element averages of the plurality of intermediate biological image feature maps to generate a plurality of target biological image feature maps.
[0051] In this embodiment, by extracting all elements in each intermediate biological image feature map and then calculating the average value of all elements in each intermediate biological image feature map, multiple one-dimensional vectors, namely target biological image feature maps, are generated, which are used to extract more abstract and stable feature representation elements while retaining the original biological image features, thereby performing dimensionality reduction processing on the intermediate biological image feature maps, reducing the complexity of subsequent analysis and calculation processes, and alleviating the computational burden of the processor.
[0052] Step S306 , normalizing the plurality of target biological image feature maps to generate biological image feature information.
[0053] In this embodiment, the normalization technique can include min-max normalization, Z-score normalization, maximum absolute value scaling (scaling the data to the interval [-1, 1]), or robust scaling (scaling using the median and interquartile range). The result of normalizing the target bioimage feature map, i.e., bioimage feature information, is used to prevent numerical results generated during the calculation process from exceeding the numerical dimension range that the computer can handle, thereby ensuring the stability and reliability of the calculation process.
[0054] The biodiversity monitoring method provided in the embodiment of the present application processes biological image features by generating a matrix with Gaussian distribution characteristics, which is used to effectively smooth the numerical values in the biological image features while retaining the main image features, reduce the negative impact of high-frequency noise on the calculation of biological image features, thereby improving the effectiveness and accuracy of feature extraction of biological image information, and thus improving the accuracy and reliability of biological monitoring work.
[0055] Figure 4 The flowchart of the biodiversity monitoring method provided in the fourth embodiment of the present application is shown. The difference between the fourth embodiment and the first embodiment is that the preset coding vector includes a preset feature alignment coding vector and a preset feature projection coding vector. The step S104 specifically includes: Step S401 : obtaining initial bio-audio alignment features and initial bio-image alignment features according to the bio-audio feature information, the bio-image feature information and preset basis vectors.
[0056] In this embodiment, the preset basis vectors can be manually set, and the number can be multiple, and can be used to represent different scales to perform alignment processing on the bio-audio feature information and bio-image feature information at different scales. The bio-audio features and bio-image features at multiple scales can be obtained by performing convolution operations on the bio-audio feature information and the bio-image feature information respectively, i.e., initial bio-audio alignment features and initial bio-image alignment features, thereby achieving a deeper extraction of the bio-audio features and bio-image features, while avoiding the problem of feature alignment failure in subsequent calculations, and improving the matching effectiveness and reliability of the bio-sound information and bio-image information.
[0057] Step S402 : obtaining a first intermediate bio-audio alignment feature and a first intermediate bio-image alignment feature according to the initial bio-audio alignment feature, the initial bio-image alignment feature, and a preset feature alignment coding vector.
[0058] In this embodiment, the preset feature alignment coding vector can be manually set. The feature alignment coding vector can be used to perform inner product operations with the initial bio-audio alignment feature and the initial bio-image alignment feature, respectively, to further extract features from the initial bio-audio alignment feature and the initial bio-image alignment feature. While achieving in-depth extraction of alignment features of the bio-audio and bio-image features, the differences between multiple bio-audio alignment features and multiple bio-image alignment features are increased, thereby amplifying details in the bio-sound information and bio-image information. The calculation results are the first intermediate bio-audio alignment feature and the first intermediate bio-image alignment feature.
[0059] Step S403: obtaining a second intermediate bio-audio alignment feature and a second intermediate bio-audio alignment feature according to the bio-audio feature information, the bio-image feature information and the preset feature projection coding vector.
[0060] In this embodiment, the preset feature projection coding vector can be manually set. Convolution operations can be performed on the feature projection coding vector with the bio-audio feature information and the bio-image feature information, respectively, to quantify the values of the bio-audio feature and the bio-image feature information after projecting them into the nonlinear space. The calculated second intermediate bio-audio alignment feature and the second intermediate bio-audio alignment feature are subsequently used as variables in the nonlinear space for subsequent calculations.
[0061] Step S404: Obtain a bio-audio alignment feature and a bio-image alignment feature according to the first intermediate bio-audio alignment feature, the second intermediate bio-audio alignment feature, the first intermediate image-audio alignment feature, the second intermediate bio-image alignment feature, and a preset spatial projection function.
[0062] In this embodiment, the preset spatial projection function can be a Tanh function, a ReLU function, or an APReLU function, which is used to project the biological audio features and the biological image features into a nonlinear space for further analysis and calculation. The results after further analysis and calculation are the biological audio alignment features and the biological image alignment features.
[0063] The biodiversity monitoring method provided in the embodiment of the present application further deeply extracts the biological audio feature information and the biological image feature information to perform alignment processing on the biological audio feature information and the biological image feature information at different granularities, thereby effectively avoiding the problem of feature alignment failure during the calculation process, and then performing a specific spatial projection on the aligned biological audio feature information and the biological image feature information, and quantifying the feature values after the spatial projection, so as to further analyze and calculate the biological audio feature information and the biological image feature information through the quantized values projected into the nonlinear space, so as to enhance the feature expression of the aligned biological audio feature information and the biological image feature information, thereby avoiding the missed monitoring of detailed features in the biological sound information and the biological image information, and improving the accuracy and reliability of timely monitoring of organisms.
[0064] Figure 5 The flowchart of the biodiversity monitoring method provided in the fifth embodiment of the present application is shown. The difference between the fifth embodiment and the fourth embodiment is that step S404 specifically includes: Step S501: performing an inner product operation on the first intermediate bio-audio alignment feature and the second intermediate bio-audio alignment feature to obtain a third intermediate bio-audio alignment feature.
[0065] In this embodiment, an inner product operation is performed on the first intermediate bio-audio alignment feature and the second intermediate bio-audio alignment feature to quantify the displacement of the first intermediate bio-audio alignment feature in the nonlinear space through the second intermediate bio-audio alignment feature, so as to further increase the distance between each bio-audio feature, thereby highlighting the feature representation of each bio-audio feature in the nonlinear space.
[0066] Step S502 : performing an inner product operation on the first intermediate biological image alignment feature and the second intermediate biological image alignment feature to obtain a third intermediate biological image alignment feature.
[0067] In this embodiment, an inner product operation is performed on the first intermediate bio-image alignment feature and the second intermediate bio-image alignment feature to quantify the displacement of the first intermediate bio-image alignment feature in the nonlinear space through the second intermediate bio-image alignment feature, so as to further increase the distance between each bio-image feature, thereby highlighting the feature representation of each bio-image feature in the nonlinear space.
[0068] Step S503: obtaining a bio-audio feature projection variable and a bio-image feature projection variable according to the third intermediate bio-audio alignment feature, the third intermediate bio-image alignment feature, and a preset spatial projection function.
[0069] In this embodiment, the third intermediate bio-audio alignment feature and the third intermediate bio-image alignment feature may be used as independent variables of the spatial projection function respectively, and the calculated function values are the bio-audio feature projection variable and the bio-image feature projection variable.
[0070] Step S504 , performing weighted summation on the bio-audio feature projection variables and the bio-image feature projection variables to obtain a bio-audio alignment feature and a bio-image alignment feature.
[0071] In this embodiment, the biological audio feature projection variables and the biological image feature projection variables are weighted and summed respectively to perform fusion processing on the biological audio features and the biological image features. The results of the fusion processing are the biological audio alignment features and the biological image alignment features.
[0072] The biodiversity monitoring method provided in the embodiment of the present application performs in-depth calculations on the aligned features to increase the distance between each biological sound feature and biological image feature, thereby enhancing the feature representation of the biological sound feature and biological image feature in the nonlinear space, effectively avoiding the omission of important details in the biological sound information and biological image information, so as to improve the accuracy and sensitivity of biodiversity monitoring. By performing weighted summation on the feature variables in the nonlinear space to summarize the extracted features, while ensuring that the effective features can be fully considered in the calculation process, the computer's storage space is saved, and it is convenient to analyze and calculate more biological sound information and biological image information.
[0073] Figure 6 The flowchart of the biodiversity monitoring method provided in the sixth embodiment of the present application is shown. The difference between the sixth embodiment and the first embodiment is that step S105 specifically includes: Step S601: numerically screen the biological audio alignment features according to preset audio feature mask information to obtain multiple biological behavior sound features; the biological behavior sound features include biological behavior sound feature nodes and biological behavior sound feature connection arcs.
[0074] In this embodiment, the preset audio feature mask information can be a character setting, or can be obtained through deep feature extraction and spatial transformation of audio information of various types of biological behaviors extracted from past studies, and is used to characterize the audio features of biological behaviors that are of greater concern to researchers in biodiversity samples. The biological audio alignment features calculated from the acquired biological sound information are screened using the audio feature mask information to screen out the biological audio features corresponding to the biological behaviors that are of concern to researchers from the biological audio alignment features, and construct the biological audio features into a spatial structure based on the principle of spatial correlation, which can be nodes of a spatial structure constructed based on numerical values in the biological audio features, namely, biological behavior sound feature nodes, and arcs connecting nodes of a spatial structure constructed based on the sequential relationship of numerical values in the biological audio features in the time domain, namely, biological behavior sound feature arcs.
[0075] Step S602 : numerically screening the biological image alignment features according to preset image feature mask information to obtain a plurality of biological behavior image features; the biological behavior image features include biological behavior image feature nodes and biological behavior image feature connection arcs.
[0076] In this embodiment, the preset image feature mask information can be set by a person, or can be obtained through deep feature extraction and spatial transformation of image information of various types of biological behaviors extracted from previous studies, and is used to characterize image features of biological behaviors that are of particular interest to researchers in biodiversity samples. The image feature mask information is used to filter the biological image alignment features calculated from the acquired biological image information, and is used to filter out the biological image features corresponding to the biological behaviors that are of particular interest to researchers from the biological image alignment features. The biological image features are then constructed into a spatial structure based on the principle of spatial correlation. The spatial structure can be constructed based on the numerical values in the biological image features, i.e., the biological behavior image feature nodes, and the connecting arcs of the nodes of the spatial structure can be constructed based on the temporal sequence of the numerical values in the biological image features, i.e., the biological behavior image feature connecting arcs.
[0077] Step S603 , calculating the Euclidean distance between each of the biological behavior sound feature nodes and the arcs connecting the plurality of biological behavior sound features to obtain a first biological behavior sound feature correlation degree.
[0078] In this embodiment, considering that different sound information of biological behaviors have different correlations, for example, the calls of birds during courtship are more significantly different than the calls of juveniles, the calls of different birds during courtship have a high correlation, and the biological behavior sound features of the calls of birds during courtship can be screened out from the numerous biological behavior sound features by calculating the correlation. Therefore, it is necessary to calculate the Euclidean distance between each biological behavior sound feature node and the arc connecting the biological behavior sound features to quantify the correlation between each biological behavior sound feature, so as to facilitate the subsequent screening of biological behavior sound features with high correlation for effective monitoring of the special behavior of organisms.
[0079] Step S604 , respectively calculating the Euclidean distances of the bio-behavior sound feature connection arcs and the plurality of bio-behavior sound feature nodes to obtain a second bio-behavior sound feature correlation degree.
[0080] In this embodiment, considering the spatiotemporal correlation between the different sounds of various biological behaviors, this spatiotemporal correlation is represented by connecting arcs of biological behavior sound features. It is understandable that the biological sound information acquired in the same time and space has spatiotemporal consistency, and therefore the correlation between the connecting arcs of biological behavior sound features is relatively strong. By calculating the Euclidean distance between the connecting arcs of biological behavior sound features and multiple nodes of the biological behavior sound features, the spatiotemporal correlation between the various biological behavior sound features can be quantified, which can be used to identify the spatiotemporal occurrence of specific biological behaviors, thereby effectively monitoring them.
[0081] Step S605 , normalizing and weighted summing the first biological behavior sound feature correlation and the second biological behavior sound feature correlation, respectively, to obtain the first biological behavior sound dimensional conversion feature and the second biological behavior sound dimensional conversion feature.
[0082] In this embodiment, the correlation degree of the first biological behavior sound feature and the correlation degree of the second biological behavior sound feature can be normalized respectively, and the normalized values are used as the weight values of the two groups of biological behavior sound features. Based on the weight values, the biological behavior sound features are weighted and summed up, and the results obtained are the first biological behavior sound dimensional conversion feature and the second biological behavior sound dimensional conversion feature.
[0083] Step S606 , calculating the Euclidean distance between each of the biological behavior image feature nodes and a plurality of biological behavior image feature connection arcs to obtain a first biological behavior image feature correlation degree.
[0084] In this embodiment, considering that different image information of biological behaviors have different correlations, for example, the behavioral actions displayed by birds during courtship are more significantly different from the behavioral actions of birds when they are hungry. Therefore, the behavioral actions displayed by different birds during courtship have a high correlation, and the biological behavior image features of the behavioral actions displayed by birds during courtship can be screened out from the many biological behavior image features by calculating the correlation. Therefore, it is necessary to calculate the Euclidean distance between each biological behavior image feature node and the biological behavior image feature connection arc to quantify the correlation between each biological behavior image feature, so as to facilitate the subsequent screening of biological behavior image features with high correlation for effective monitoring of special behaviors of organisms. Among them, one biological behavior image feature node corresponds to multiple first biological behavior image feature correlations, and one biological behavior image feature connection arc corresponds to one first biological behavior image feature correlation.
[0085] Step S607 , respectively calculating the Euclidean distances of the biological behavior image feature connection arcs and the plurality of biological behavior image feature nodes to obtain a second biological behavior image feature correlation degree.
[0086] In this embodiment, considering that different image information of biological behaviors have different correlations, for example, the behavioral actions displayed by birds during courtship are more significantly different from the behavioral actions of birds when they are hungry. Therefore, the behavioral actions displayed by different birds during courtship have a high correlation, and the biological behavior image features of the behavioral actions displayed by birds during courtship can be screened out from the many biological behavior image features by calculating the correlation. Therefore, it is necessary to calculate the Euclidean distance between each biological behavior image feature node and the biological behavior image feature connection arc to quantify the correlation between each biological behavior image feature, so as to facilitate the subsequent screening of biological behavior image features with high correlation for effective monitoring of special behaviors of organisms. Among them, one biological behavior image feature connection arc corresponds to multiple second biological behavior image feature correlations, and one biological behavior image feature node corresponds to one second biological behavior image feature correlation.
[0087] Step S608 , normalizing and weighted summing the first biological behavior image feature correlation degree and the second biological behavior image feature correlation degree respectively to obtain the first biological behavior image dimensional conversion feature and the second biological behavior image dimensional conversion feature.
[0088] In this embodiment, the correlation degree of the first biological behavior image feature and the correlation degree of the second biological behavior image feature are normalized respectively, and the normalized values are used as the weight values of the two groups of biological behavior image features. Based on the weight values, the biological behavior image features are weighted and summed, and the obtained results are the first biological behavior image dimensional conversion feature and the second biological behavior image dimensional conversion feature.
[0089] Step S609, obtaining the bio-audio monitoring feature and the bio-image monitoring feature based on the first bio-behavior sound dimensional conversion feature, the second bio-behavior sound dimensional conversion feature, the first bio-behavior image dimensional conversion feature, the second bio-behavior image dimensional conversion feature, the preset bio-audio feature weight matrix, the preset bio-image feature weight matrix, the preset bio-audio feature mapping function and the preset bio-image feature mapping function.
[0090] In this embodiment, the first biological behavior sound dimensional conversion feature and the second biological behavior sound dimensional conversion feature may be fused to obtain a fused biological behavior sound dimensional conversion feature, and then multiplied by a preset biological audio feature weight matrix with the biological behavior sound dimensional conversion feature to quantify the importance of each numerical value in the biological behavior sound dimensional conversion feature. The multiplication result is mapped to a multidimensional space through a preset biological audio feature mapping function for processing to further enhance the feature representation of each numerical value in the biological behavior sound dimensional conversion feature, widen the gap between each feature value, and enable even the smallest biological behavior feature reflected in the sound information to be monitored. The first bio-behavior image dimensional conversion feature and the second bio-behavior image dimensional conversion feature may be fused to obtain a fused bio-behavior image dimensional conversion feature, which is then multiplied by a preset bio-image feature weight matrix to quantify the importance of each value in the bio-behavior image dimensional conversion feature. The multiplication result is mapped to a multidimensional space through a preset bio-image feature mapping function for processing to further enhance the feature representation of each value in the bio-behavior image dimensional conversion feature, thereby widening the gap between each feature value, so that even the smallest bio-behavior feature in the image information can be monitored. Among them, the preset bio-audio feature weight matrix, the preset bio-image feature weight matrix, the preset bio-audio feature mapping function, and the preset bio-image feature mapping function can all be manually set.
[0091] The biodiversity monitoring method provided in the embodiment of the present application further extracts biological audio alignment features and biological image alignment features through audio feature mask information and image feature mask information, effectively filters out biological behavior information irrelevant to biodiversity research work, and highlights the information of biological special behaviors required for research work for analysis and processing. By constructing a spatial structure based on nodes and connecting arcs, the content correlation and spatiotemporal correlation between each biological sound feature and biological image feature are calculated, thereby further screening the biological sound features and biological image features, and mapping the biological sound features and biological image features after deep screening to multidimensional space for analysis and processing to enhance the differences between the sound features and image features of different biological behaviors, thereby achieving feature enhancement of the biological audio alignment features and biological image alignment features, and improving the accuracy and effectiveness of monitoring biological special behaviors through sound and image.
[0092] Figure 7 The flowchart of the biodiversity monitoring method provided in the seventh embodiment of the present application is shown. The difference between the seventh embodiment and the sixth embodiment is that step S609 specifically includes: Step S701: concatenate the first biological behavior sound dimensional conversion feature and the second biological behavior sound dimensional conversion feature to obtain an initial biological audio monitoring feature.
[0093] In this embodiment, the first biological behavior sound dimensional conversion feature and the second biological behavior sound dimensional conversion feature can be horizontally spliced or vertically spliced, and the spliced matrix is the initial biological audio monitoring feature.
[0094] Step S702 : performing an inner product operation on the initial bio-audio monitoring feature and a preset first bio-audio feature weight matrix to obtain a first intermediate bio-audio monitoring feature.
[0095] In this embodiment, the preset first bio-audio feature weight matrix can be manually set and is used to quantify the importance of each value in the initial bio-audio monitoring feature to represent the importance of the bio-audio feature values. An inner product operation is performed on the initial bio-audio monitoring feature and the first bio-audio feature weight matrix. The result of this operation serves as the first intermediate bio-audio monitoring feature, which is used to reflect the importance of each value in the initial bio-audio monitoring feature.
[0096] Step S703: performing an inner product operation on the first intermediate bio-audio monitoring feature and a preset second bio-audio feature weight matrix to obtain a second intermediate bio-audio monitoring feature.
[0097] In this embodiment, the preset second bio-audio feature weight matrix can be manually set to quantify the degree of deviation of each bio-audio monitoring feature in multidimensional space, thereby increasing the differences between each bio-audio monitoring feature in multidimensional space and achieving enhanced feature representation of the bio-audio monitoring feature. An inner product operation is performed on the first intermediate bio-audio monitoring feature and the second bio-audio feature weight matrix, and the result of the operation is used as the second intermediate bio-audio monitoring feature to facilitate further in-depth analysis and calculation in multidimensional space.
[0098] Step S704 : performing splicing processing on the first biological behavior image dimensional conversion feature and the second biological behavior image dimensional conversion feature to obtain an initial biological image monitoring feature.
[0099] In this embodiment, the first biological behavior image dimensional conversion feature and the second biological behavior image dimensional conversion feature may be spliced horizontally or vertically, and the matrix obtained by splicing is the initial biological image monitoring feature.
[0100] Step S705 : performing an inner product operation on the initial biometric image monitoring feature and a preset first biometric image feature weight matrix to obtain a first intermediate biometric image monitoring feature.
[0101] In this embodiment, the preset first biometric image feature weight matrix can be manually set and is used to quantify the importance of each value in the initial biometric image monitoring feature to represent the importance of the biometric image feature values. An inner product operation is performed on the initial biometric image monitoring feature and the first biometric image feature weight matrix. The result of this operation serves as the first intermediate biometric image monitoring feature, which is used to reflect the importance of each value in the initial biometric image monitoring feature.
[0102] Step S706 : performing an inner product operation on the first intermediate biometric image monitoring feature and a preset second biometric image feature weight matrix to obtain a second intermediate biometric image monitoring feature.
[0103] In this embodiment, the preset second biometric feature weight matrix can be manually set to quantify the degree of deviation of each biometric feature in multidimensional space, thereby increasing the differences between each biometric feature in multidimensional space and achieving enhanced feature representation of the biometric feature. An inner product operation is performed on the first intermediate biometric feature and the second biometric feature weight matrix, and the result is used as the second intermediate biometric feature to facilitate further in-depth analysis and calculation in multidimensional space.
[0104] Step S707: Obtain a bio-audio monitoring feature and a bio-image monitoring feature according to the second intermediate bio-audio monitoring feature, the second intermediate bio-image monitoring feature, a preset bio-audio feature mapping function, and a preset bio-image feature mapping function.
[0105] In this embodiment, the preset bio-audio feature mapping function and the preset bio-image feature mapping function may be a sigmoid function, which is used to map the bio-audio feature and the bio-image feature into a multidimensional space, thereby improving the nonlinear analysis effect of the bio-audio feature and the bio-image feature. The second intermediate bio-audio monitoring feature may be used as the independent variable of the bio-audio feature mapping function, and the calculated function value may be used as the bio-audio monitoring feature. The second intermediate bio-image monitoring feature may be used as the independent variable of the bio-image feature mapping function, and the calculated function value may be used as the bio-image monitoring feature.
[0106] In this embodiment, the bio-audio monitoring features and the bio-image monitoring features can be further fused to obtain the bio-monitoring features. This fusion can be performed by concatenating the bio-audio monitoring features and the bio-image monitoring features, or by averaging the values in the bio-audio monitoring features and the bio-image monitoring features, using the matrix of the average values as the bio-monitoring features. This combines the bio-sound information and the bio-image information, thereby combining audio information with image information. This allows audio information to compensate for the shortcomings of image information in biodiversity identification, while image information compensates for the limitations of audio information in biodiversity monitoring, thereby enhancing the expressiveness and reliability of the monitoring data.
[0107] The biodiversity monitoring method provided in the embodiments of the present application maps biological sound features and biological image features into a multidimensional space for analysis and processing to enhance the differences between the sound features and image features of different biological behaviors, thereby achieving feature enhancement processing of biological audio alignment features and biological image alignment features. The reliability of biodiversity monitoring is improved by aligning the audio information and image information after processing. At the same time, the audio information compensates for the shortcomings of image information in biodiversity identification, and the image information compensates for the limitations of audio information in biodiversity monitoring, thereby improving the accuracy and effectiveness of monitoring special biological behaviors through both sound and image.
[0108] Corresponding to the method of the above embodiment, Figure 8 The structural block diagram of the biodiversity monitoring device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown. Figure 8 The exemplary biodiversity monitoring device may be an executing entity of the biodiversity monitoring method provided in the aforementioned first embodiment.
[0109] Reference Figure 8 , the biodiversity monitoring device comprises: Information acquisition module 810, for acquiring biological image information and biological sound information; The bio-audio feature information generating module 820 is configured to extract features from the bio-sound information based on a plurality of preset bio-sound feature extraction windows, a preset filtering window, and a preset window processing function to obtain a plurality of bio-audio feature information; A biological image feature information generating module 830 is configured to extract features from the biological image information according to a plurality of preset biological image feature extraction matrices to obtain a plurality of biological image feature information; A bio-audio alignment feature and a bio-image alignment feature generation module 840 is configured to align the bio-audio feature information and the bio-image feature information based on a preset basis vector, a preset encoding vector, and a preset spatial projection function to obtain a bio-audio alignment feature and a bio-image alignment feature; A bio-audio monitoring feature and a bio-image monitoring feature generating module 850 is used to enhance the bio-audio alignment feature and the bio-image alignment feature to obtain a bio-audio monitoring feature and a bio-image monitoring feature respectively; The biological monitoring module 860 is used to monitor the biological being according to the biological audio monitoring characteristics and the biological image monitoring characteristics.
[0110] The process of each module in the biodiversity monitoring device provided in the embodiment of the present application realizing its own function can be specifically referred to the aforementioned Figure 1 The description of the first embodiment is omitted here.
[0111] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0112] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.
[0113] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
[0114] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.
[0115] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish descriptions and should not be understood as indicating or implying relative importance. It should also be understood that although the terms "first", "second", etc. are used in the text to describe various elements in some embodiments of the present application, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first table can be named a second table, and similarly, a second table can be named a first table without departing from the scope of the various described embodiments. Both the first table and the second table are tables, but they are not the same table.
[0116] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.
[0117] The biodiversity monitoring method provided in the embodiments of the present application can be applied to terminal devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). The embodiments of the present application do not impose any restrictions on the specific type of terminal device.
[0118] For example, the terminal device can be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a vehicle networking terminal, a computer, a laptop computer, a handheld communication device, a handheld computing device, a satellite wireless device, a wireless modem card, a TV set-top box (STB), customer premise equipment (CPE) and / or other devices for communicating on a wireless system and a next-generation communication system, such as a mobile terminal in a 5G network or a mobile terminal in a future evolved Public Land Mobile Network (PLMN) network.
[0119] As an example and not a limitation, when the terminal device is a wearable device, the wearable device can also be a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are full-featured, large in size, and can achieve complete or partial functions without relying on smartphones, such as smart watches or smart glasses, as well as those that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.
[0120] Figure 9 This is a schematic diagram of the structure of a terminal device provided by an embodiment of the present application. Figure 9 As shown, the terminal device 9 of this embodiment includes: at least one processor 90 ( Figure 9 Only one is shown), a memory 91, wherein the memory 91 stores a computer program 92 that can be run on the processor 90. When the processor 90 executes the computer program 92, the steps in the above-mentioned embodiments of the biodiversity monitoring method are implemented, such as Figure 1 Alternatively, when the processor 90 executes the computer program 92, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 8Functions of modules 810 to 860 are shown.
[0121] The terminal device 9 can be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device can include, but is not limited to, a processor 90 and a memory 91. It can be understood by those skilled in the art that Figure 9 It is only an example of the terminal device 9 and does not constitute a limitation on the terminal device 9. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include an input and sending device, a network access device, a bus, etc.
[0122] The processor 90 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.
[0123] In some embodiments, the memory 91 may be an internal storage unit of the terminal device 9, such as a hard drive or memory of the terminal device 9. The memory 91 may also be an external storage device of the terminal device 9, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, the memory 91 may include both an internal storage unit of the terminal device 9 and an external storage device. The memory 91 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program. The memory 91 may also be used to temporarily store data that has been sent or is about to be sent.
[0124] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.
[0125] An embodiment of the present application also provides a terminal device, which includes at least one memory, at least one processor, and a computer program stored in the at least one memory and executable on the at least one processor. When the processor executes the computer program, the terminal device implements the steps of any of the above-mentioned method embodiments.
[0126] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments can be implemented.
[0127] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.
[0128] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the processes in the above-mentioned embodiment method by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium.
[0129] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.
[0130] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0131] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0132] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.
Claims
1. A biodiversity monitoring method, characterized in that: include: Acquiring biological image information and biological sound information; Extracting features from the bio-sound information based on a plurality of preset bio-sound feature extraction windows, a preset filtering window, and a preset window processing function to obtain a plurality of bio-audio feature information; Extracting features from the bio-image information according to a plurality of preset bio-image feature extraction matrices to obtain a plurality of bio-image feature information; Based on a preset basis vector, a preset encoding vector, and a preset spatial projection function, the bio-audio feature information and the bio-image feature information are aligned to obtain a bio-audio alignment feature and a bio-image alignment feature; Respectively performing enhancement processing on the bio-audio alignment feature and the bio-image alignment feature to obtain a bio-audio monitoring feature and a bio-image monitoring feature; The living being is monitored based on the biological audio monitoring features and the biological image monitoring features.
2. The biodiversity monitoring method according to claim 1, wherein: The step of extracting features from the bio-sound information based on a plurality of preset bio-sound feature extraction windows, a preset filtering window, and a preset window processing function to obtain a plurality of bio-audio feature information specifically includes: Performing analog-to-digital conversion on the biological sound information to obtain biological sound digital information; According to a preset biological sound feature extraction window, the biological sound digital information is intercepted to obtain a plurality of biological audio frame information; Using the plurality of biological audio frame information as independent variables of a preset window processing function to calculate biological audio frame variable information; Performing Fourier transform on the biological audio frame variable information to obtain biological audio frame frequency domain information; Performing a filtering operation on the biological audio frame frequency domain information according to a preset filtering window to obtain biological audio frame spectrum information; The biological audio frame spectrum information is subjected to frequency domain analysis calculation and normalization processing to obtain multiple biological audio feature information.
3. The biodiversity monitoring method according to claim 1, wherein: The step of extracting features from the biological image information according to a plurality of preset biological image feature extraction matrices to obtain a plurality of biological image feature information specifically includes: Randomly generate multiple digital strings that conform to Gaussian distribution; Convert the formats of the plurality of digital strings to obtain a plurality of Gaussian numerical matrices; Performing multiple convolution calculations on a plurality of preset biological image feature extraction matrices and the biological image information to obtain a plurality of initial biological image feature maps; Performing inner product operations on the multiple initial biological image feature maps and the multiple Gaussian numerical matrices to obtain multiple intermediate biological image feature maps; respectively calculating element averages of the plurality of intermediate biological image feature maps to generate a plurality of target biological image feature maps; Normalization processing is performed on the plurality of target biological image feature maps to generate biological image feature information.
4. The biodiversity monitoring method according to claim 1, wherein: The preset coding vector includes a preset feature alignment coding vector and a preset feature projection coding vector; The step of aligning the biological audio feature information and the biological image feature information based on the preset basis vectors, the preset coding vectors, and the preset spatial projection function to obtain the biological audio alignment feature and the biological image alignment feature specifically includes: Obtaining an initial bio-audio alignment feature and an initial bio-image alignment feature based on the bio-audio feature information, the bio-image feature information, and a preset basis vector; Obtaining a first intermediate bio-audio alignment feature and a first intermediate bio-image alignment feature according to the initial bio-audio alignment feature, the initial bio-image alignment feature, and a preset feature alignment coding vector; Obtaining a second intermediate bio-audio alignment feature and a second intermediate bio-audio alignment feature according to the bio-audio feature information, the bio-image feature information, and a preset feature projection coding vector; According to the first intermediate biological audio alignment feature, the second intermediate biological audio alignment feature, the first intermediate image audio alignment feature, the second intermediate biological image alignment feature and the preset spatial projection function, the biological audio alignment feature and the biological image alignment feature are obtained.
5. The biodiversity monitoring method according to claim 4, wherein: The step of obtaining the bio-audio alignment feature and the bio-image alignment feature based on the first intermediate bio-audio alignment feature, the second intermediate bio-audio alignment feature, the first intermediate image-audio alignment feature, the second intermediate bio-image alignment feature, and a preset spatial projection function specifically includes: Performing an inner product operation on the first intermediate bio-audio alignment feature and the second intermediate bio-audio alignment feature to obtain a third intermediate bio-audio alignment feature; performing an inner product operation on the first intermediate biological image alignment feature and the second intermediate biological image alignment feature to obtain a third intermediate biological image alignment feature; Obtaining a bio-audio feature projection variable and a bio-image feature projection variable according to the third intermediate bio-audio alignment feature, the third intermediate bio-image alignment feature, and a preset spatial projection function; The bio-audio feature projection variables and the bio-image feature projection variables are weightedly summed to obtain a bio-audio alignment feature and a bio-image alignment feature.
6. The biodiversity monitoring method according to claim 1, wherein: The step of respectively enhancing the biological audio alignment feature and the biological image alignment feature to obtain the biological audio monitoring feature and the biological image monitoring feature specifically includes: According to the preset audio feature mask information, the biological audio alignment feature is numerically screened to obtain a plurality of biological behavior sound features; the biological behavior sound features include biological behavior sound feature nodes and biological behavior sound feature connection arcs; According to preset image feature mask information, the biological image alignment features are numerically screened to obtain a plurality of biological behavior image features; the biological behavior image features include biological behavior image feature nodes and biological behavior image feature connection arcs; Calculating the Euclidean distance between each of the biological behavior sound feature nodes and a plurality of biological behavior sound feature connection arcs to obtain a first biological behavior sound feature correlation degree; Calculating the Euclidean distances of the bio-behavior sound feature connection arcs and the multiple bio-behavior sound feature nodes respectively to obtain a second bio-behavior sound feature correlation degree; Normalizing and weighted summing the first biological behavior sound feature correlation and the second biological behavior sound feature correlation to obtain the first biological behavior sound dimensional conversion feature and the second biological behavior sound dimensional conversion feature; Calculating the Euclidean distance between each of the biological behavior image feature nodes and a plurality of biological behavior image feature connection arcs to obtain a first biological behavior image feature correlation degree; respectively calculating the Euclidean distances of the biological behavior image feature connection arcs and the plurality of biological behavior image feature nodes to obtain a second biological behavior image feature correlation degree; Normalizing and weighted summing the first biological behavior image feature correlation degree and the second biological behavior image feature correlation degree respectively to obtain the first biological behavior image dimensional conversion feature and the second biological behavior image dimensional conversion feature; According to the first biological behavior sound dimensional conversion feature, the second biological behavior sound dimensional conversion feature, the first biological behavior image dimensional conversion feature, the second biological behavior image dimensional conversion feature, the preset biological audio feature weight matrix, the preset biological image feature weight matrix, the preset biological audio feature mapping function and the preset biological image feature mapping function, the biological audio monitoring feature and the biological image monitoring feature are obtained.
7. The biodiversity monitoring method according to claim 6, wherein: The step of obtaining the bio-audio monitoring feature and the bio-image monitoring feature according to the first bio-behavior sound dimensional conversion feature, the second bio-behavior sound dimensional conversion feature, the first bio-behavior image dimensional conversion feature, the second bio-behavior image dimensional conversion feature, the preset bio-audio feature weight matrix, the preset bio-image feature weight matrix, the preset bio-audio feature mapping function, and the preset bio-image feature mapping function specifically includes: splicing the first biological behavior sound dimensional conversion feature and the second biological behavior sound dimensional conversion feature to obtain an initial biological audio monitoring feature; Performing an inner product operation on the initial bio-audio monitoring feature and a preset first bio-audio feature weight matrix to obtain a first intermediate bio-audio monitoring feature; Performing an inner product operation on the first intermediate bio-audio monitoring feature and a preset second bio-audio feature weight matrix to obtain a second intermediate bio-audio monitoring feature; splicing the first biological behavior image dimensional conversion feature and the second biological behavior image dimensional conversion feature to obtain an initial biological image monitoring feature; Performing an inner product operation on the initial biometric image monitoring feature and a preset first biometric image feature weight matrix to obtain a first intermediate biometric image monitoring feature; Performing an inner product operation on the first intermediate biometric image monitoring feature and a preset second biometric image feature weight matrix to obtain a second intermediate biometric image monitoring feature; The bio-audio monitoring feature and the bio-image monitoring feature are obtained according to the second intermediate bio-audio monitoring feature, the second intermediate bio-image monitoring feature, the preset bio-audio feature mapping function and the preset bio-image feature mapping function.
8. A biodiversity monitoring device, characterized in that: include: An information acquisition module, used to acquire biological image information and biological sound information; a bio-audio feature information generation module, configured to extract features from the bio-sound information based on a plurality of preset bio-sound feature extraction windows, a preset filtering window, and a preset window processing function, to obtain a plurality of bio-audio feature information; A biological image feature information generation module, configured to extract features from the biological image information according to a plurality of preset biological image feature extraction matrices to obtain a plurality of biological image feature information; A bio-audio alignment feature and a bio-image alignment feature generation module, configured to align the bio-audio feature information and the bio-image feature information based on a preset basis vector, a preset encoding vector, and a preset spatial projection function to obtain a bio-audio alignment feature and a bio-image alignment feature; a bio-audio monitoring feature and a bio-image monitoring feature generation module, configured to enhance the bio-audio alignment feature and the bio-image alignment feature to obtain a bio-audio monitoring feature and a bio-image monitoring feature; The biological monitoring module is used to monitor the biological being according to the biological audio monitoring characteristics and the biological image monitoring characteristics.
9. A terminal device, characterized in that: The terminal device includes a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Biodiversity monitoring method and system for wetland beach
CN118537895A
Data processing system and method for biodiversity monitoring
CN119226954A
Subject detection for remote biometric monitoring
US20190279481A1
Multi-modal data sentiment analysis method and apparatus, device, and storage medium
WO2024046012A1
Cited By
Automatic screening system for fundus color illumination sugar net lesions based on image discrimination
CN122023950A
Image recognition-based fundus color photograph retinopathy screening system
CN122023950B
AI interaction system for bird scientific research
CN122115488A
An AI interactive system for ornithological research
CN122115488B
