Bird voiceprint and visual fusion real-time identification method for oil exploitation operation area

Abstract

The present application relates to the oil exploitation operation area-oriented bird voiceprint and visual fusion real-time identification method, belongs to the oil exploitation operation technical field, the method includes: collecting the current monitoring area of the set nature reserve in the current time segment in the field bird song signal, and converting into voiceprint time-frequency graph; acquire the pertinence visual data of each frame of field monitoring picture of the current monitoring area in the current time segment; using the corresponding lightweight neural network intelligent identification of the set nature reserve in the current time segment in the current monitoring area of various different bird data; the present application aims at the technical problem that bird identification lacks scene adaptability, time integrity and data comprehensiveness, for the set nature reserve distributed with oil exploitation operation area, using the lightweight neural network of custom structure design, at the same time, the intelligent identification of various different bird data is completed, so that the above technical problems are solved.

Description

Technical Field

[0001] This invention belongs to the field of oil extraction operation technology, and in particular relates to a real-time recognition method for bird voiceprint and visual fusion in oil extraction operation areas. Background Technology

[0002] In the field of oil extraction operations, neural networks are subjected to massive amounts of learning, i.e., reinforcement learning, to obtain a stable and reliable content mapping relationship from input content to output content. This is then applied to oil extraction operation scenarios to achieve various data processing effects, including intelligent recognition, analysis, identification, and judgment. For example, based on the content conversion of on-site images, the type or number of birds on-site can be identified to facilitate subsequent bird data statistics, bird protection, or bird defense operations.

[0003] For example, Chinese invention patent publication CN117762259A proposes an intelligent wildlife bird monitoring system and method. The system includes a bird name recognition module, a behavior analysis module, a call recognition module, an environmental analysis module, and a VR interaction module. The bird name recognition module uses a convolutional neural network to display the bird's name through multi-layer convolution and pooling operations. The behavior analysis module uses a recurrent neural network to process time-series data, analyze the bird's movement trajectory and flight pattern, and promptly alert when abnormal bird behavior is detected. The call recognition module acquires call data from multiple angles and frequencies, identifying and classifying different types of calls. The environmental analysis module uses charts and graphs to display real-time trends in meteorological and environmental data, combined with displays of abnormal bird behavior. The VR interaction module uses real geographic data to create a virtual wildlife environment for user interaction.

[0004] For example, Chinese invention patent publication CN117789731A discloses a method, device, computer equipment, and storage medium for bird call recognition. The method includes: acquiring bird call sound data; determining the directional source of the bird call based on a sound source localization algorithm; analyzing the type of bird call using artificial intelligence; and obtaining bird species information based on the type of bird call. By embedding this method into a portable device, the device is small, lightweight, and easy to carry, suitable for outdoor activities and field observation. Users can use it anytime, anywhere to identify birds in their surroundings without additional equipment or complex setup.

[0005] However, the aforementioned existing technologies all suffer from the following three shortcomings: First, intelligent identification of wild birds is based on conventional scenarios and does not consider the special scenarios of nature reserves with oil extraction operations. Parameters such as the size of the oil extraction area, drilling density, and rated drilling speed all have a certain impact on the species and number of birds living in the nature reserve. Furthermore, various related parameters of the nature reserve are also influencing factors to a certain extent. Second, the identified bird data is generally real-time data, making it impossible to perform overall identification of bird data within a time segment, leading to increased identification frequency and costs, as well as overall bias in the identified data. Third, the types of identified bird data are relatively limited, making it impossible to perform simultaneous identification of existing bird species, the number of each species, and the activity area of ​​each bird. These shortcomings seriously affect the effectiveness and efficiency of bird identification in the special scenarios of nature reserves with oil extraction operations. Summary of the Invention

[0006] To address technical challenges in related fields, this invention provides a real-time bird recognition method that integrates voiceprints and visual data for oil extraction areas. This method targets designated nature reserves within oil extraction areas. It employs a lightweight neural network with a customized structure corresponding to the nature reserve, utilizing data including drilling density and rated drilling speed within the oil extraction area, the shortest distance from each monitored area to the oil extraction area, bird voiceprint video images of each monitored area in each time segment, and targeted visual data. This enables intelligent recognition of the number and names of various birds within each monitored area in each time segment of a specific scenario—a nature reserve containing oil extraction areas—as well as the 3D coordinates of each bird's activity area within that time segment. This allows for the simultaneous recognition of multiple bird species within a time segment rather than a single instant, achieving comprehensive synchronous acquisition of bird data for each time segment of the specific scenario. This enhances the scenario adaptability, temporal consistency, and data comprehensiveness of real-time bird recognition in the wild.

[0007] According to the present invention, a real-time recognition method for birds based on a fusion of voiceprint and visual characteristics in oil extraction operation areas is provided, the method comprising:

[0008] Collect wild bird calls in the current time segment of the current monitoring area of ​​the nature reserve, and then convert them into a time-frequency sound pattern with a preset resolution.

[0009] The system captures each frame of the current monitoring area in the current time segment from an overhead view. The red-green component gradient value, black-and-white component gradient value, yellow-and-blue component gradient value, and curvature value of each edge pixel in each frame are used as the targeted visual data for that frame.

[0010] Capture and define the regional correlation parameters corresponding to the nature reserve;

[0011] A lightweight neural network corresponding to a nature reserve is used. Based on the drilling distribution density and rated drilling speed in the oil extraction operation area, the area of ​​the current monitoring area, the shortest distance from the current monitoring area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the voiceprint time-frequency map of the current monitoring area in the current time segment, the targeted visual data of each frame of the on-site monitoring screen in the current time segment, and the relevant parameters of the nature reserve, the system can intelligently identify the number and name of various birds in the current monitoring area in the current time segment, as well as the three-dimensional coordinate data set of the activity area of ​​each bird in the current time segment.

[0012] Among them, the number of times the lightweight neural network corresponding to the nature reserve is trained is positively correlated with the area of ​​the oil extraction operation area within the nature reserve.

[0013] Compared with the prior art, the present invention has at least the following main inventive points:

[0014] Invention Point (1): For nature reserves with oil extraction operations, a lightweight neural network with a customized structure corresponding to the nature reserve is adopted. It uses drilling distribution density and rated drilling speed in the oil extraction operation area, the shortest distance from each monitoring area to the oil extraction operation area, bird call voiceprint video images in each monitoring area in each time segment, and targeted visual data to complete the intelligent identification of the number and name of various birds in each monitoring area in each time segment and the three-dimensional coordinate data set of each bird's activity area in each time segment in the special scenario, namely, the nature reserve with oil extraction operations. Thus, it can realize the synchronous identification of multiple bird data in a time segment rather than a moment in the special scenario, and achieve the synchronous collection of comprehensive bird data matching each time segment of the special scenario, thereby improving the scene adaptability, time integrity and data comprehensiveness of real-time identification of wild birds.

[0015] Invention Point (2): In order to synchronously identify the number and name of various birds in the current monitoring area within the current time segment, as well as the three-dimensional coordinate data set of the activity area of ​​each bird within the current time segment, a lightweight neural network with a customized structure design for a designated nature reserve is introduced. The lightweight neural network corresponding to the designated nature reserve undergoes reinforcement learning operations exceeding or equal to a preset threshold number of times. The number of times it learns is positively correlated with the area occupied by the oil extraction operation area within the designated nature reserve. The lightweight neural network corresponding to the designated nature reserve is a lightweight convolutional neural network. The number of convolutional layers in the lightweight convolutional neural network is proportional to the total number of bird species historically identified in the designated nature reserve. Each convolutional layer in the lightweight convolutional neural network uses the ReLU function as the activation function. The customized structure design at each of the above places ensures the reliability and stability of synchronous identification of various types of wild bird data.

[0016] Invention Point (3): In each learning process of the lightweight neural network corresponding to the designated nature reserve, the number and names of various birds in a certain monitoring area of ​​the designated nature reserve within a certain historical time segment, as well as the three-dimensional coordinate data of the activity area of ​​each bird within the certain historical time segment, are combined into the output content of the lightweight neural network corresponding to the designated nature reserve. The drilling distribution density and rated drilling speed in the oil extraction operation area, the area of ​​the certain monitoring area, the shortest distance from the certain monitoring area to the oil extraction operation area, the duration of the certain historical time segment, the preset resolution, the voiceprint time-frequency map of the certain monitoring area in the certain historical time segment, the targeted visual data of each frame of the on-site monitoring screen of the certain monitoring area in the certain historical time segment, and the various related parameters of the designated nature reserve are used as the input content of the lightweight neural network corresponding to the designated nature reserve to complete this learning, thereby ensuring the reliability and stability of the synchronous identification of various types of wild bird data throughout the entire time segment.

[0017] Invention Point (4): In order to synchronously identify the number and name of various birds in the current monitoring area within the current time segment, as well as the three-dimensional coordinate data set of the activity area of ​​each bird within the current time segment, multi-source basic data is introduced, including the drilling distribution density and rated drilling speed in the oil extraction operation area, the area of ​​the current monitoring area, the shortest distance from the current monitoring area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the voiceprint time-frequency map of the current monitoring area in the current time segment, the targeted visual data of each frame of the on-site monitoring screen of the current monitoring area in the current time segment, and the set of various related parameters of the nature reserve. The full and comprehensive screening of the above multi-source basic data further ensures the reliability and stability of synchronous identification of various types of wild bird data throughout the time segment;

[0018] Invention Point (5): Specifically, in the multi-source basic data used for synchronous identification, the preset resolution is the preset time resolution and the preset frequency resolution. The targeted visual data of each frame of the on-site monitoring screen is the red-green component gradient value, black-white component gradient value, yellow-blue component gradient value and curvature value of each edge pixel in each frame of the on-site monitoring screen. The regional correlation parameters corresponding to the nature reserve are set as the area, longitude information, latitude information of the nature reserve, and the historical sunshine duration, historical average humidity and historical average temperature of the nature reserve on the date of the current time segment. Thus, the customized data structure design of the multi-source basic data used for synchronous identification is completed. Attached Figure Description

[0019] The embodiments of the present invention will now be described with reference to the accompanying drawings, wherein:

[0020] Figure 1 This is a schematic diagram of a working scenario for the real-time recognition method of bird voiceprint and visual fusion for oil extraction operation areas according to the present invention.

[0021] Figure 2 This is a flowchart illustrating the steps of a real-time bird voiceprint and visual fusion recognition method for oil extraction operation areas according to Embodiment 1 of the present invention.

[0022] Figure 3 This is a flowchart illustrating the steps of a real-time bird voiceprint and visual fusion recognition method for oil extraction operation areas according to Embodiment 2 of the present invention.

[0023] Figure 4 This is a flowchart illustrating the steps of a real-time bird voiceprint and visual fusion recognition method for oil extraction operation areas according to Embodiment 3 of the present invention.

[0024] Figure 5 This is a flowchart illustrating the steps of a real-time bird voiceprint and visual fusion recognition method for oil extraction operation areas according to Embodiment 4 of the present invention.

[0025] Figure 6 This is a flowchart illustrating the steps of a real-time bird voiceprint and visual fusion recognition method for oil extraction operation areas according to Embodiment 5 of the present invention. Detailed Implementation

[0026] like Figure 1 The diagram illustrates a working scenario for a real-time bird voiceprint and visual fusion recognition method for oil extraction areas, based on the present invention. The reinforcement learning method proposed in this invention belongs to the field of electronic digital data processing.

[0027] The specific technical process of this invention is as follows:

[0028] Technical Process A: To simultaneously identify the number and names of various birds in the current monitored area within the current time segment, as well as the 3D coordinate data set of the activity area of ​​each bird within the current time segment, a lightweight neural network with a customized structure designed for the designated nature reserve was introduced, such as... Figure 1 As shown;

[0029] For example, the nature reserve is set as the Shandong Yellow River Delta National Nature Reserve, and the nature reserve is set as consisting of multiple monitoring areas and a single oil extraction operation area. Each monitoring area in the Shandong Yellow River Delta National Nature Reserve can be used as the current monitoring area to synchronously identify the number of birds, their names, and the three-dimensional coordinate data of the activity areas of each bird in the current monitoring area within the current time segment.

[0030] like Figure 1 As shown, the diagram presents aerial frames corresponding to each monitoring area within the nature reserve, with one frame for each monitoring area, thus constructing a [data structure / system]. Figure 1 The screen array in the image is obviously more than that. Figure 1 Nine overhead shots in the video;

[0031] Similarly, each frame of the overhead view corresponds to a monitored area, such as... Figure 1 As shown, the following will be performed on each monitoring area, for example, the monitoring area corresponding to the overhead view in the lower left corner of the screen array, as the current monitoring area, to perform real-time identification of wild birds in the present invention.

[0032] Specifically, different lightweight neural networks can be designed for different nature reserves. The lightweight neural networks designed for specific nature reserves are mainly customized in the following aspects:

[0033] First: The lightweight neural network designed for the nature reserve has undergone reinforcement learning operations exceeding or equal to a preset threshold number of times, and the number of times it has been learned is positively correlated with the area of ​​the oil extraction operation zone within the nature reserve.

[0034] For example, if the area of ​​the oil extraction operation area within a nature reserve is set to 10,000 hectares, the lightweight neural network corresponding to the nature reserve will undergo 600 training iterations; if the area of ​​the oil extraction operation area within a nature reserve is set to 20,000 hectares, the lightweight neural network corresponding to the nature reserve will undergo 800 training iterations; if the area of ​​the oil extraction operation area within a nature reserve is set to 30,000 hectares, the lightweight neural network corresponding to the nature reserve will undergo 1,000 training iterations, and so on.

[0035] Second: The lightweight neural network designed for the nature reserve is a type of lightweight convolutional neural network, and the number of convolutional layers in the selected lightweight convolutional neural network is proportional to the total number of bird species historically identified in the nature reserve.

[0036] For example, when the total number of bird species historically identified in the nature reserve is set to 50, the number of convolutional layers in the lightweight convolutional neural network is 1; when the total number of bird species historically identified in the nature reserve is set to 100, the number of convolutional layers in the lightweight convolutional neural network is 2; when the total number of bird species historically identified in the nature reserve is set to 150, the number of convolutional layers in the lightweight convolutional neural network is 3, and so on.

[0037] Third: The selected lightweight convolutional neural network uses the ReLU function as the activation function for each convolutional layer;

[0038] Fourth: In each learning iteration of the lightweight neural network corresponding to the designated nature reserve, the number and names of various birds in a specific monitoring area within the designated nature reserve during a specific historical time segment, along with the 3D coordinates of each bird's activity area during that historical time segment, are collected as the output content of the lightweight neural network corresponding to the designated nature reserve. The drilling distribution density and rated drilling speed within the oil extraction operation area, the area of ​​the specific monitoring area, the shortest distance from the specific monitoring area to the oil extraction operation area, the duration of the specific historical time segment, the preset resolution, the voiceprint time-frequency map of the specific monitoring area during the specific historical time segment, the targeted visual data of each frame of the on-site monitoring footage of the specific monitoring area during the specific historical time segment, and various related parameters of the designated nature reserve are used as the input content of the lightweight neural network corresponding to the designated nature reserve to complete this learning process. This ensures the reliability and stability of the synchronous identification of various types of wild bird data throughout the entire time segment.

[0039] In this way, the reliability and stability of synchronous identification of various types of bird data in the wild are ensured through the customized structural designs described above.

[0040] Technical Process B: To synchronously identify the number and names of various birds in the current monitoring area within the current time segment, as well as the three-dimensional coordinate data set of the activity area of ​​each bird within the current time segment, multi-source basic data was introduced;

[0041] Specifically, the multi-source basic data includes the drilling distribution density and rated drilling speed in the oil extraction operation area, the area of ​​the current monitoring area, the shortest distance from the current monitoring area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the acoustic frequency map of the current monitoring area in the current time segment, the targeted visual data of each frame of the on-site monitoring screen of the current monitoring area in the current time segment, and the associated parameters of the nature reserve.

[0042] like Figure 1 As shown, the multi-source basic data includes the shortest distance from the current monitoring area to the oil extraction operation area, the voiceprint time-frequency map of the current monitoring area in the current time segment, the targeted visual data of each frame of the on-site monitoring screen in the current time segment of the current monitoring area, and also includes auxiliary data 1 and auxiliary data 2.

[0043] exist Figure 1 In the data, auxiliary data 1 includes the drilling distribution density and rated drilling speed within the oil extraction operation area, the area of ​​the current monitoring area, and the associated parameters of the nature reserve. Auxiliary data 2 includes the duration of the current time segment and the preset resolution.

[0044] More specifically, in the multi-source basic data used for synchronous identification, the preset resolution is the preset time resolution and the preset frequency resolution. The targeted visual data for each frame of the on-site monitoring image is the red-green component gradient value, black-and-white component gradient value, yellow-and-blue component gradient value and curvature value of each edge pixel in each frame of the on-site monitoring image. The regional correlation parameters corresponding to the nature reserve are set as the area of ​​the nature reserve, longitude information, latitude information, and the historical sunshine duration, historical average humidity and historical average temperature of the nature reserve on the date of the current time segment. Thus, the customized data structure design of the multi-source basic data used for synchronous identification is completed.

[0045] In this way, after the thorough and comprehensive screening of the above-mentioned multi-source basic data, the reliability and stability of the synchronous identification of various types of wild bird data in segments throughout the entire time period are further guaranteed.

[0046] Technical Process C: Using a lightweight neural network with a customized structure designed for the nature reserve based on Technical Process A, and leveraging the multi-source basic data thoroughly and comprehensively screened by Technical Process B, synchronous intelligent identification of various bird species is achieved, such as... Figure 1 As shown;

[0047] Specifically, the data on various birds that are simultaneously and intelligently identified are the number and names of various birds in the current monitoring area within the current time segment, as well as the set of three-dimensional coordinate data of the activity area of ​​each bird within the current time segment.

[0048] Technical Process D: Based on the various bird data synchronously and intelligently identified in Technical Process C, perform corresponding bird data statistics, bird protection, or bird defense operations, such as... Figure 1 As shown;

[0049] For example, the system receives the number and names of various birds in the current monitoring area within the current time segment, and statistically sets the annual encounter rate of various birds in the nature reserve based on the number and names of various birds in the current monitoring area within the current time segment. Based on the number and names of various birds in the current monitoring area within the current time segment, the system draws a trend analysis chart of the patrol and monitoring quantity of each bird species in the nature reserve, and / or, when there are rare bird species among the various birds in the current monitoring area within the current time segment, the system raises the bird protection level of the current monitoring area within the current time segment.

[0050] Therefore, through the coordinated operation of the above-mentioned technical processes, this invention, targeting nature reserves with distributed oil extraction areas, employs a lightweight neural network with a customized structure corresponding to the nature reserve. It utilizes data including drilling density and rated drilling speed within the oil extraction area, the shortest distance from each monitoring area to the oil extraction area, bird call video images of each monitoring area in each time segment, and targeted visual data. This enables intelligent identification of the number and names of various birds within each monitoring area in each time segment of a specific scenario—a nature reserve with distributed oil extraction areas—as well as the three-dimensional coordinates of each bird's activity area within each time segment. This allows for the simultaneous identification of multiple bird data within a time segment rather than a single instant, achieving comprehensive synchronous collection of bird data for each time segment of the specific scenario. This improves the scene adaptability, temporal integrity, and data comprehensiveness of real-time wild bird identification.

[0051] The key points of this invention are: intelligent identification of bird data in special scenarios of nature reserves with oil extraction operations, overall identification of bird data across the entire time segment, simultaneous identification of various bird data, directional design of different lightweight neural networks for different nature reserves, and targeted screening of multi-source basic data including bird call sound pattern time-frequency maps, targeted visual data, and multiple related parameters of oil extraction operations and nature reserves.

[0052] The following will describe in detail the real-time recognition method for bird voiceprint and visual fusion in oil extraction operation areas according to the present invention through embodiments.

[0053] Example 1

[0054] Figure 2 This is a flowchart illustrating the steps of a real-time bird voiceprint and visual fusion recognition method for oil extraction operation areas according to Embodiment 1 of the present invention.

[0055] like Figure 2 As shown, the real-time recognition method for bird voiceprint and visual fusion in oil extraction operation areas includes the following specific steps:

[0056] Step S21: Collect the wild bird call signals of the current monitoring area of ​​the nature reserve in the current time segment, and then convert them into a soundprint time-frequency map with a preset resolution;

[0057] For example, the nature reserve is set as the Shandong Yellow River Delta National Nature Reserve, and the nature reserve is set as consisting of multiple monitoring areas and a single oil extraction operation area. Each monitoring area in the Shandong Yellow River Delta National Nature Reserve can be used as the current monitoring area to synchronously identify the number of birds, their names, and the three-dimensional coordinate data of the activity areas of each bird in the current monitoring area within the current time segment.

[0058] Specifically, the voiceprint time-frequency diagram is a graphical representation of voiceprint information with time as the horizontal axis and frequency as the vertical axis;

[0059] Thus, in a voiceprint time-frequency diagram with a preset resolution, the preset resolution refers to the time resolution and / or frequency resolution being preset. The value of the time resolution refers to the value between every two adjacent coordinates in the evenly spaced coordinates on the horizontal axis, and the value of the frequency resolution refers to the value between every two adjacent coordinates in the evenly spaced coordinates on the vertical axis.

[0060] Step S22: Take an overhead view of each frame of the current monitoring area in the current time segment, and use the red-green component gradient value, black-and-white component gradient value, yellow-and-blue component gradient value, and curvature value of each edge pixel in each frame of the live monitoring image as the targeted visual data of that frame of the live monitoring image.

[0061] Specifically, each pixel has L component values ​​(red-green component values), A component values ​​(black-white component values), and B component values ​​(yellow-blue component values) in the LAB color space. The value range of each component value is between 0 and 255.

[0062] Step S23: Capture and define the regional correlation parameters corresponding to the nature reserve;

[0063] Specifically, capturing the regional correlation parameters corresponding to the designated nature reserve includes: using multiple different parameter capture components to capture the regional correlation parameters corresponding to the designated nature reserve separately;

[0064] Step S24: Using a lightweight neural network corresponding to the nature reserve, based on the drilling distribution density and rated drilling speed in the oil extraction operation area, the area of ​​the current monitoring area, the shortest distance from the current monitoring area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the voiceprint time-frequency map of the current monitoring area in the current time segment, the targeted visual data of each frame of the on-site monitoring screen in the current monitoring area in the current time segment, and the various related parameters of the nature reserve, the system intelligently identifies the number and name of various birds in the current monitoring area in the current time segment, as well as the three-dimensional coordinate data set of the activity area of ​​each bird in the current time segment.

[0065] Therefore, it can be seen that in the intelligent recognition results of the present invention, the number of birds, their names, and the three-dimensional coordinate data of the activity area of ​​each bird in a specific area and time period are simultaneously identified and obtained;

[0066] Among them, the number of times the lightweight neural network corresponding to the nature reserve is learned is positively correlated with the area of ​​the oil extraction operation area within the nature reserve.

[0067] For example, setting the number of times the lightweight neural network corresponding to the nature reserve is positively correlated with the area of ​​the oil extraction operation area within the nature reserve includes: setting the area of ​​the oil extraction operation area within the nature reserve to 10,000 hectares, and selecting the number of times the lightweight neural network corresponding to the nature reserve is trained to 600 times; setting the area of ​​the oil extraction operation area within the nature reserve to 20,000 hectares, and selecting the number of times the lightweight neural network corresponding to the nature reserve is trained to 800 times; setting the area of ​​the oil extraction operation area within the nature reserve to 30,000 hectares, and selecting the number of times the lightweight neural network corresponding to the nature reserve is trained to 1,000 times, and so on.

[0068] Among them, the distances from the edge of each area of ​​the current monitoring area to the oil extraction operation area are obtained, and the minimum value of each distance is taken as the shortest distance from the current monitoring area to the oil extraction operation area. It is also set that there are multiple monitoring areas within the nature reserve, and the current time segment is based on the current time.

[0069] For example, the current time segment is 9:30 AM, and the current time segment is from 9:30 AM to 9:45 AM, meaning that each time segment lasts for 15 minutes.

[0070] Among them, the captured regional correlation parameters corresponding to the nature reserve include: the area, longitude information, latitude information of the nature reserve, and the historical sunshine duration, historical average humidity and historical average temperature of the nature reserve on the current time segment, which are used as the regional correlation parameters corresponding to the nature reserve.

[0071] For example, when the Shandong Yellow River Delta National Nature Reserve was designated as a nature reserve, its area was 153,000 hectares.

[0072] Among them, the three-dimensional coordinate data set of the activity area of ​​each bird in the current time segment is a three-dimensional coordinate data set composed of each set of three-dimensional coordinate data corresponding to each activity position of each bird in the current time segment, and the center position of the nature reserve is set as the origin of the three-dimensional coordinate system.

[0073] Each pixel in each frame of the on-site monitoring image has an L component value (red and green component value), an A component value (black and white component value), and a B component value (yellow and blue component value) in the LAB color space.

[0074] Specifically, for each edge pixel in each frame of the live monitoring image, the standard deviation of the red and green component values ​​corresponding to each of the surrounding pixels is used as its red and green component gradient value, the standard deviation of the black and white component values ​​corresponding to each of the surrounding pixels is used as its black and white component gradient value, and the standard deviation of the yellow and blue component values ​​corresponding to each of the surrounding pixels is used as its yellow and blue component gradient value.

[0075] Specifically, you can choose to use a gradient calculation formula to simultaneously calculate the gradient values ​​of the red-green component, the black-and-white component, and the yellow-and-blue component for each pixel.

[0076] In each frame of the on-site monitoring image, each edge pixel forms multiple contour curves in each frame of the on-site monitoring image. For each edge pixel in each frame of the on-site monitoring image, the curvature value of its contour curve is taken as the curvature value of its position.

[0077] The lightweight neural network corresponding to the nature reserve is a lightweight convolutional neural network, and the number of convolutional layers in the lightweight convolutional neural network is proportional to the total number of bird species historically identified in the nature reserve. Each convolutional layer in the lightweight convolutional neural network uses the ReLU function as the activation function.

[0078] For example, the number of convolutional layers in the lightweight convolutional neural network is proportional to the total number of bird species historically identified in the nature reserve, including: when the total number of bird species historically identified in the nature reserve is 50, the number of convolutional layers in the lightweight convolutional neural network is 1; when the total number of bird species historically identified in the nature reserve is 100, the number of convolutional layers in the lightweight convolutional neural network is 2; when the total number of bird species historically identified in the nature reserve is 150, the number of convolutional layers in the lightweight convolutional neural network is 3, and so on.

[0079] In each learning iteration of the lightweight neural network corresponding to the designated nature reserve, the number and names of various birds in a specific monitoring area within the designated nature reserve during a specific historical time segment, along with the 3D coordinates of each bird's activity area during that historical time segment, are collected as the output content of the lightweight neural network. The drilling density and rated drilling speed within the oil extraction operation area, the area of ​​the monitoring area, the shortest distance from the monitoring area to the oil extraction operation area, the duration of the historical time segment, the preset resolution, the voiceprint time-frequency map of the monitoring area during the historical time segment, the targeted visual data of each frame of on-site monitoring footage of the monitoring area during the historical time segment, and various related parameters of the designated nature reserve are used as the input content of the lightweight neural network to complete this learning process.

[0080] Specifically, the MATLAB toolbox can be used to test and simulate each learning process performed on the lightweight neural network corresponding to the designated nature reserve.

[0081] In addition, the lightweight neural network corresponding to the nature reserve is set to have undergone more than or equal to a preset threshold number of learning iterations.

[0082] Example 2

[0083] Figure 3 This is a flowchart illustrating the steps of a real-time bird voiceprint and visual fusion recognition method for oil extraction operation areas according to Embodiment 2 of the present invention.

[0084] like Figure 3 As shown, with Figure 2Unlike the previous embodiment, the real-time bird voiceprint and visual fusion recognition method for oil extraction operation areas, after employing a lightweight neural network corresponding to a designated nature reserve, and based on the drilling distribution density and rated drilling speed within the oil extraction operation area, the area of ​​the currently monitored area, the shortest distance from the currently monitored area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the voiceprint time-frequency map of the current monitored area in the current time segment, the targeted visual data of each frame of the on-site monitoring image of the current monitored area in the current time segment, and the associated parameters of the designated nature reserve, intelligently identifies the number and names of various birds in the current monitored area within the current time segment, as well as the three-dimensional coordinate data set of the activity area of ​​each bird within the current time segment. That is, after step S24, the method further includes:

[0085] Step S25: Receive the number and names of various birds in the current monitoring area within the current time segment, and statistically set the annual encounter rate of various birds in the nature reserve based on the number and names of various birds in the current monitoring area within the current time segment;

[0086] Among them, the annual encounter rate of various birds in the nature reserve is set based on the statistics of the number and name of various birds in the current monitoring area within the current time segment. This includes: determining whether the number of various birds in the current monitoring area within the current time segment is used for the statistics of the encounter rate in the new year, based on whether the current time segment is in the new year.

[0087] Specifically, programmable logic devices can be used to determine whether the number of various birds in the current monitored area within the current time segment is included in the count of the encounter rate in the new year, based on whether the current time segment is within the new year.

[0088] Example 3

[0089] Figure 4 This is a flowchart illustrating the steps of a real-time bird voiceprint and visual fusion recognition method for oil extraction operation areas according to Embodiment 3 of the present invention.

[0090] like Figure 4 As shown, with Figure 2Unlike the previous embodiment, the real-time bird voiceprint and visual fusion recognition method for oil extraction operation areas, after employing a lightweight neural network corresponding to a designated nature reserve, and based on the drilling distribution density and rated drilling speed within the oil extraction operation area, the area of ​​the currently monitored area, the shortest distance from the currently monitored area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the voiceprint time-frequency map of the current monitored area in the current time segment, the targeted visual data of each frame of the on-site monitoring image of the current monitored area in the current time segment, and the associated parameters of the designated nature reserve, intelligently identifies the number and names of various birds in the current monitored area within the current time segment, as well as the three-dimensional coordinate data set of the activity area of ​​each bird within the current time segment. That is, after step S24, the method further includes:

[0091] Step S26: Receive the number and names of various birds in the current monitoring area within the current time segment, and draw a trend analysis chart of the patrol and monitoring number of each bird species in the current nature reserve based on the number and names of various birds in the current monitoring area within the current time segment.

[0092] Among them, the number and name of various birds in the current monitoring area within the current time segment are used to draw the patrol and monitoring number trend analysis chart for each bird species in the nature reserve, including drawing different patrol and monitoring number trend analysis charts for different bird species.

[0093] Specifically, the process of drawing a patrol and monitoring trend analysis chart for each bird species in the nature reserve based on the number and name of various birds in the current monitoring area within the current time segment also includes storing the different patrol and monitoring trend analysis charts drawn for different bird species in different physical storage spaces.

[0094] Example 4

[0095] Figure 5 This is a flowchart illustrating the steps of a real-time bird voiceprint and visual fusion recognition method for oil extraction operation areas according to Embodiment 4 of the present invention.

[0096] like Figure 5 As shown, with Figure 2 Unlike the previous embodiment, in the real-time recognition method for bird voiceprint and visual fusion for oil extraction operation areas, before collecting wild bird call signals in the current time segment of the current monitoring area of ​​the designated nature reserve and converting them into a voiceprint time-frequency map with a preset resolution, i.e. before step S21, the method further includes:

[0097] Step S27: Perform each learning iteration on the lightweight neural network to obtain the lightweight neural network after each learning iteration, and use it as the output of the lightweight neural network corresponding to the designated nature reserve;

[0098] Specifically, the lightweight neural network corresponding to the nature reserve is represented by the numerical values ​​of the various network parameters of the lightweight neural network corresponding to the nature reserve.

[0099] The process of performing each learning iteration on the lightweight neural network to obtain the lightweight neural network after each learning iteration and using it as the output of the lightweight neural network corresponding to the designated nature reserve includes: completing the output of the lightweight neural network corresponding to the designated nature reserve by outputting the various network parameters of the lightweight neural network corresponding to the designated nature reserve.

[0100] Example 5

[0101] Figure 6 This is a flowchart illustrating the steps of a real-time bird voiceprint and visual fusion recognition method for oil extraction operation areas according to Embodiment 5 of the present invention.

[0102] like Figure 6 As shown, with Figure 2 Unlike the previous embodiment, the real-time bird voiceprint and visual fusion recognition method for oil extraction operation areas, after employing a lightweight neural network corresponding to a designated nature reserve, and based on the drilling distribution density and rated drilling speed within the oil extraction operation area, the area of ​​the currently monitored area, the shortest distance from the currently monitored area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the voiceprint time-frequency map of the current monitored area in the current time segment, the targeted visual data of each frame of the on-site monitoring image of the current monitored area in the current time segment, and the associated parameters of the designated nature reserve, intelligently identifies the number and names of various birds in the current monitored area within the current time segment, as well as the three-dimensional coordinate data set of the activity area of ​​each bird within the current time segment. That is, after step S24, the method further includes:

[0103] Step S28: Receive the number and names of various birds in the current monitoring area within the current time segment, as well as the three-dimensional coordinate data set of the activity area of ​​each bird within the current time segment. If there are rare bird species among the various birds in the current monitoring area within the current time segment, upgrade the bird protection level of the current monitoring area within the current time segment.

[0104] Conversely, if there are no rare bird species among the various bird species present in the current monitoring area in the current time segment, the bird protection level in the current monitoring area in the current time segment can be reduced or maintained.

[0105] Among the various bird species present in the current monitoring area within the current time segment, when rare bird species are present, the bird protection level of the current monitoring area within the current time segment is improved by: the improved bird protection level of the current monitoring area within the current time segment is positively correlated with the rare species level of the present rare bird species.

[0106] Next, the various method embodiments of the present invention will be described in detail.

[0107] In the real-time bird voiceprint and visual fusion recognition method for oil extraction operation areas according to various method embodiments of the present invention:

[0108] Collecting wild bird calls in the current time segment of the current monitoring area of ​​the nature reserve and converting them into a time-frequency sound pattern with a preset resolution includes: collecting wild bird calls in the current time segment of the current monitoring area of ​​the nature reserve and converting the wild bird calls into a time-frequency sound pattern with a preset resolution using short-time Fourier transform.

[0109] Specifically, an FPGA chip designed using VHDL language can be selected to convert the wild bird song signal into a time-frequency sound pattern with a preset resolution using short-time Fourier transform.

[0110] Among them, the voiceprint time-frequency with preset resolution is a voiceprint time-frequency map with preset time resolution and preset frequency resolution;

[0111] The process of collecting wild bird calls in the current time segment of the current monitoring area of ​​the designated nature reserve and converting them into a voiceprint time-frequency map with a preset resolution also includes: collecting wild bird calls in the current time segment of the current monitoring area of ​​the designated nature reserve through an audio sensor set at the center of the current monitoring area of ​​the designated nature reserve.

[0112] In the real-time bird voiceprint and visual fusion recognition method for oil extraction operation areas according to various method embodiments of the present invention:

[0113] The above-ground images of the current monitoring area in the current time segment include: the above-ground images of the current monitoring area in the current time segment are captured by an ultra-high-definition camera device set directly above the center of the current monitoring area of ​​the designated nature reserve.

[0114] Specifically, the overhead shooting of each frame of the current monitoring area in the current time segment by using an ultra-high-definition camera device set directly above the center of the current monitoring area of ​​the designated nature reserve includes: using a drone platform as an ultra-high-definition camera device to shoot each frame of the current monitoring area in the current time segment.

[0115] Specifically, the overhead shooting of each frame of the current monitoring area in the current time segment by the ultra-high-definition camera device set directly above the center of the current monitoring area of ​​the designated nature reserve also includes: fixed overhead shooting height, that is, the flight altitude of the drone platform carrying the drone is fixed when the drone takes overhead shots.

[0116] Among them, the aerial view of the current monitoring area in the current time segment also includes: the number of frames of the selected on-site monitoring images in the current time segment follows the numerical trend of the total number of bird species historically identified in the nature reserve.

[0117] And in the real-time recognition method for bird voiceprint and visual fusion for oil extraction operation areas according to various method embodiments of the present invention:

[0118] The drilling distribution density and rated drilling speed within the oil extraction operation area, the area of ​​the current monitoring area, the shortest distance from the current monitoring area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the acoustic frequency map of the current monitoring area in the current time segment, the targeted visual data of each frame of the on-site monitoring screen of the current monitoring area in the current time segment, and the relevant parameters of the designated nature reserve are synchronously input into the lightweight neural network corresponding to the designated nature reserve. The lightweight neural network corresponding to the designated nature reserve is then run to obtain the number and names of various birds in the current monitoring area within the current time segment, as well as the three-dimensional coordinate data set of the activity area of ​​each bird within the current time segment, output by the lightweight neural network corresponding to the designated nature reserve.

[0119] For example, GAL devices can be selected to perform synchronous input of drilling distribution density and rated drilling speed in the oil extraction operation area, the area of ​​the current monitoring area, the shortest distance from the current monitoring area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the acoustic frequency map of the current monitoring area in the current time segment, the targeted visual data of each frame of the on-site monitoring screen of the current monitoring area in the current time segment, and the setting of various related parameters of the nature reserve.

[0120] Among them, the number and names of various birds in the current monitoring area within the current time segment, the three-dimensional coordinate data set of the activity area of ​​each bird within the current time segment, the drilling distribution density and rated drilling speed in the oil extraction operation area, the area of ​​the current monitoring area, the shortest distance from the current monitoring area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the voiceprint time-frequency map of the current monitoring area within the current time segment, the targeted visual data of each frame of the on-site monitoring screen of the current monitoring area within the current time segment, and the various related parameters of the nature reserve are all in the form of numerical representation after binary conversion.

[0121] The lightweight neural network corresponding to the nature reserve has multiple input ports, which are used to synchronously input the drilling distribution density and rated drilling speed in the oil extraction operation area, the area of ​​the current monitoring area, the shortest distance from the current monitoring area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the voiceprint time-frequency map of the current monitoring area in the current time segment, the targeted visual data of each frame of the on-site monitoring screen of the current monitoring area in the current time segment, and the various related parameters of the nature reserve.

[0122] In addition, the following technical content can be cited to further highlight the essential features of the present invention:

[0123] The frame rate of the selected on-site monitoring footage in the current time segment follows the numerical trend of the total number of bird species historically identified in the set nature reserve, including: using a frame rate change curve to represent the numerical trend of the frame rate of the selected on-site monitoring footage in the current time segment, and using a total number change curve to represent the numerical trend of the total number of bird species historically identified in the set nature reserve.

[0124] The selection of the number of frames in the current time segment of the on-site monitoring footage to follow the numerical trend of the total number of bird species historically identified in the nature reserve also includes: normalizing the length of the frame number change curve and the total number change curve to obtain the first processed curve and the second processed curve.

[0125] Among them, the number of frames of the selected on-site monitoring footage in the current time segment that follows the numerical trend of the total number of bird species historically identified in the nature reserve also includes: the first processed curve and the second processed curve are of equal length and completely overlap.

[0126] For example, a programmable logic device can be selected to normalize the length of the frame number change curve and the total number change curve to obtain a first processed curve and a second processed curve. The programmable logic device is programmed using VHDL.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A real-time recognition method for birds using a fusion of voiceprint and visual characteristics in oil extraction areas, characterized in that... The method includes: Collect wild bird calls in the current time segment of the current monitoring area of ​​the nature reserve, and then convert them into a time-frequency sound pattern with a preset resolution. The system captures each frame of the current monitoring area in the current time segment from an overhead view. The red-green component gradient value, black-and-white component gradient value, yellow-and-blue component gradient value, and curvature value of each edge pixel in each frame are used as the targeted visual data for that frame. Capture and define the regional correlation parameters corresponding to the nature reserve; A lightweight neural network corresponding to a nature reserve is used. Based on the drilling distribution density and rated drilling speed in the oil extraction operation area, the area of ​​the current monitoring area, the shortest distance from the current monitoring area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the voiceprint time-frequency map of the current monitoring area in the current time segment, the targeted visual data of each frame of the on-site monitoring screen in the current time segment, and the relevant parameters of the nature reserve, the system can intelligently identify the number and name of various birds in the current monitoring area in the current time segment, as well as the three-dimensional coordinate data set of the activity area of ​​each bird in the current time segment. Among them, the number of times the lightweight neural network corresponding to the nature reserve is learned is positively correlated with the area of ​​the oil extraction operation area within the nature reserve. Among them, the distances from the edge of each area of ​​the current monitoring area to the oil extraction operation area are obtained, and the minimum value of each distance is taken as the shortest distance from the current monitoring area to the oil extraction operation area. It is also set that there are multiple monitoring areas within the nature reserve, and the current time segment is based on the current time. The lightweight neural network corresponding to the nature reserve is a lightweight convolutional neural network, and the number of convolutional layers in the lightweight convolutional neural network is proportional to the total number of bird species historically identified in the nature reserve. Each convolutional layer in the lightweight convolutional neural network uses the ReLU function as the activation function. Among them, the number of frames of the selected on-site monitoring footage in the current time segment follows the trend of the total number of bird species historically identified in the nature reserve. The frame number change curve represents the numerical trend of the frame number of the selected on-site monitoring footage in the current time segment, and the total number change curve represents the numerical trend of the total number of bird species historically identified in the nature reserve. The frame number change curve and the total number change curve are normalized to obtain a first processed curve and a second processed curve. The first processed curve and the second processed curve are of equal length and completely overlap. Where S represents the area of ​​the oil extraction operation area within the nature reserve, in hectares, and T represents the number of times the lightweight neural network corresponding to the nature reserve has been trained, then T = 0.02 × S + 400, where T and S are both positive integers. Where N = 50 × L, N represents the total number of bird species historically identified in the nature reserve, in species, and L represents the number of convolutional layers in the lightweight convolutional neural network, in layers. Both N and L are positive integers.

2. The real-time recognition method for bird voiceprint and visual fusion in oil extraction operation areas as described in claim 1, characterized in that: The captured regional correlation parameters corresponding to the designated nature reserve include: the area, longitude, latitude, historical sunshine duration, historical average humidity, and historical average temperature of the designated nature reserve on the current time segment, and these are used as the regional correlation parameters corresponding to the designated nature reserve. Among them, the three-dimensional coordinate data set of the activity area of ​​each bird in the current time segment is a three-dimensional coordinate data set composed of each set of three-dimensional coordinate data corresponding to each activity position of each bird in the current time segment, and the center position of the nature reserve is set as the origin of the three-dimensional coordinate system. Each pixel in each frame of the on-site monitoring image has an L component value (red and green component value), an A component value (black and white component value), and a B component value (yellow and blue component value) in the LAB color space. Specifically, for each edge pixel in each frame of the live monitoring image, the standard deviation of the red and green component values ​​corresponding to each of the surrounding pixels is used as its red and green component gradient value, the standard deviation of the black and white component values ​​corresponding to each of the surrounding pixels is used as its black and white component gradient value, and the standard deviation of the yellow and blue component values ​​corresponding to each of the surrounding pixels is used as its yellow and blue component gradient value. In each frame of the on-site monitoring image, the edge pixels form multiple contour curves within that frame. For each edge pixel in each frame, the curvature value of its corresponding contour curve is used as the curvature value of its location.

3. The real-time recognition method for bird voiceprint and visual fusion in oil extraction operation areas as described in claim 2, characterized in that: In each learning iteration of the lightweight neural network corresponding to the designated nature reserve, the number and names of various birds in a specific monitoring area within the designated nature reserve during a specific historical time segment, along with the 3D coordinates of each bird's activity area during that historical time segment, are collected as the output content of the lightweight neural network corresponding to the designated nature reserve. The drilling distribution density and rated drilling speed within the oil extraction operation area, the area of ​​the specific monitoring area, the shortest distance from the specific monitoring area to the oil extraction operation area, the duration of the specific historical time segment, the preset resolution, the voiceprint time-frequency map of the specific monitoring area during the specific historical time segment, the targeted visual data of each frame of on-site monitoring footage of the specific monitoring area during the specific historical time segment, and various related parameters of the designated nature reserve are used as the input content of the lightweight neural network corresponding to the designated nature reserve to complete this learning iteration. Among them, the number of times the lightweight neural network corresponding to the nature reserve has been trained exceeds or equals a preset threshold.

4. The real-time recognition method for bird voiceprint and visual fusion in oil extraction operation areas as described in claim 3, characterized in that, After employing a lightweight neural network corresponding to a designated nature reserve, and based on the drilling distribution density and rated drilling speed within the oil extraction operation area, the area of ​​the currently monitored area, the shortest distance from the current monitored area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the acoustic frequency map of the current monitored area in the current time segment, the targeted visual data of each frame of the on-site monitoring image of the current monitored area in the current time segment, and various related parameters of the designated nature reserve, the method further includes: Receive the number and names of various birds in the current monitoring area within the current time segment, and statistically set the annual encounter rate of various birds in the nature reserve based on the number and names of various birds in the current monitoring area within the current time segment; Among them, the annual encounter rate of various birds in the nature reserve is set based on the statistics of the number and names of various birds in the current monitoring area within the current time segment. This includes determining whether the number of various birds in the current monitoring area within the current time segment is used for the statistics of the encounter rate in the new year, based on whether the current time segment is in the new year.

5. The real-time recognition method for bird voiceprint and visual fusion in oil extraction operation areas as described in claim 3, characterized in that, After employing a lightweight neural network corresponding to a designated nature reserve, and based on the drilling distribution density and rated drilling speed within the oil extraction operation area, the area of ​​the currently monitored area, the shortest distance from the current monitored area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the acoustic frequency map of the current monitored area in the current time segment, the targeted visual data of each frame of the on-site monitoring image of the current monitored area in the current time segment, and various related parameters of the designated nature reserve, the method further includes: Receive the number and names of various birds in the current monitoring area within the current time segment, and draw a trend analysis chart of the patrol and monitoring numbers of each bird species in the designated nature reserve based on the number and names of various birds in the current monitoring area within the current time segment; Among them, the number and name of various birds in the current monitoring area within the current time segment are used to draw the patrol and monitoring quantity trend analysis chart for each bird species in the nature reserve, including drawing different patrol and monitoring quantity trend analysis charts for different bird species.

6. The real-time recognition method for bird voiceprint and visual fusion in oil extraction operation areas as described in claim 3, characterized in that, Before collecting wild bird call signals from the current monitoring area of ​​the designated nature reserve in the current time segment and converting them into a voiceprint time-frequency map with a preset resolution, the method further includes: Perform each learning iteration on the lightweight neural network to obtain the lightweight neural network after each learning iteration, and use it as the output of the lightweight neural network corresponding to the designated nature reserve; The process of performing each learning iteration on the lightweight neural network to obtain the lightweight neural network after each learning iteration and using it as the output of the lightweight neural network corresponding to the designated nature reserve includes: completing the output of the lightweight neural network corresponding to the designated nature reserve by outputting the various network parameters of the lightweight neural network corresponding to the designated nature reserve.

7. The real-time recognition method for bird voiceprint and visual fusion in oil extraction operation areas as described in claim 3, characterized in that, After employing a lightweight neural network corresponding to a designated nature reserve, and based on the drilling distribution density and rated drilling speed within the oil extraction operation area, the area of ​​the currently monitored area, the shortest distance from the current monitored area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the acoustic frequency map of the current monitored area in the current time segment, the targeted visual data of each frame of the on-site monitoring image of the current monitored area in the current time segment, and various related parameters of the designated nature reserve, the method further includes: Receive the number and names of various birds in the current monitoring area within the current time segment, as well as the three-dimensional coordinate data set of the activity area of ​​each bird within the current time segment. If there are rare bird species among the various birds in the current monitoring area within the current time segment, upgrade the bird protection level of the current monitoring area within the current time segment. Among the various bird species present in the current monitoring area within the current time segment, when rare bird species are present, the bird protection level of the current monitoring area within the current time segment is improved by: the improved bird protection level of the current monitoring area within the current time segment is positively correlated with the rare species level of the present rare bird species.

8. The real-time recognition method for bird voiceprint and visual fusion in oil extraction operation areas as described in any one of claims 3-7, characterized in that: Collecting wild bird calls in the current time segment of the current monitoring area of ​​the nature reserve and converting them into a time-frequency sound pattern with a preset resolution includes: collecting wild bird calls in the current time segment of the current monitoring area of ​​the nature reserve and converting the wild bird calls into a time-frequency sound pattern with a preset resolution using short-time Fourier transform. Among them, the voiceprint time-frequency with preset resolution is a voiceprint time-frequency map with preset time resolution and preset frequency resolution; The process of collecting wild bird calls in the current time segment of the current monitoring area of ​​the designated nature reserve and converting them into a voiceprint time-frequency map with a preset resolution also includes: collecting wild bird calls in the current time segment of the current monitoring area of ​​the designated nature reserve through an audio sensor set at the center of the current monitoring area of ​​the designated nature reserve.

9. The real-time recognition method for bird voiceprint and visual fusion in oil extraction operation areas as described in any one of claims 3-7, characterized in that: The overhead view of each frame of the current monitoring area in the current time segment includes: overhead views of each frame of the current monitoring area in the current time segment are captured by an ultra-high-definition camera set directly above the center of the current monitoring area in the designated nature reserve.

10. The real-time recognition method for bird voiceprint and visual fusion in oil extraction operation areas as described in any one of claims 3-7, characterized in that: The drilling distribution density and rated drilling speed within the oil extraction operation area, the area of ​​the current monitoring area, the shortest distance from the current monitoring area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the acoustic frequency map of the current monitoring area in the current time segment, the targeted visual data of each frame of the on-site monitoring screen of the current monitoring area in the current time segment, and the relevant parameters of the designated nature reserve are synchronously input into the lightweight neural network corresponding to the designated nature reserve. The lightweight neural network corresponding to the designated nature reserve is then run to obtain the number and names of various birds in the current monitoring area within the current time segment, as well as the three-dimensional coordinate data set of the activity area of ​​each bird within the current time segment, output by the lightweight neural network corresponding to the designated nature reserve. Among them, the number and names of various birds in the current monitoring area within the current time segment, the three-dimensional coordinate data set of the activity area of ​​each bird within the current time segment, the drilling distribution density and rated drilling speed in the oil extraction operation area, the area of ​​the current monitoring area, the shortest distance from the current monitoring area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the voiceprint time-frequency map of the current monitoring area within the current time segment, the targeted visual data of each frame of the on-site monitoring screen of the current monitoring area within the current time segment, and the various related parameters of the nature reserve are all in the form of numerical representation after binary conversion. The lightweight neural network corresponding to the nature reserve has multiple input ports, which are used to synchronously input the drilling distribution density and rated drilling speed in the oil extraction operation area, the area of ​​the current monitoring area, the shortest distance from the current monitoring area to the oil extraction operation area, the duration of the current time segment, the preset resolution, the voiceprint time-frequency map of the current monitoring area in the current time segment, the targeted visual data of each frame of the on-site monitoring screen of the current monitoring area in the current time segment, and the various related parameters of the nature reserve.

Images