A real-time AI identification and positioning method and system for birds around offshore wind farms

Abstract

The present invention provides a real-time AI-powered bird identification and location method and system around offshore wind farms. The method includes: creating a small target recognition model for identifying bird objects; calibrating the angles of cameras and wind turbines in the offshore wind farm; capturing surveillance footage using the camera; identifying bird objects in the surveillance footage using the small target recognition model; extracting real-time parameters of the camera, pan / tilt head, and wind turbine; and locating the identified bird objects based on the real-time parameters. This method can efficiently and accurately identify and track birds in complex offshore environments, contributing to the safe operation of wind farms and the protection of the bird ecosystem. It has broad application prospects and social value.

Description

Technical Field

[0001] The present invention belongs to the technical field of bird identification, and specifically relates to a real-time AI identification and positioning method and system for birds around an offshore wind farm. Background Art

[0002] Automatic bird identification systems utilize artificial intelligence (AI) technologies, particularly deep learning and computer vision, to automatically detect, identify, and classify birds. Typically based on deep learning frameworks and recognition models, these systems utilize high-altitude, telephoto cameras to automatically patrol and capture surrounding birds, and then utilize AI computing services for real-time processing and identification. This technology is currently being used for bird surveys in wetlands, nature reserves, and other areas.

[0003] With the rapid development of offshore wind farms, the impact of offshore wind power generation on the migration of surrounding birds has attracted considerable attention. Traditional monitoring methods rely primarily on manual observation and statistics, which are inefficient and prone to large errors. Although bird identification based on computer vision technology is gradually being applied to daily bird monitoring and surveys, its application at sea is rare and subject to numerous limitations. These are mainly due to the following: 1) The difficulty of powering the sea and the limited operating space make it difficult to deploy monitoring equipment such as cameras; 2) The changing environmental conditions at sea significantly reduce the accuracy of bird identification in low-definition field scenes; and 3) The lack of reference objects at sea makes it difficult to further track and locate birds after photographing them. Therefore, there are still many challenges in effectively using AI recognition technology to monitor and analyze birds in real time in offshore wind farm environments. Summary of the Invention

[0004] In response to the deficiencies in the prior art, the present invention provides a real-time AI identification and positioning method and system for birds around offshore wind farms, which can efficiently and accurately identify and track birds in complex offshore environments.

[0005] A real-time AI-based bird identification and positioning method for offshore wind farms, applicable to offshore wind farms comprising cameras and wind turbines, wherein the cameras are mounted via a pan / tilt platform. The method comprises:

[0006] Create a small object recognition model for identifying bird objects;

[0007] Calibrate the angles of cameras and wind turbines in offshore wind farms;

[0008] Use cameras to collect monitoring images;

[0009] Use the small target recognition model to identify bird objects in the monitoring image;

[0010] Extract real-time parameters of camera, gimbal, and fan;

[0011] The identified bird objects are located according to real-time parameters.

[0012] Furthermore, the small target recognition model adopts an improved neural network structure, which is improved by adding a fine-grained feature pyramid to the feature pyramid network.

[0013] Furthermore, the loss function in the small target recognition model is:

[0014]

[0015] Among them, L is the true value of the i-th target, L i is the predicted value of the i-th target, N is the total number of targets, w i is the weight of the i-th target, A small is the preset small target area threshold, A i is the area of the i-th target, and k is the preset adjustment coefficient.

[0016] Furthermore, the real-time parameters include the focal length and field of view of the camera, the direction angle of the gimbal, and the rotation angle of the fan.

[0017] Furthermore, locating the identified bird object according to the real-time parameters specifically includes:

[0018] Determine the position coordinates of the bird object according to the recognition results of the bird object in the adjacent frame monitoring images;

[0019] The flight speed, flight direction angle and actual position of each bird object are calculated based on real-time parameters.

[0020] Furthermore, the flight speed v is expressed as:

[0021]

[0022] Among them, P j is the position coordinate of the bird object in the jth frame monitoring screen, and Δt is the time interval.

[0023] Furthermore, the flight direction angle θ is expressed as:

[0024]

[0025] Among them, x j is the horizontal coordinate of the position of the bird object in the jth frame monitoring screen, y j is the vertical coordinate of the position coordinate of the bird object in the j-th frame monitoring image.

[0026] Furthermore, the actual position R is expressed as:

[0027] R=(R x ,R y,d);

[0028] in, f is the focal length of the camera, α is the camera's field of view, β is the fan's rotation angle, and γ is the azimuth angle of the pan / tilt head. d is the horizontal distance from the camera to the bird, determined by the camera's pitch angle ε and installation height h: d = h·cot(ε).

[0029] Furthermore, after the flight direction angle θ is calculated, the following steps are also included:

[0030] The flight direction angle θ is compensated according to the rotation angle of the fan and the direction angle of the gimbal to obtain the actual flight direction angle θ actual :

[0031] θ actual =θ+β+γ.

[0032] Secondly, a real-time AI-powered bird identification and positioning system around offshore wind farms is developed. This system is applicable to offshore wind farms and includes cameras and wind turbines. The cameras are mounted via a pan / tilt system. The system includes:

[0033] Model creation unit: used for creating a small target recognition model for identifying bird objects;

[0034] Calibration unit: used to calibrate the angles of cameras and wind turbines in offshore wind farms;

[0035] Acquisition unit: used to collect monitoring images using cameras;

[0036] Recognition unit: used to identify bird objects in the monitoring image using a small target recognition model;

[0037] Positioning unit: used to extract the real-time parameters of the camera, gimbal, and fan, and locate the identified bird objects based on the real-time parameters.

[0038] As can be seen from the above technical solutions, the real-time AI-based bird identification and positioning method and system provided by the present invention can efficiently and accurately identify and track birds in complex offshore environments, contributing to the safe operation of wind farms and the protection of the bird ecological environment. It has broad application prospects and social value, which is specifically reflected in the following aspects:

[0039] 1) Effective real-time monitoring of birds around offshore wind farms, effective identification and tracking of bird flight paths, can assist relevant personnel in timely discovering and recording bird migration paths, accurately implementing relevant intervention measures, and reducing the potential threat of wind farms to birds.

[0040] 2) Improve the ecological and environmental supervision mechanism for offshore wind power. Accurate data recording and analysis can help environmental protection departments and wind farm managers formulate scientific protection measures, reduce the interference and harm caused by wind turbines to birds, promote the coordinated development of offshore wind power and marine environmental protection, provide a basis for scientific decision-making to reduce the negative impact of the ecological environment, and provide more ideas for future environmental impact assessments of offshore wind power planning and projects.

[0041] 3) Improve wind farm operational efficiency. Wind farm operators can obtain timely bird activity information, avoiding equipment damage and downtime due to bird collisions, thereby improving wind turbine operating efficiency and reliability. This can also assist wind farm managers in optimizing wind turbine layout and operating strategies, minimizing the impact on bird migration and achieving sustainable development goals. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly describes the drawings required for the specific embodiments or the description of the prior art. Similar elements or parts are generally identified by similar reference numerals throughout the drawings. Elements or parts in the drawings are not necessarily drawn to scale.

[0043] Figure 1 Flowchart of the method for real-time AI identification and positioning of birds around an offshore wind farm provided in the embodiment.

[0044] Figure 2 A block diagram of a real-time AI-based bird identification and positioning system around an offshore wind farm, provided in an embodiment. DETAILED DESCRIPTION

[0045] The following embodiments of the technical solution of the present invention are described in detail with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention and are therefore only examples and are not intended to limit the scope of protection of the present invention. It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0046] It will be understood that when used in this specification and the appended claims, the terms “comprises” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0047] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0048] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0049] Example:

[0050] A real-time AI identification and positioning method for birds around offshore wind farms, applicable to offshore wind farms that include cameras and wind turbines, where the cameras are mounted via a pan / tilt platform; see Figure 1 , methods include:

[0051] Create a small object recognition model for identifying bird objects;

[0052] Calibrate the angles of cameras and wind turbines in offshore wind farms;

[0053] Use cameras to collect monitoring images;

[0054] Use the small target recognition model to identify bird objects in the monitoring image;

[0055] Extract real-time parameters of camera, gimbal, and fan;

[0056] The identified bird objects are located according to real-time parameters.

[0057] In this embodiment, since the images of birds flying above the sea are extremely small and their features are not obvious, the method creates a small target recognition model that can recognize small target bird objects.

[0058] While traditional feature pyramid networks can identify objects of varying scales, they are still limited in recognizing small objects. Therefore, this method employs an improved neural network structure for small object recognition. This structure is achieved by adding a fine-grained feature pyramid to the feature pyramid network. This fine-grained feature pyramid extracts and enhances the features of small objects. The fine-grained feature pyramid is represented as: FGFP = Concat(C2, Up(Concat(C3, Up(Concat(C4, Up(C5))))))); where C2, C3, C4, and C5 are feature maps of different scales, Up represents an upsampling operation, and Concat represents the concatenation of the feature maps.

[0059] To enhance the features of small targets, the small target recognition model introduces a small target weight factor, i.e., a small target area threshold, into the loss function. The small target weight factor can be adjusted based on the average size of the targets to be recognized, giving small targets a higher weight. The loss function in the small target recognition model is: Among them, L is the true value of the i-th target, L i is the predicted value of the i-th target, N is the total number of targets, w i is the weight of the i-th target, A small is the preset small target area threshold, A i is the area of the i-th target, and k is the preset adjustment coefficient.

[0060] The small target recognition model can be trained using a multi-scale training strategy. The multi-scale training strategy improves the robustness of the small target recognition model to targets of different scales by randomly scaling the monitoring image at multiple scales. The multi-scale training strategy Re size is expressed as: I / =Re size (I, s), I is the monitoring image before scaling, I / is the scaled surveillance image, and s is the random scaling factor.

[0061] In order to better identify small objects, the small object recognition model can also introduce the attention mechanism of contextual information to enhance the feature expression of small objects by capturing the surrounding environment information. The attention mechanism is expressed as: B = σ(W·(Concat(F c ,F s ))+b), where B is the attention weight, F c is the current feature, F s is the context information, W is the weight of the current feature, b is the bias of the current feature, and σ is the activation function.

[0062] The small target recognition model also introduces a small target aggregation detection layer, which is used to aggregate the features of multiple candidate small targets to improve the detection accuracy of small targets. The small target aggregation detection layer is expressed as: Among them, E agg is the aggregated feature, E m is the mth feature, is the weight of the mth feature, and M is the total number of features. This small target recognition model is efficient and accurate, and can accurately identify and classify birds in complex marine environments, reducing false positives and false negatives.

[0063] In this embodiment, after creating the small target recognition model, the method can capture video footage captured by the camera and calibrate the camera and wind turbine angles based on the video information, for example, defining the angle facing due east as 0 degrees. The camera then captures surveillance footage, and the small target recognition model is used to identify birds within the footage. Real-time parameters of the camera, PTZ, and wind turbine are extracted, and the identified birds are located based on these real-time parameters.

[0064] This real-time AI-powered bird identification and positioning method around offshore wind farms can efficiently and accurately identify and track birds in complex offshore environments, contributing to the safe operation of wind farms and the protection of the bird ecosystem. It has broad application prospects and social value, as embodied in the following aspects:

[0065] 1) Effective real-time monitoring of birds around offshore wind farms, effective identification and tracking of bird flight paths, can assist relevant personnel in timely discovering and recording bird migration paths, accurately implementing relevant intervention measures, and reducing the potential threat of wind farms to birds.

[0066] 2) Improve the ecological and environmental supervision mechanism for offshore wind power. Accurate data recording and analysis can help environmental protection departments and wind farm managers formulate scientific protection measures, reduce the interference and harm caused by wind turbines to birds, promote the coordinated development of offshore wind power and marine environmental protection, provide a basis for scientific decision-making to reduce the negative impact of the ecological environment, and provide more ideas for future environmental impact assessments of offshore wind power planning and projects.

[0067] 3) Improve wind farm operational efficiency. Wind farm operators can obtain timely bird activity information, avoiding equipment damage and downtime due to bird collisions, thereby improving wind turbine operating efficiency and reliability. This can also assist wind farm managers in optimizing wind turbine layout and operating strategies, minimizing the impact on bird migration and achieving sustainable development goals.

[0068] Furthermore, in some embodiments, the real-time parameters include the focal length and field of view of the camera, the direction angle of the gimbal, and the rotation angle of the fan.

[0069] Furthermore, in some embodiments, locating the identified bird object according to the real-time parameters specifically includes:

[0070] Determine the position coordinates of the bird object according to the recognition results of the bird object in the adjacent frame monitoring images;

[0071] The flight speed, flight direction angle and actual position of each bird object are calculated based on real-time parameters.

[0072] In this embodiment, the method can determine the position coordinates P of the bird object based on the recognition results of the bird object in two adjacent frames of monitoring images. j (x j ,y j ), calculate the flight speed, flight direction angle and actual position of each bird object according to the real-time parameters of the camera, gimbal and fan, so that the position coordinates of the bird object in multiple consecutive frames of monitoring images can be obtained, and more comprehensive and accurate monitoring data can be obtained.

[0073] Furthermore, in some embodiments, the flight speed v is expressed as:

[0074]

[0075] Among them, P j is the position coordinate of the bird object in the jth frame monitoring screen, and Δt is the time interval.

[0076] In this embodiment, P j+1 -P j It represents the displacement of the bird object in two adjacent frames of monitoring images, and is used to calculate the flight time of the bird object.

[0077] Furthermore, in some embodiments, the flight direction angle θ is expressed as:

[0078]

[0079] Among them, x j is the horizontal coordinate of the position of the bird object in the jth frame monitoring screen, y j is the vertical coordinate of the position coordinate of the bird object in the j-th frame monitoring image.

[0080] In this embodiment, y j+1 -y j The ordinate component representing the displacement of the bird object in two adjacent frames of monitoring images, x j+1 -x j The horizontal coordinate component representing the displacement of the bird object in two adjacent frames of monitoring images is used to calculate the flight direction angle of the bird object.

[0081] Furthermore, in some embodiments, the actual position R is expressed as:

[0082] R=(R x ,R y ,d);

[0083] in, f is the focal length of the camera, α is the field of view of the camera, β is the rotation angle of the fan, γ is the azimuth angle of the gimbal; d is the horizontal distance from the camera to the bird, which is determined by the camera's pitch angle ε and installation height h: d = h·cot(ε).

[0084] In this embodiment, the method calculates the actual position of the bird object in the actual space based on the focal length and field of view of the camera, the rotation angle of the fan, and the direction angle of the pan / tilt platform.

[0085] Furthermore, in some embodiments, after calculating the flight direction angle θ, the method further includes:

[0086] The flight direction angle θ is compensated according to the rotation angle of the fan and the direction angle of the gimbal to obtain the actual flight direction angle θ actual :

[0087] θ actual =θ+β+γ.

[0088] In this embodiment, considering that the camera and the fan define due east as 0 degrees during calibration, and considering the characteristic of the fan rotating with the wind, the flight direction angle of the bird is compensated in combination with the rotation angle of the fan and the direction angle of the pan-tilt head to ensure the accuracy of recognition.

[0089] A real-time AI identification and positioning system for birds around offshore wind farms, applicable to offshore wind farms, including cameras and wind turbines, with the cameras installed via a pan / tilt platform; see Figure 2 , the system includes:

[0090] Model creation unit: used for creating a small target recognition model for identifying bird objects;

[0091] Calibration unit: used to calibrate the angles of cameras and wind turbines in offshore wind farms;

[0092] Acquisition unit: used to collect monitoring images using cameras;

[0093] Recognition unit: used to identify bird objects in the monitoring image using a small target recognition model;

[0094] Positioning unit: used to extract the real-time parameters of the camera, gimbal, and fan, and locate the identified bird objects based on the real-time parameters.

[0095] In this embodiment, the real-time AI-powered bird identification and location system around offshore wind farms utilizes a modular design, with each functional module independent of the others, making it easy to maintain and upgrade. This allows the system to be expanded based on actual needs, for example, by adding modules to monitor and analyze more environmental parameters. The system also supports integration with other intelligent systems, such as combining meteorological data and other sensor data, to further enhance its intelligence.

[0096] The system provided in the embodiment of the present invention is briefly described. For matters not mentioned in the embodiment part, reference may be made to the corresponding content in the aforementioned embodiment.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and description of the present invention.

Claims

1. A real-time AI identification and positioning method for birds around an offshore wind farm, characterized in that: Applicable to an offshore wind farm comprising a camera and a wind turbine, wherein the camera is mounted via a pan / tilt platform; the method comprises: A small target recognition model for identifying bird objects is created; the loss function in the small target recognition model is: Among them, L is the true value of the i-th target, L i is the predicted value of the i-th target, N is the total number of targets, w i is the weight of the i-th target, A small is the preset small target area threshold, A i is the area of the i-th target, k is the preset adjustment coefficient; Calibrate the angles of the camera and the wind turbine in the offshore wind farm; Using the camera to collect monitoring images; Identifying bird objects in the monitoring image using the small target recognition model; Extract real-time parameters of camera, gimbal, and fan; The identified bird object is positioned according to the real-time parameters.

2. The real-time AI identification and positioning method for birds around an offshore wind farm according to claim 1 is characterized in that: The small target recognition model adopts an improved neural network structure, which is obtained by adding a fine-grained feature pyramid to the feature pyramid network.

3. The real-time AI identification and positioning method for birds around an offshore wind farm according to claim 1 is characterized in that: The real-time parameters include the focal length and field of view of the camera, the direction angle of the pan / tilt platform, and the rotation angle of the fan.

4. The real-time AI identification and positioning method for birds around an offshore wind farm according to claim 3 is characterized in that: Positioning the identified bird object according to the real-time parameters specifically includes: Determine the position coordinates of the bird object according to the recognition results of the bird object in the adjacent frame monitoring images; The flight speed, flight direction angle and actual position of each bird object are calculated according to the real-time parameters.

5. The real-time AI identification and positioning method for birds around an offshore wind farm according to claim 4 is characterized in that: The flight speed v is expressed as: Among them, P j is the position coordinate of the bird object in the jth frame monitoring screen, and Δt is the time interval.

6. The real-time AI identification and positioning method for birds around an offshore wind farm according to claim 4 is characterized in that: The flight direction angle θ is expressed as: Among them, x j is the horizontal coordinate of the position of the bird object in the jth frame monitoring screen, y j is the vertical coordinate of the position coordinate of the bird object in the j-th frame monitoring image.

7. The real-time AI identification and positioning method for birds around an offshore wind farm according to claim 6 is characterized in that: The actual position R is expressed as: R=(R x ,R y ,d); in, f is the focal length of the camera, α is the camera's field of view, β is the fan's rotation angle, and γ is the azimuth angle of the pan / tilt head. d is the horizontal distance from the camera to the bird, determined by the camera's pitch angle ε and installation height h: d = h·cot(ε).

8. The real-time AI identification and positioning method for birds around an offshore wind farm according to claim 7 is characterized in that: After calculating the flight direction angle θ, it also includes: The flight direction angle θ is compensated according to the rotation angle of the fan and the direction angle of the gimbal to obtain the actual flight direction angle θ actual : i actual =θ+β+γ.

9. A real-time AI identification and positioning system for birds around offshore wind farms, characterized by: Applicable to offshore wind farms comprising cameras and wind turbines, wherein the cameras are mounted via a pan / tilt system; the system comprises: Model creation unit: used to create a small target recognition model for identifying bird objects; the loss function in the small target recognition model is: Among them, L is the true value of the i-th target, L i is the predicted value of the i-th target, N is the total number of targets, w i is the weight of the i-th target, A small is the preset small target area threshold, A i is the area of the i-th target, k is the preset adjustment coefficient; Calibration unit: used to calibrate the angles of the camera and the wind turbine in the offshore wind farm; Acquisition unit: used to collect monitoring images using the camera; Recognition unit: used for recognizing bird objects in the monitoring image using the small target recognition model; Positioning unit: used to extract real-time parameters of the camera, pan / tilt head, and fan, and locate the identified bird objects based on the real-time parameters.

Images