Real-time AI identification and positioning method and system for birds around offshore wind plant

Abstract

The invention provides a real-time AI identification and positioning method and system for birds around an offshore wind plant. The method comprises the following steps: creating a small target identification model for identifying bird objects; calibrating angles of a camera and a fan in the offshore wind power plant; collecting a monitoring picture by using a camera; recognizing bird objects in the monitoring picture by using a small target recognition model; extracting real-time parameters of the camera, the holder and the fan; and positioning the identified bird object according to the real-time parameters. According to the method, the birds can be efficiently and accurately recognized and tracked in a complex marine environment, safe operation of a wind power plant and protection of the ecological environment of the birds are facilitated, and the method has wide application prospects and social value.

Description

Technical Field

[0001] The present invention belongs to the technical field of bird identification, and in particular relates to a method and system for real-time AI identification and positioning of birds around an offshore wind farm. Background Art

[0002] The bird automatic identification system is a system that uses artificial intelligence technology, especially deep learning and computer vision, to automatically detect, identify and classify birds. The bird automatic identification system is usually based on a deep learning framework and recognition model, using a high-altitude telephoto camera to automatically cruise and capture surrounding birds, and uses AI computing services for real-time processing and identification. This technology has been used for bird surveys in wetlands, nature reserves and other areas.

[0003] With the rapid development of offshore wind farms, the impact of offshore wind power generation on the migration of surrounding birds has attracted much attention. Traditional monitoring methods mainly rely on manual observation and statistics, which are inefficient and have large errors. Although bird identification based on computer vision technology has gradually been applied to daily bird monitoring and investigation, it is rarely used at sea and has many limiting factors, mainly because: 1) It is difficult to provide power at sea and the operating space is limited, which makes it difficult to deploy monitoring equipment such as cameras; 2) The environmental conditions at sea are changeable, and the accuracy of bird identification is greatly reduced in low-definition field scenes; 3) There is a lack of reference objects at sea, and it is difficult to further track and locate birds after shooting. Therefore, there are still many challenges in how to effectively use AI recognition technology to monitor and analyze birds in real time in an offshore wind farm environment. Summary of the invention

[0004] In view of the defects in the prior art, the present invention provides a real-time AI identification and positioning method and system for birds around an offshore wind farm, which can efficiently and accurately identify and track birds in a complex offshore environment.

[0005] A real-time AI recognition and positioning method for birds around an offshore wind farm, applicable to an offshore wind farm including a camera and a wind turbine, wherein the camera is installed via a pan / tilt platform; the method comprises:

[0006] Create a small object recognition model for identifying bird objects;

[0007] Calibrate the angles of cameras and wind turbines in offshore wind farms;

[0008] Use cameras to collect monitoring images;

[0009] Use the small target recognition model to identify bird objects in the monitoring screen;

[0010] Extract real-time parameters of camera, gimbal and fan;

[0011] The identified bird objects are located according to real-time parameters.

[0012] Furthermore, the small target recognition model adopts an improved neural network structure, and the improved neural network structure is improved by adding a fine-grained feature pyramid in the feature pyramid network.

[0013] Furthermore, the loss function in the small target recognition model is:

[0014]

[0015] Among them, L is the true value of the i-th target, L i is the predicted value of the i-th target, N is the total number of targets, and w i is the weight of the ith target, A small is the preset small target area threshold, A i is the area of ​​the ith target, and k is the preset adjustment coefficient.

[0016] Furthermore, the real-time parameters include the focal length and field of view of the camera, the direction angle of the gimbal, and the rotation angle of the fan.

[0017] Furthermore, positioning the identified bird object according to the real-time parameters specifically includes:

[0018] Determine the position coordinates of the bird object according to the recognition results of the bird object in the adjacent frame monitoring screen;

[0019] The flight speed, flight direction angle and actual position of each bird object are calculated based on real-time parameters.

[0020] Furthermore, the flight speed v is expressed as:

[0021]

[0022] Among them, P j is the position coordinate of the bird object in the jth frame of the monitoring screen, and Δt is the time interval.

[0023] Furthermore, the flight direction angle θ is expressed as:

[0024]

[0025] Among them, x j is the horizontal coordinate of the position of the bird object in the jth frame of the monitoring screen, y j It is the ordinate of the position coordinate of the bird object in the jth frame monitoring screen.

[0026] Furthermore, the actual position R is expressed as:

[0027] R=(R x ,R y,d);

[0028] in, f is the focal length of the camera, α is the field of view of the camera, β is the rotation angle of the fan, and γ is the direction angle of the pan / tilt. d is the horizontal distance from the camera to the bird, which is determined according to the pitch angle ε of the camera and the installation height h: d = h·cot(ε).

[0029] Furthermore, after the flight direction angle θ is calculated, the following is further included:

[0030] The flight direction angle θ is compensated according to the rotation angle of the fan and the direction angle of the gimbal to obtain the actual flight direction angle θ actual :

[0031] θ actual =θ+β+γ.

[0032] In the second aspect, a real-time AI recognition and positioning system for birds around an offshore wind farm is applicable to an offshore wind farm including a camera and a wind turbine, wherein the camera is installed via a pan / tilt platform; the system includes:

[0033] Model creation unit: used for creating a small target recognition model for identifying bird objects;

[0034] Calibration unit: used to calibrate the angles of cameras and wind turbines in offshore wind farms;

[0035] Collection unit: used to collect monitoring images using cameras;

[0036] Recognition unit: used to identify bird objects in the monitoring screen using a small target recognition model;

[0037] Positioning unit: used to extract the real-time parameters of the camera, gimbal, and fan, and locate the identified bird objects based on the real-time parameters.

[0038] It can be seen from the above technical solutions that the real-time AI identification and positioning method and system for birds around offshore wind farms provided by the present invention can efficiently and accurately identify and track birds in a complex offshore environment, which is conducive to the safe operation of wind farms and the protection of the bird ecological environment. It has broad application prospects and social value, which is specifically reflected in the following aspects:

[0039] 1) Effective real-time monitoring of birds around offshore wind farms, effective identification and tracking of bird flight paths, can assist relevant personnel in timely discovering and recording bird migration paths, accurately taking relevant intervention measures, and reducing the potential threat of wind farms to birds.

[0040] 2) Improve the ecological and environmental supervision mechanism for offshore wind power. Through accurate data recording and analysis, it can help environmental protection departments and wind farm managers formulate scientific protection measures, reduce the interference and harm of wind turbines to birds, promote the coordinated development of offshore wind power and marine environmental protection, and provide a basis for making scientific decisions to reduce the negative impact of the ecological environment. At the same time, it also provides more ideas for future offshore wind power planning and project environmental assessments.

[0041] 3) Improve wind farm operation efficiency. Wind farm operators can obtain bird activity information in a timely manner to avoid equipment damage and downtime due to bird collisions, thereby improving wind turbine operation efficiency and reliability. At the same time, it can assist wind farm managers in optimizing wind turbine layout and operation strategies, minimize the impact on bird migration, and achieve the goal of sustainable development. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0043] Figure 1 A flowchart of a method for real-time AI identification and positioning of birds around an offshore wind farm provided in an embodiment.

[0044] Figure 2 A block diagram of a real-time AI identification and positioning system for birds around an offshore wind farm provided in an embodiment. DETAILED DESCRIPTION

[0045] The embodiments of the technical solution of the present invention will be described in detail below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are therefore only used as examples, and the protection scope of the present invention cannot be limited thereto. It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should be the common meanings understood by those skilled in the art to which the present invention belongs.

[0046] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0047] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0048] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0049] Example:

[0050] A real-time AI recognition and positioning method for birds around an offshore wind farm, applicable to an offshore wind farm including a camera and a wind turbine, wherein the camera is installed via a pan / tilt platform; see Figure 1 , methods include:

[0051] Create a small object recognition model for identifying bird objects;

[0052] Calibrate the angles of cameras and wind turbines in offshore wind farms;

[0053] Use cameras to collect monitoring images;

[0054] Use the small target recognition model to identify bird objects in the monitoring screen;

[0055] Extract real-time parameters of camera, gimbal and fan;

[0056] The identified bird objects are located according to real-time parameters.

[0057] In this embodiment, since the images of birds flying above the sea are particularly small and their features are not obvious, the method creates a small target recognition model that can recognize small target bird objects.

[0058] Since the traditional feature pyramid network can identify targets of different scales, but it is still insufficient for identifying small targets, the small target recognition model of this method adopts an improved neural network structure, which is improved by adding a fine-grained feature pyramid to the feature pyramid network. The features of small targets are extracted and enhanced through the fine-grained feature pyramid. The fine-grained feature pyramid is expressed as: FGFP = Concat (C2, Up (Concat (C3, Up (Concat (C4, Up (C5))))))); where C2, C3, C4, C5 are feature maps of different scales, Up is an upsampling operation, and Concat is a concatenation operation of feature maps.

[0059] In order to enhance the features of small targets, the small target recognition model introduces a small target weight factor, i.e., a small target area threshold, into the loss function. The small target weight factor can be adjusted according to the average size of the target to be recognized, giving small targets a higher weight. The loss function in the small target recognition model is: Among them, L is the true value of the i-th target, L i is the predicted value of the i-th target, N is the total number of targets, and w i is the weight of the ith target, A small is the preset small target area threshold, A i is the area of ​​the ith target, and k is the preset adjustment coefficient.

[0060] The small target recognition model can be trained using a multi-scale training strategy. The multi-scale training strategy improves the robustness of the small target recognition model to targets of different scales by randomly scaling the monitoring screen at multiple scales. The multi-scale training strategy Re size is expressed as: / =Re size (I, s), I is the monitoring image before scaling, I / is the scaled surveillance image, and s is the random scaling factor.

[0061] In order to better identify small objects, the small object recognition model can also introduce the attention mechanism of contextual information to enhance the feature expression of small objects by capturing the surrounding environment information. The attention mechanism is expressed as: B = σ(W·(Concat(F c ,F s ))+b), where B is the attention weight, F c is the current feature, F s is the context information, W is the weight of the current feature, b is the bias of the current feature, and σ is the activation function.

[0062] The small target recognition model also introduces a small target aggregation detection layer, which is used to aggregate the features of multiple candidate small targets to improve the detection accuracy of small targets. The small target aggregation detection layer is expressed as: Among them, E agg is the aggregated feature, E m is the mth feature, is the weight of the mth feature, and M is the total number of features. The small target recognition model is efficient and accurate, and can accurately identify and classify birds in a complex marine environment, reducing false positives and false negatives.

[0063] In this embodiment, after the method creates the small target recognition model, the video information captured by the camera can be collected, and the angles of the camera and the fan can be calibrated according to the video information, for example, the angle facing due east is defined as 0 degrees. Then, the camera is used to collect the monitoring screen, the small target recognition model is used to identify the bird object in the monitoring screen, the real-time parameters of the camera, the pan / tilt head, and the fan are extracted, and the identified bird object is located according to the real-time parameters.

[0064] This real-time AI identification and positioning method for birds around offshore wind farms can efficiently and accurately identify and track birds in complex offshore environments, which is helpful for the safe operation of wind farms and the protection of bird ecological environment. It has broad application prospects and social value, which is specifically reflected in the following aspects:

[0065] 1) Effective real-time monitoring of birds around offshore wind farms, effective identification and tracking of bird flight paths, can assist relevant personnel in timely discovering and recording bird migration paths, accurately taking relevant intervention measures, and reducing the potential threat of wind farms to birds.

[0066] 2) Improve the ecological and environmental supervision mechanism for offshore wind power. Through accurate data recording and analysis, it can help environmental protection departments and wind farm managers formulate scientific protection measures, reduce the interference and harm of wind turbines to birds, promote the coordinated development of offshore wind power and marine environmental protection, and provide a basis for making scientific decisions to reduce the negative impact of the ecological environment. At the same time, it also provides more ideas for future offshore wind power planning and project environmental assessments.

[0067] 3) Improve wind farm operation efficiency. Wind farm operators can obtain bird activity information in a timely manner to avoid equipment damage and downtime due to bird collisions, thereby improving wind turbine operation efficiency and reliability. At the same time, it can assist wind farm managers in optimizing wind turbine layout and operation strategies, minimize the impact on bird migration, and achieve the goal of sustainable development.

[0068] Furthermore, in some embodiments, the real-time parameters include the focal length and field of view of the camera, the direction angle of the gimbal, and the rotation angle of the fan.

[0069] Furthermore, in some embodiments, positioning the identified bird object according to the real-time parameters specifically includes:

[0070] Determine the position coordinates of the bird object according to the recognition results of the bird object in the adjacent frame monitoring screen;

[0071] The flight speed, flight direction angle and actual position of each bird object are calculated based on real-time parameters.

[0072] In this embodiment, the method can determine the position coordinates P of the bird object based on the recognition results of the bird object in two adjacent frames of monitoring images. j (x j ,y j ), calculate the flight speed, flight direction angle and actual position of each bird object according to the real-time parameters of the camera, gimbal and fan, so that the position coordinates of the bird object in multiple consecutive frames of monitoring screen can be obtained, and more comprehensive and accurate monitoring data can be obtained.

[0073] Further, in some embodiments, the flight speed v is expressed as:

[0074]

[0075] Among them, P j is the position coordinate of the bird object in the jth frame of the monitoring screen, and Δt is the time interval.

[0076] In this embodiment, P j+1 -P j It represents the displacement of the bird object in two adjacent frames of monitoring images, and is used to calculate the flight time of the bird object.

[0077] Further, in some embodiments, the flight direction angle θ is expressed as:

[0078]

[0079] Among them, x j is the horizontal coordinate of the position of the bird object in the jth frame of the monitoring screen, y j It is the ordinate of the position coordinate of the bird object in the jth frame monitoring screen.

[0080] In this embodiment, y j+1 -y j The ordinate component representing the displacement of the bird object in two adjacent frames of monitoring images, x j+1 -x j The horizontal coordinate component representing the displacement of the bird object in two adjacent frames of monitoring images is used to calculate the flight direction angle of the bird object.

[0081] Further, in some embodiments, the actual position R is expressed as:

[0082] R=(R x ,R y ,d);

[0083] in, f is the focal length of the camera, α is the field of view of the camera, β is the rotation angle of the fan, γ is the azimuth angle of the gimbal; d is the horizontal distance from the camera to the bird, which is determined according to the camera's pitch angle ε and installation height h: d = h·cot(ε).

[0084] In this embodiment, the method calculates the actual position of the bird object in the actual space according to the focal length and field angle of the camera, the rotation angle of the fan, and the direction angle of the pan / tilt.

[0085] Furthermore, in some embodiments, after the flight direction angle θ is calculated, the method further includes:

[0086] The flight direction angle θ is compensated according to the rotation angle of the fan and the direction angle of the gimbal to obtain the actual flight direction angle θ actual :

[0087] θ actual =θ+β+γ.

[0088] In this embodiment, taking into account that the camera and the fan define due east as 0 degrees during calibration, and taking into account the characteristic of the fan rotating with the wind, the flight direction angle of the bird is compensated in combination with the rotation angle of the fan and the direction angle of the gimbal to ensure the accuracy of recognition.

[0089] A real-time AI identification and positioning system for birds around offshore wind farms, applicable to offshore wind farms including cameras and wind turbines, where the cameras are installed via a pan / tilt platform; see Figure 2 , the system includes:

[0090] Model creation unit: used for creating a small target recognition model for identifying bird objects;

[0091] Calibration unit: used to calibrate the angles of cameras and wind turbines in offshore wind farms;

[0092] Collection unit: used to collect monitoring images using cameras;

[0093] Recognition unit: used to identify bird objects in the monitoring screen using a small target recognition model;

[0094] Positioning unit: used to extract the real-time parameters of the camera, gimbal, and fan, and locate the identified bird objects based on the real-time parameters.

[0095] In this embodiment, the real-time AI identification and positioning system for birds around offshore wind farms adopts a modular design, and each functional module is independent of each other, which is easy to maintain and upgrade. In this way, the system can also be expanded according to actual needs, such as adding more environmental parameter monitoring and analysis modules. The system also supports integration with other intelligent systems, such as combining meteorological data and other sensor data to further enhance the intelligence level of the system.

[0096] The system provided in the embodiment of the present invention is briefly described. For matters not mentioned in the embodiment part, reference may be made to the corresponding contents in the aforementioned embodiment.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A real-time AI identification and positioning method for birds around an offshore wind farm, characterized in that: Applicable to an offshore wind farm comprising a camera and a wind turbine, wherein the camera is installed via a pan / tilt platform; the method comprises: Create a small object recognition model for identifying bird objects; Calibrate the angles of the camera and the wind turbine in the offshore wind farm; Using the camera to collect monitoring images; Using the small target recognition model to identify bird objects in the monitoring picture; Extract real-time parameters of camera, gimbal and fan; The identified bird object is positioned according to the real-time parameters.

2. According to claim 1, the real-time AI identification and positioning method for birds around an offshore wind farm is characterized in that: The small target recognition model adopts an improved neural network structure, and the improved neural network structure is improved by adding a fine-grained feature pyramid in the feature pyramid network.

3. According to claim 1, the real-time AI identification and positioning method for birds around an offshore wind farm is characterized in that: The loss function in the small target recognition model is: Among them, L is the true value of the i-th target, L i is the predicted value of the i-th target, N is the total number of targets, and w i is the weight of the ith target, A small is the preset small target area threshold, A i is the area of ​​the ith target, and k is the preset adjustment coefficient.

4. According to claim 1, the method for real-time AI identification and positioning of birds around an offshore wind farm is characterized in that: The real-time parameters include the focal length and field of view of the camera, the direction angle of the pan / tilt, and the rotation angle of the fan.

5. According to claim 4, the real-time AI identification and positioning method for birds around offshore wind farms is characterized in that: Positioning the identified bird object according to the real-time parameters specifically includes: Determine the position coordinates of the bird object according to the recognition results of the bird object in the adjacent frame monitoring screen; The flight speed, flight direction angle and actual position of each bird object are calculated according to the real-time parameters.

6. The real-time AI identification and positioning method for birds around an offshore wind farm according to claim 5 is characterized in that: The flight speed v is expressed as: Among them, P j is the position coordinate of the bird object in the jth frame of the monitoring screen, and Δt is the time interval.

7. The real-time AI identification and positioning method for birds around an offshore wind farm according to claim 5 is characterized in that: The flight direction angle θ is expressed as: Among them, x j is the horizontal coordinate of the position of the bird object in the jth frame of the monitoring screen, y j It is the ordinate of the position coordinate of the bird object in the jth frame monitoring screen.

8. The real-time AI identification and positioning method for birds around an offshore wind farm according to claim 7 is characterized in that: The actual position R is expressed as: R=(R x ,R y ,d); in, f is the focal length of the camera, α is the field of view of the camera, β is the rotation angle of the fan, γ is the azimuth angle of the gimbal; d is the horizontal distance from the camera to the bird, which is determined according to the camera's pitch angle ε and installation height h: d = h·cot(ε).

9. The real-time AI identification and positioning method for birds around an offshore wind farm according to claim 7 is characterized in that: After calculating the flight direction angle θ, it also includes: The flight direction angle θ is compensated according to the rotation angle of the fan and the direction angle of the gimbal to obtain the actual flight direction angle θ actual : i actual =θ+β+γ.

10. A real-time AI identification and positioning system for birds around offshore wind farms, characterized in that: Applicable to an offshore wind farm including a camera and a wind turbine, wherein the camera is installed via a pan / tilt platform; the system comprises: Model creation unit: used for creating a small target recognition model for identifying bird objects; Calibration unit: used to calibrate the angles of the camera and the wind turbine in the offshore wind farm; Collection unit: used to collect monitoring images using the camera; Recognition unit: used to recognize bird objects in the monitoring picture using the small target recognition model; Positioning unit: used to extract the real-time parameters of the camera, pan / tilt head and fan, and locate the identified bird object according to the real-time parameters.

Images