Monitoring device and method for moving object in wind field and storage medium

Abstract

The invention relates to the technical field of remote moving object detection, and discloses a device and method for monitoring a moving object in a wind field and a storage medium. The device for monitoring the moving object in the wind field comprises a tower used for erecting a first bearing part and a second bearing part at a position higher than the ground by a preset distance, and a first camera used for identifying the moving object in the wind field; the second bearing part rotates relative to the first bearing part, the second bearing part rotates based on the identified moving object, and the moving object is located in the measurement range of the second camera and the range finder; the second camera is used for capturing the moving object and controlling the second bearing part to rotate in real time to realize synchronization of the monitoring area and the position of the moving object; the range finder is used for determining the distance of the moving object. Accurate monitoring of the position of the moving object is realized, so that start-stop control can be performed on the fan at a proper time, the shutdown frequency and time of the fan are reduced, and the generating capacity of the fan is improved. And meanwhile, a monitoring blind area caused by shielding is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of long-distance moving object detection, and in particular to a monitoring device, method and storage medium for moving objects in a wind farm. Background Art

[0002] Wind turbines convert wind energy into mechanical energy and then into electrical energy, providing renewable energy to the power grid. The area where wind turbines are located is called a wind farm. Because wind farms are typically located in open, rural areas, they are susceptible to interference from moving objects, potentially affecting the safe operation of the wind turbines. If these moving objects are living organisms, this could also cause harm to them.

[0003] To mitigate these safety issues, wind turbines are typically equipped with devices to monitor moving objects or repel critters. However, due to obstruction by the wind turbine, the monitoring devices are less effective in detecting moving objects in blind spots, resulting in less effective repelling devices. If, to prevent interference from moving objects, the wind turbine is shut down or the repelling device is activated during fixed periods of time when moving objects are likely to be present, this results in reduced wind turbine power generation and unnecessary waste of energy for the repelling device. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a device, method, and storage medium for monitoring moving objects in a wind farm. By installing the monitoring device in the wind farm rather than on the wind turbines, blind spots caused by wind turbine obstruction are avoided. A first camera with a larger field of view is used to monitor moving objects over a wide area. Once the presence of a moving object in the wind farm is confirmed, a second camera with a smaller field of view and a rangefinder are activated to determine the image and distance of the moving object. This allows for more accurate monitoring of the moving object, precise control of wind turbine startup and shutdown, and reduced power generation losses.

[0005] To solve the above technical problems, an embodiment of the present invention provides a monitoring device for moving objects in a wind farm, comprising: a first camera, a second camera, a rangefinder, a first supporting part for supporting the first camera, a second supporting part for supporting the second camera and the rangefinder, and a tower; wherein the first field of view of the first camera is larger than the second field of view of the second camera; the tower is used to erect the first supporting part and the second supporting part at a position preset distance above the ground; the first camera is used to capture a first image and identify a moving object in the first image; the second supporting part rotates relative to the first supporting part, the second supporting part rotates based on the moving object in the first image, and by rotating, the moving object is placed within the measuring range of the second camera and the rangefinder; the second camera is used to capture a second image of the moving object, the second supporting part rotates in real time based on the second image, and the rotation of the second supporting part is synchronized with the movement of the moving object; the rangefinder is used to determine the distance between the moving object and the monitoring device.

[0006] An embodiment of the present invention also provides a method for monitoring moving objects in a wind field, which is applied to the above-mentioned monitoring device for moving objects in a wind field, including: when the first camera recognizes a moving object, determining a first azimuth angle based on the pixel coordinates of the moving object in the first image recognized by the first camera, and controlling the second bearing part to rotate based on the first azimuth angle until the moving object is within the second field of view of the second camera; using the second camera to capture a second image of the moving object, the second bearing part rotates in real time based on the second image of the moving object, and the rotation of the second bearing part is synchronized with the movement of the moving object; using the rangefinder to continuously detect the distance between the moving object and the monitoring device; determining the coordinate information of the moving object in the wind field based on the position information of the rangefinder and the distance information of the moving object; and judging whether there is a risk of collision between the moving object and a wind turbine in the wind field based on the coordinate information of the moving object in the wind field.

[0007] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which implements the above-mentioned method for monitoring moving objects in a wind farm when executed by a processor.

[0008] Compared with the related art, the embodiment of the present invention sets up the first camera, the second camera and the rangefinder at a position above the ground at a preset distance through a tower, thereby avoiding the monitoring blind spot caused by the obstruction of vegetation and wind turbines on the ground. The first camera is used to identify the moving object in the wind farm, and the second supporting part carrying the second camera and the rangefinder is controlled to rotate relative to the first supporting part according to the identified moving object, and the second image of the moving object is captured by the second camera. The second supporting part rotates in real time based on the second image, and the angle of rotation is synchronized with the movement of the moving object, thereby realizing real-time tracking of the moving object, and then the real-time image of the moving object can be captured by the second camera, and the real-time distance of the moving object can be determined by the rangefinder. The GIS coordinate information of the moving object can be determined by combining the real-time image and real-time distance of the moving object with the position information of the monitoring device, thereby realizing precise control of the start and stop of the wind turbine and reducing power generation loss.

[0009] In addition, there are multiple first cameras; the adjacent first cameras have partially overlapping first fields of view.

[0010] In addition, the shooting angle of at least one of the first cameras is an elevation angle, and the moving object identified by at least one of the first cameras is a moving object in the airspace.

[0011] In addition, the second camera is a zoom lens, the optical axis of the rangefinder is aligned with the optical axis of the zoom lens, and the ranging angle of the rangefinder is dynamically adjusted according to the zoom parameter of the zoom lens.

[0012] In addition, the monitoring device for moving objects in a wind farm also includes: a first computing module connected to the first camera, and a second computing module connected to the second camera; the first computing module is used to determine a first azimuth angle based on the pixel coordinates of the moving object in the first image, and the first azimuth angle is used to control the rotation of the second bearing part; the second computing module is used to determine a second azimuth angle based on the pixel coordinates of the moving object in the second image, and the second azimuth angle is used to control the rotation of the second bearing part.

[0013] In addition, the monitoring device for mobile objects in a wind farm further includes: a server; the server is configured to receive the second azimuth angle sent by the second calculation module and the distance sent by the rangefinder.

[0014] In addition, the second computing module is also used to identify whether the moving object in the second image is a target object, and the server is used to receive the type of the target object sent by the second computing module; or, the server is used to determine whether the moving object is a target object based on the received second image.

[0015] In addition, when using the first camera to identify moving objects, it includes: using the first camera to collect monitoring images within a 360-degree range; denoising the monitoring images; and using a mobile target detection algorithm to identify moving objects in the denoised monitoring images.

[0016] In addition, a non-monitoring area within the first field of view of the first camera is determined, wherein the non-monitoring area is a predetermined area with a fixed object; when identifying a moving object in the first image, the remaining areas except the non-monitoring area are identified.

[0017] In addition, when the first camera recognizes a moving object, it also includes: identifying whether the moving object is a target object based on the second image captured by the second camera; if it is not a target object, shielding the non-target object in the first image captured by the first camera within a preset time.

[0018] In addition, when it is determined that there is a risk of collision between the moving object and a wind turbine in the wind farm, it includes: when the distance between the moving object and the wind turbine is less than a first preset threshold and greater than a second preset threshold, controlling the wind turbine to operate at a low speed; the first preset threshold is greater than the second preset threshold; when the distance between the moving object and the wind turbine is less than or equal to the second preset threshold, controlling the wind turbine to stop.

[0019] In addition, when it is determined that there is a risk of collision between the mobile object and a wind turbine in the wind farm, it also includes: when the distance between the mobile object and the wind turbine is less than a third preset threshold and greater than a second preset threshold, the mobile object is expelled by a deterrence module; the third preset threshold is less than the first preset threshold, and the third preset threshold is greater than the second preset threshold.

[0020] In addition, when there are multiple first cameras, controlling the rotation of the second bearing unit based on the first azimuth angle includes: determining the first azimuth angle obtained by each first camera; deduplicating the first azimuth angles obtained by all first cameras based on the time when each first camera obtains the first azimuth angle and the size of the first azimuth angle; and controlling the rotation of the second bearing unit based on the first azimuth angle after deduplication.

[0021] In addition, when the first camera recognizes that there are multiple moving objects, it also includes: determining the priority of the multiple identified moving objects based on the size of the moving object, the moving speed of the moving object and the appearance time of the moving object; the controlling the rotation of the second bearing part based on the first azimuth angle includes: based on the priority of the multiple moving objects, controlling the second bearing part to rotate according to the first azimuth angle of the moving object with a higher priority.

[0022] In addition, the type of the mobile object is determined based on the second image captured by the second camera; the method of determining the type of the mobile object includes: identifying image features through a deep learning algorithm, and determining the type of the mobile object based on the image features; or, when the mobile object is a flying object, determining the type of the mobile object based on the overall size or wingspan size of the flying object in the second image. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] One or more embodiments are exemplarily described by the pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Unless otherwise stated, the pictures in the drawings do not constitute proportional limitations.

[0024] Figure 1 is a schematic structural diagram of a monitoring device for moving objects in a wind farm according to an embodiment of the present invention;

[0025] Figure 2 is a schematic structural diagram of a monitoring device for moving objects in a wind farm according to another embodiment of the present invention;

[0026] Figure 3 is a flow chart of a method for monitoring a moving object in a wind farm according to an embodiment of the present invention;

[0027] Figure 4 4 is an overall flow chart of a method for monitoring moving objects in a wind farm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will appreciate that many technical details are provided in the embodiments of the present invention to help readers better understand the present application. However, even without these technical details and the various variations and modifications based on the following embodiments, the technical solutions claimed in the present application can still be implemented.

[0029] The following embodiments are divided for the convenience of description and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined with each other and referenced to each other under the premise of no contradiction.

[0030] The embodiment of the present invention relates to a monitoring device for moving objects in a wind farm, such as Figure 1 As shown, it includes: a first camera 1, a second camera 4, a rangefinder 5, a first supporting part 2 for supporting the first camera 1, a second supporting part 3 for supporting the second camera 4 and the rangefinder 5, and a tower 6; wherein the first field of view of the first camera 1 is larger than the second field of view of the second camera 4, and the tower 6 is used to set the first supporting part 2 and the second supporting part 3 at a position above the ground by a preset distance; the first camera 1 is used to capture a first image and identify a moving object in the first image; the second supporting part 3 rotates relative to the first supporting part 2, and the second supporting part 3 rotates based on the moving object in the first image, and through the rotation, the moving object is placed within the measurement range of the second camera 4 and the rangefinder 5; the second camera 4 is used to capture a second image of the moving object, and the second supporting part 3 rotates in real time based on the second image, and the rotation of the second supporting part 3 is synchronized with the movement of the moving object; the rangefinder 5 is used to determine the distance between the moving object and the monitoring device.

[0031] First camera 1 can be a wide-field-of-view camera. Multiple first cameras 1 are arranged around second camera 4, that is, spaced apart around the second camera 4 in a 360-degree direction. Adjacent first cameras 1 have partially overlapping first fields of view. At least one first camera 1 has an elevation angle, and at least one first camera 1 identifies moving objects in the airspace. This combination of multiple first cameras enables 360-degree horizontal monitoring of the upper hemisphere's airspace without blind spots.

[0032] The first camera 1 can be located at the edge of the first supporting portion 2 to expand the monitoring range of all first cameras. The first supporting portion 2 can be a circular platform, and the second supporting portion 3 is located in the center of the first supporting portion 2. The second supporting portion 3 includes a movable base and a motor. The motor controls the movable base to rotate in three degrees of freedom: pitch, rotation, and tilt, thereby enabling the second supporting portion 3 to rotate relative to the first supporting portion 2.

[0033] The resolution of the second camera 4 can be higher than that of the first camera 1. Specifically, the second camera 4 can be a high-resolution zoom camera. By varying the zoom parameters, it can capture high-definition images of moving objects, facilitating identification of the moving object's type based on the high-definition images. The rangefinder 5 can be a laser rangefinder. Compared to optical rangefinders, using a laser rangefinder simplifies calibration, reduces maintenance complexity and costs, and improves distance measurement accuracy. The rangefinder's optical axis is aligned with that of the zoom lens, and its ranging angle dynamically adjusts with the zoom lens's zoom parameters to ensure that the ranging range matches the zoom lens's field of view, preventing the ranging range from being too large or too small, thereby improving ranging efficiency.

[0034] The first camera, second camera and rangefinder mentioned above can all be connected to independent computing units to quickly process the information collected by each of them to improve the efficiency of information processing. Specifically, the monitoring device for moving objects in a wind farm may include: a first computing module connected to the first camera, and a second computing module connected to the second camera; the first computing module is used to determine a first azimuth angle based on the pixel coordinates of the moving object in the first image, and the first azimuth angle is used to control the rotation of the second supporting part; the second computing module is used to determine a second azimuth angle based on the pixel coordinates of the moving object in the second image, and the second azimuth angle is used to control the rotation of the second supporting part. Among them, the first azimuth angle is used to control the rotation of the second supporting part to bring the moving object within the monitoring range of the second camera. After the second camera can capture the moving object, the rotation of the second supporting part will be adjusted according to the second azimuth angle to achieve real-time tracking of the moving object by the second camera.

[0035] In the embodiment of the present invention, Figure 2 As shown, the monitoring device for mobile objects in a wind farm also includes: a server 7; the server is used to receive the second azimuth angle sent by the second calculation module and the distance sent by the rangefinder. Server 7 stores and backs up the information monitored by the monitoring device, and can also determine whether there is a collision risk between the mobile object and a wind turbine in the wind farm based on the second azimuth angle and distance of the monitored mobile object. If there is a collision risk, a shutdown command is sent to the corresponding wind turbine. Alternatively, the server can also send the second azimuth angle and distance of the monitored mobile object to the wind farm's control device, and the control device can determine whether there is a collision risk and control the start and stop of the wind turbine.

[0036] In addition to the second azimuth angle and distance of the moving object, the information received by the server can also include the type of moving object, the image of the moving object, etc. Obtaining multi-dimensional information can enrich the formulation of the wind turbine start-up and shutdown strategy. Specifically, the second calculation module identifies whether the moving object in the second image is the target object. If it is the target object, the type of the target object is determined, and the target object in the second image is cropped; the server is used to receive the type of target object sent by the second calculation module, as well as the cropped second image. The target object here can be rare birds, protected animals, etc. By identifying this type of moving object, special organisms can be given priority protection. In addition, the server can also identify the type of moving object based on the received second image, thereby identifying whether the moving object in the second image is the target object.

[0037] In addition, the monitoring device for moving objects in the wind farm also includes: a deterrence module, which can be a laser emission unit, an ultrasonic emission unit or a high-decibel speaker, etc., which is used to expel the moving object through the deterrence module when it is detected that the moving object may collide with the wind turbine, thereby ensuring the safety of the moving object and avoiding damage to the wind turbine.

[0038] Compared with the related art, the embodiment of the present invention sets up the first camera, the second camera and the rangefinder at a position above the ground at a preset distance through a tower, thereby avoiding the monitoring blind spot caused by the obstruction of vegetation and wind turbines on the ground. The first camera is used to identify the moving object in the wind farm, and the second supporting part carrying the second camera and the rangefinder is controlled to rotate relative to the first supporting part according to the identified moving object, and the second image of the moving object is captured by the second camera. The second supporting part rotates in real time based on the second image, and the angle of rotation is synchronized with the movement of the moving object, thereby realizing real-time tracking of the moving object, and then the real-time image of the moving object can be captured by the second camera, and the real-time distance of the moving object can be determined by the rangefinder. The GIS coordinate information of the moving object can be determined by combining the real-time image and real-time distance of the moving object with the position information of the monitoring device, thereby realizing precise control of the start and stop of the wind turbine and reducing power generation loss.

[0039] A low-resolution first camera is used to perform preliminary screening of moving objects. After the presence of moving objects in the wind field is confirmed, a high-resolution second camera and rangefinder are turned on to determine the image and distance of the moving objects. This improves the accuracy of the calculation of the moving object's coordinate information while reducing the power consumption of the high-resolution camera.

[0040] Since wind farms occupy a large area, it is difficult for a single monitoring device for moving objects in a wind farm to monitor the entire wind farm. Therefore, multiple monitoring devices for moving objects in a wind farm can be used to monitor different locations in the wind farm. The information monitored by each monitoring device can be aggregated and analyzed by the wind farm's control equipment to achieve control of the wind turbines in the wind farm. For example, a monitoring device for moving objects in a wind farm can be set up in the first and second areas of the wind farm. When a moving object passes through the first area, the monitoring device corresponding to the first area will track and monitor the moving object. When the moving object passes through the second area, the monitoring device corresponding to the second area will track and monitor the moving object, thereby avoiding monitoring blind spots in the wind farm.

[0041] The information related to the mobile objects that each monitoring device summarizes to the control device of the wind farm may include: the type of mobile object, the relative distance and azimuth between the mobile object and the monitoring device, and the moving speed of the mobile object, etc. After receiving the information related to the mobile object, the control device of the wind farm can determine whether the mobile object is a rare creature, whether it is an object to be protected, etc. based on the type of the mobile object. In case rare creatures and objects to be protected may collide with wind turbines, corresponding protective measures need to be implemented. In addition, the GIS coordinate information of the mobile object in the wind farm is calculated based on the relative distance and azimuth between the mobile object and the monitoring device, as well as the moving speed of the mobile object, and the moving trajectory of the mobile object is determined. Then, a signal to execute corresponding protective measures is sent to the wind turbines on the moving trajectory to control the shutdown of wind turbines with collision risks, and a start signal is sent to the wind turbine after the mobile object moves away from the stopped wind turbine to control the operation of the wind turbine and reduce the duration of the shutdown. The above-mentioned actions analyzed in the control equipment of the wind farm can also be calculated and implemented in the server of each monitoring device, that is, each monitoring device directly sends the conclusion of whether the type of the mobile object is a rare creature or whether it is an object that needs protection to the control device, and sends the GIS coordinate information of the mobile object in the wind farm to the control device, thereby reducing the calculation amount of the control device of the wind farm.

[0042] Embodiments of the present invention also relate to a method for monitoring moving objects in a wind farm, applicable to the aforementioned monitoring device for moving objects in a wind farm, comprising: upon a first camera identifying a moving object, determining a first azimuth angle based on the pixel coordinates of the moving object in a first image identified by the first camera, and controlling a second supporting member to rotate based on the first azimuth angle until the moving object is within a second field of view of the second camera; capturing a second image of the moving object using the second camera, and rotating the second supporting member in real time based on the second image of the moving object, the rotation of the second supporting member being synchronized with the movement of the moving object; continuously detecting the distance between the moving object and the monitoring device using a rangefinder; determining the coordinate information of the moving object in the wind farm based on the position information of the rangefinder and the distance information of the moving object; and determining whether there is a risk of collision between the moving object and a wind turbine in the wind farm based on the coordinate information of the moving object in the wind farm. Accurate monitoring of the moving object enables the wind turbine to be started and stopped at the appropriate time, thereby reducing the number and duration of wind turbine shutdowns and increasing wind turbine power generation. The following describes the implementation details of the method for monitoring moving objects in a wind farm according to this embodiment. The following content is provided for ease of understanding only and is not required for implementation of this solution.

[0043] The monitoring method of the moving objects in the wind farm in this embodiment is as follows: Figure 3 As shown, the method includes:

[0044] In step 301, when the first camera recognizes a moving object, a first azimuth angle is determined according to the pixel coordinates of the moving object in the first image recognized by the first camera, and the second supporting part is controlled to rotate based on the first azimuth angle until the moving object is within the second field of view of the second camera.

[0045] Specifically, the first camera captures a 360-degree surveillance image, performs denoising on the image, and uses a moving object detection algorithm to identify moving objects in the denoised image. In practice, denoising and moving object detection can be performed simultaneously. Specifically, by setting a minimum analysis area threshold (i.e., X*X pixels), objects smaller than X*X pixels are ignored, allowing small noise targets to be directly filtered out.

[0046] When the first camera identifies a moving object, the pixel coordinates of the moving object in the first camera's first field of view are recorded. A mapping relationship between the pixel coordinates in the first camera's first field of view and the azimuth angle of the second supporting unit is pre-established through calibration. Subsequently, based on the mapping relationship, the first azimuth angle can be determined according to the pixel coordinates of the moving object in the first image of the first camera, and the azimuth angle of the second supporting unit can be adjusted based on the first azimuth angle. This allows the second supporting unit to track the moving object in real time, ensuring that the second camera and rangefinder on the second supporting unit can capture the moving object. The second supporting unit changes its azimuth angle through motor control, and the azimuth angle can be controlled using a closed-loop control method to achieve the effect of quickly converging to the target azimuth angle.

[0047] Step 302: Use a second camera to capture a second image of the moving object. The second supporting part rotates in real time based on the second image of the moving object. The rotation of the second supporting part is synchronized with the movement of the moving object. Use a rangefinder to continuously detect the distance between the moving object and the monitoring device.

[0048] Specifically, the second camera is a high-resolution camera that can capture high-definition images of moving objects, and then the type of the moving object can be determined based on the high-definition images.

[0049] Methods for determining the type of a moving object include identifying image features using a deep learning algorithm and determining the type of the moving object based on the image features. Specifically, a deep learning model can be constructed and trained using a large number of images of moving objects. The trained deep learning model can then classify the images to be identified based on the image features, and then output whether a moving object exists in the identified second image and determine the type of the moving object.

[0050] Alternatively, when the moving object is a flying object, the type of the moving object can also be determined based on the overall size or wingspan of the flying object in the second image. Specifically, the key points of the moving object in the second image, such as the wingtips, beak, and tail of the flying object, are determined using a target detection algorithm (YOLO). The posture of the flying object in the second image is analyzed and converted into a planar image parallel to the wingspan of the flying object. Then, based on the positions of the key points determined above, the overall size or wingspan of the flying object is determined. The overall size or wingspan is then matched with the standard overall size or standard wingspan of different types of flying objects recorded in a database to determine the type of the flying object.

[0051] Since the position of the moving object changes in real time, when the moving object moves, the shooting range of the second camera needs to be adjusted accordingly. The moving object is always controlled to be in the center area of ​​the second field of view of the second camera. This can minimize the moving object from moving out of the shooting range. In addition, the rangefinder has a limited monitoring range for the moving object. The measuring range of the rangefinder can be kept consistent with the center area of ​​the second field of view of the second camera. That is, the distance of the moving object measured by the rangefinder is most accurate when the moving object is in the center area of ​​the second field of view. In actual operation, the measuring range of the rangefinder can also be matched with any area of ​​the second field of view of the second camera. During monitoring, it is only necessary to ensure that the moving object is within the corresponding measuring range of the rangefinder. The second azimuth angle is determined based on the pixel coordinates of the moving object in the second field of view of the second camera, and the rotation of the second supporting part is continuously controlled based on the second azimuth angle to achieve continuous tracking of the moving object.

[0052] Step 303: Determine the coordinate information of the moving object in the wind field according to the position information of the rangefinder and the distance information of the moving object.

[0053] Specifically, the rangefinder is a laser rangefinder, the optical axis of the rangefinder is aligned with the optical axis of the zoom lens, and the ranging angle of the rangefinder is dynamically adjusted with the zoom parameter of the zoom lens to ensure that the ranging range matches the field of view of the zoom lens, avoiding the ranging range being too large or too small, thereby improving the ranging efficiency.

[0054] Step 304 : determining whether there is a collision risk between the moving object and a wind turbine in the wind farm based on the coordinate information of the moving object in the wind farm.

[0055] Specifically, if the distance between the moving object and the fan is less than a first preset threshold and greater than a second preset threshold, the fan is controlled to operate at a low speed; if the first preset threshold is greater than the second preset threshold; if the distance between the moving object and the fan is less than a third preset threshold and greater than the second preset threshold, the moving object is expelled through the deterrence module; if the third preset threshold is less than the first preset threshold and greater than the second preset threshold; if the distance between the moving object and the fan is less than or equal to the second preset threshold, the fan is controlled to stop. After the moving object moves away from the stopped fan, the fan is controlled to resume operation.

[0056] In order to further ensure the accuracy of the monitoring results, the non-monitoring area within the first field of view of the first camera can also be determined, wherein the non-monitoring area is a predetermined area with fixed objects, such as the area where vegetation and trees are located in the wind farm. Such fixed objects in the wind farm will not affect the operation of the wind turbine, but if the first image captured by the first camera is recognized, it will still be determined that there may be moving objects in the non-monitoring area, thereby occupying computing resources. In order to reduce the occupation of computing resources, when identifying the moving object in the first image, the remaining areas except the non-monitoring area can be identified. Specifically, the non-monitoring area in the first image can be masked, that is, the pixel values ​​in the non-monitoring area are the same (set to zero). When the first image is recognized using the moving target detection algorithm, the non-monitoring area will not be identified as a moving object, thereby playing an anti-interference role.

[0057] In addition, other methods can also be used to prevent interference. When the first camera identifies a moving object, the second image captured by the second camera is used to determine whether the moving object is a target object. If it is not a target object, the non-target object in the first image captured by the first camera is blocked for a preset time. The target object here is an object that is determined to affect the operation of the wind turbine, such as a rare organism, a protected object, etc. Similarly, the second image captured can be used to identify whether the moving object is a non-target object. Non-target objects can be set to clouds, the sun, airplanes, etc., which are objects that will not collide with the wind turbine. When an object that does not affect the operation of the wind turbine is identified, since the non-target object is within the monitoring range of the first camera for a certain period of time, in order to avoid repeated calculation of useless information and thus affecting the calculation results, the non-target object in the first image can be blocked. The blocking method can be to cover the non-target object in the first image by masking. The blocking of non-target objects in the first image is usually set with a blocking time to avoid missing moving objects that have newly entered the monitoring range due to long blocking. The specific method for identifying whether a moving object is a target object mentioned above can adopt the method for determining the type of a moving object described previously, that is, determining whether a moving object is a target object through a deep learning method or through the overall size or wingspan size of the flying object.

[0058] When there are multiple first cameras, there may be a situation where multiple first cameras all detect a moving object, but the second supporting unit can only monitor one azimuth angle. Therefore, it is necessary to consider the priority of the multiple detected moving objects, and the second supporting unit will prioritize the adjustment based on the azimuth angle of the high-priority moving object. Specifically, the priority of the multiple identified moving objects can be determined based on the size of the moving object, the moving speed of the moving object, and the appearance time of the moving object. Moving objects that are larger, move faster, and appear earlier have higher priority.

[0059] In addition, because the first fields of view of multiple first cameras overlap, two first cameras may recognize the same moving object. In this case, both first cameras will send the first azimuth angle to the second supporting unit. In this case, the second supporting unit needs to deduplicate the first azimuth angles received by all first cameras based on the size and transmission time of the first azimuth angles sent by different first cameras, and control the rotation of the second supporting unit based on the deduplicated first azimuth angles.

[0060] The above-mentioned deduplication processing and priority determination of the first azimuth angle can be implemented separately or in combination, such as performing priority determination on the first azimuth angle after deduplication processing, or deduplication processing on the first azimuth angle after priority sorting, and then controlling the rotation of the second bearing part according to the first azimuth angle. While ensuring priority identification of urgent moving objects, repeated identification of the same moving object is avoided, thereby improving the efficiency of identification.

[0061] The following is based on Figure 4 The method for monitoring moving objects in a wind farm of the present invention is generally described as follows:

[0062] The monitoring device for moving objects in a single wind farm includes several image acquisition devices (image acquisition device 1, image acquisition device 2, image acquisition device N). The image acquisition device can be a first camera or a low-power acquisition unit with image recognition function. Each image acquisition device is connected to a corresponding computing unit (image acquisition device 1 is connected to computing unit 1, image acquisition device 2 is connected to computing unit 2, and image acquisition device N is connected to computing unit N). The computing unit performs moving object recognition on the images captured by the connected image acquisition devices. When the computing unit corresponding to any image acquisition device recognizes a moving object, the azimuth angle (first azimuth angle) of the moving object relative to the pan-tilt platform (second supporting part) is determined based on the moving object, and the pan-tilt platform is controlled to rotate to the position and orientation of the moving object (flying object) based on the first azimuth angle. When a single image acquisition device identifies a moving object, the pan-tilt head rotates based on the first azimuth angle of the moving object identified by the image acquisition device. When multiple image acquisition devices simultaneously identify a moving object, the multiple image acquisition devices send the first azimuth angles of the identified moving object to the pan-tilt head, which performs deduplication processing based on the size of the first azimuth angle and the time of identification, and prioritizes the multiple first azimuth angles after deduplication, and tracks them in order of urgency. If a single image acquisition device identifies multiple moving objects, the image acquisition device can prioritize the multiple identified moving objects, and then send the priorities and first azimuth angles of the moving objects to the pan-tilt head, which tracks them in order based on the priorities of the moving objects. The first image captured by the image acquisition device may not be a replica of the real scene, and may selectively block images in the real scene that are useless for the monitoring results, such as vegetation and trees in a wind farm, and clouds and the sun passing through the sky.

[0063] When the second support unit rotates, the second camera and laser rangefinder on the pan-tilt head begin operating. The second camera captures a second image and continuously controls the rotation of the second support unit based on the second azimuth angle of the moving object in the second image to track the moving object. Simultaneously, the second camera magnifies the captured second image to identify the type of the moving object. If the type of the moving object matches the monitored type, indicating that the moving object is likely to affect the start and shutdown of the wind turbine, the second camera continues monitoring the moving object until it leaves the monitoring range of the monitoring device. If the moving object does not affect the start and shutdown of the wind turbine, tracking of the moving object is abandoned. The second support unit rotates based on another first azimuth angle transmitted by the first camera, bringing another moving object into the monitoring range of the second camera, and the monitoring process for the second camera repeats. The laser rangefinder on the pan-tilt head measures the distance of the moving object relative to the rangefinder. The computing unit of the second camera on the pan-tilt head uses an image AI model to determine the type of the moving object (flying object). Finally, the calculated distance, type, and azimuth angle of the moving object are transmitted to the server, which analyzes the wind turbine based on the relevant information about the moving object and implements anti-collision protection measures.

[0064] The first camera may be in working state for a long time. In order to further save power, the first camera may be put into sleep mode during a time period when it is determined by experience that there are few or even no moving objects.

[0065] In addition, when the movement trajectory of small, non-rare protected objects overlaps with the position of the fan, normal operation or low-speed operation of the fan can be considered. For example, when a small insect is identified approaching the fan, its impact on the fan is almost negligible. In order to avoid frequent starting and stopping of the fan, the fan can be kept in operation.

[0066] The steps of the various methods above are divided only for the purpose of clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are within the scope of protection of the present invention. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of the invention.

[0067] The method in this embodiment is implemented based on the above-mentioned device and system embodiments. Therefore, the features and implementation effects mentioned in the above-mentioned device and system embodiments are also applicable to this embodiment.

[0068] An embodiment of the present invention also relates to a monitoring device for moving objects in a wind farm, comprising: a control module, for determining a first azimuth angle based on the pixel coordinates of the moving object in a first image recognized by the first camera when a first camera recognizes the moving object, and controlling the rotation of a second supporting part based on the first azimuth angle until the moving object is within a second field of view of the second camera; an acquisition module, for acquiring a second image of the moving object using the second camera, the second supporting part rotates in real time based on the second image of the moving object, and the rotation of the second supporting part is synchronized with the movement of the moving object; a distance module, for continuously detecting the distance between the moving object and the monitoring device using a rangefinder, and determining the coordinate information of the moving object in the wind farm based on the position information of the rangefinder and the distance of the moving object; and a judgment module, for judging whether there is a risk of collision between the moving object and a wind turbine in the wind farm based on the coordinate information of the moving object in the wind farm.

[0069] In addition, the denoising module is used to use the first camera to collect monitoring images within a 360-degree range; denoise the monitoring images; and the control module is used to use a moving target detection algorithm to identify moving objects in the denoised monitoring images.

[0070] In addition, the control module is used to determine a non-monitoring area within the first field of view of the first camera, wherein the non-monitoring area is a predetermined area with a fixed object; when identifying a moving object in the first image, the remaining areas outside the non-monitoring area are identified.

[0071] In addition, the control module is used to identify whether the moving object is a target object based on the second image captured by the second camera. If it is not a target object, the non-target object in the first image captured by the first camera is shielded within a preset time.

[0072] In addition, the judgment module is used to control the fan to run at a low speed when the distance between the moving object and the fan is less than a first preset threshold and greater than a second preset threshold; the first preset threshold is greater than the second preset threshold; and when the distance between the moving object and the fan is less than or equal to the second preset threshold, control the fan to stop.

[0073] In addition, the judgment module is used to expel the moving object through the deterrence module when the distance between the moving object and the wind turbine is less than a third preset threshold and greater than a second preset threshold; the third preset threshold is less than the first preset threshold, and the third preset threshold is greater than the second preset threshold.

[0074] In addition, the control module is used to determine the first azimuth angle obtained by each first camera when there are multiple first cameras; deduplicate the first azimuth angles obtained by all first cameras based on the time when each first camera obtains the first azimuth angle and the size of the first azimuth angle; and control the rotation of the second bearing part based on the first azimuth angle after deduplication.

[0075] In addition, the control module is used to, when the first camera identifies multiple moving objects, also include: determining the priority of the multiple identified moving objects based on the size of the moving object, the moving speed of the moving object and the appearance time of the moving object; controlling the rotation of the second bearing part based on the first azimuth angle, including: based on the priorities of the multiple moving objects, controlling the second bearing part to rotate according to the first azimuth angle of the moving object with a higher priority.

[0076] In addition, the acquisition module is also used to determine the type of the moving object based on the second image captured by the second camera; the method of determining the type of the moving object includes: identifying image features through a deep learning algorithm, and determining the type of the moving object based on the image features; or, when the mobile object is a flying object, determining the type of the moving object based on the overall size or wingspan size of the flying object in the second image.

[0077] It is worth noting that all modules involved in this embodiment are logical modules. In actual applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, to highlight the innovations of the present invention, this embodiment does not include units that are not closely related to solving the technical problems proposed by the present invention. However, this does not mean that other units do not exist in this embodiment.

[0078] This embodiment can be implemented in conjunction with the above-mentioned method embodiment. The relevant technical details mentioned in the method embodiment are still valid in this embodiment and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above-mentioned method embodiment.

[0079] An embodiment of the present invention relates to an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above-mentioned method for monitoring moving objects in a wind farm.

[0080] The memory and processor are connected using a bus, which can include any number of interconnected buses and bridges. The bus connects various circuits of one or more processors and memories. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and are therefore not described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to the processor.

[0081] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0082] An embodiment of the present invention relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0083] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0084] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present invention, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present invention.

Claims

1. A monitoring device for moving objects in a wind farm, characterized in that: include: A first camera, a second camera, a rangefinder, a first carrying portion for carrying the first camera, a second carrying portion for carrying the second camera and the rangefinder, and a tower; wherein a first field of view of the first camera is larger than a second field of view of the second camera; The tower is used to erect the first bearing part and the second bearing part at a position above the ground by a preset distance; The first camera is used to capture a first image and identify a moving object in the first image; The second carrying portion rotates relative to the first carrying portion, the second carrying portion rotates based on a moving object in the first image, and the moving object is placed within a measurement range of the second camera and the rangefinder by the rotation; The second camera is used to capture a second image of the moving object, the second supporting member rotates in real time based on the second image, and the rotation of the second supporting member is synchronized with the movement of the moving object; The rangefinder is used to determine the distance between the moving object and the monitoring device.

2. The device for monitoring moving objects in a wind farm according to claim 1, characterized in that: There are multiple first cameras; The adjacently arranged first cameras have partially overlapping first fields of view.

3. The device for monitoring moving objects in a wind farm according to claim 2, characterized in that: The shooting angle of at least one of the first cameras is an elevation angle, and the moving object identified by at least one of the first cameras is a moving object in the airspace.

4. The device for monitoring moving objects in a wind farm according to claim 1, characterized in that: The second camera is a zoom lens, the optical axis of the rangefinder is aligned with the optical axis of the zoom lens, and the ranging angle of the rangefinder is dynamically adjusted according to the zoom parameter of the zoom lens.

5. The monitoring device for moving objects in a wind farm according to claim 1, characterized in that: Also includes: a first computing module connected to the first camera, and a second computing module connected to the second camera; The first calculation module is used to determine a first azimuth angle according to the pixel coordinates of the moving object in the first image, and the first azimuth angle is used to control the rotation of the second bearing part; The second calculation module is used to determine a second azimuth angle according to the pixel coordinates of the moving object in the second image, and the second azimuth angle is used to control the rotation of the second bearing part.

6. The device for monitoring moving objects in a wind farm according to claim 5, characterized in that: Also includes: server; The server is used to receive the second azimuth angle sent by the second calculation module and the distance sent by the rangefinder.

7. The device for monitoring moving objects in a wind farm according to claim 6, characterized in that: The second calculation module is further configured to identify whether the moving object in the second image is a target object, and the server is configured to receive the type of the target object sent by the second calculation module; Alternatively, the server is configured to determine whether the moving object is a target object based on the received second image.

8. A method for monitoring moving objects in a wind farm, characterized in that: The device for monitoring a moving object in a wind farm according to any one of claims 1 to 7, wherein the method comprises: When the first camera recognizes a moving object, determining a first azimuth angle according to pixel coordinates of the moving object in a first image recognized by the first camera, and controlling the second supporting portion to rotate based on the first azimuth angle until the moving object is within a second field of view of the second camera; capturing a second image of the moving object using the second camera, wherein the second supporting member rotates in real time based on the second image of the moving object, and the rotation of the second supporting member is synchronized with the movement of the moving object; Continuously detecting the distance between the moving object and the monitoring device using the rangefinder; determining coordinate information of the mobile object in the wind field according to the position information of the rangefinder and the distance of the mobile object; It is determined whether there is a collision risk between the mobile object and a wind turbine in the wind farm according to the coordinate information of the mobile object in the wind farm.

9. The method for monitoring moving objects in a wind farm according to claim 8, characterized in that: Also includes: Determine a non-monitoring area within a first field of view of the first camera, wherein the non-monitoring area is a predetermined area having a fixed object; When identifying the moving object in the first image, the remaining areas except the non-monitoring area are identified.

10. The method for monitoring moving objects in a wind farm according to claim 8, characterized in that: In the case where the first camera recognizes a moving object, the method further includes: Identify whether the moving object is a target object based on the second image captured by the second camera; if it is not a target object, shield the non-target object in the first image captured by the first camera within a preset time.

11. The method for monitoring moving objects in a wind farm according to claim 8, characterized in that: When determining whether there is a collision risk between the mobile object and a wind turbine in the wind farm, the method includes: When the distance between the moving object and the fan is less than a first preset threshold and greater than a second preset threshold, controlling the fan to operate at a low speed; the first preset threshold is greater than the second preset threshold; When the distance between the moving object and the fan is less than or equal to the second preset threshold, the fan is controlled to stop.

12. The method for monitoring moving objects in a wind farm according to claim 8, characterized in that: When there are multiple first cameras, controlling the rotation of the second carrying portion based on the first azimuth angle includes: Determine the first orientation angle obtained by each first camera; Deduplication processing is performed on the first azimuth angles obtained by all the first cameras according to the time when each first camera obtains the first azimuth angle and the size of the first azimuth angle; The second bearing portion is controlled to rotate according to the first azimuth angle after deduplication processing.

13. The method for monitoring moving objects in a wind farm according to claim 8, characterized in that: In the case where the first camera recognizes that there are multiple moving objects, the method further includes: determining priorities of the identified multiple moving objects based on the size of the moving object, the moving speed of the moving object, and the appearance time of the moving object; The controlling the rotation of the second bearing portion based on the first azimuth angle includes: Based on the priorities of the plurality of moving objects, the second carrying portion is controlled to rotate according to the first azimuth angle of the moving object with a high priority.

14. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for monitoring a moving object in a wind farm according to any one of claims 8 to 13 is implemented.

Images