Fire point location methods and systems
By integrating digital elevation models and gimbal camera data into photovoltaic power plants, and utilizing terrain constraints and multi-view weighted fusion technology, the problem of low fire point positioning accuracy in photovoltaic power plants has been solved, achieving sub-meter level fire point positioning accuracy and supporting precise fire suppression operations by drones.
Patent Information
- Application Number
- CN202610968955.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-30
- Publication Date
- 2026-11-10
AI Technical Summary
Existing photovoltaic power plant fire detection systems cannot accurately locate fire points, forcing maintenance personnel to conduct visual searches over large areas, delaying firefighting opportunities. Furthermore, the low accuracy of monocular positioning cannot meet the requirements for automatic firefighting or precise handling by drones.
By integrating digital elevation models, real-time operating parameters of PTZ cameras, and visual images, and utilizing terrain constraints, the location of fire points can be determined in complex terrain scenarios of photovoltaic power plants. Weighted fusion of observation data from multiple PTZ cameras can improve positioning accuracy.
It achieved sub-meter-level precise positioning of fire points in complex terrain of photovoltaic power plants, providing accurate coordinate references for subsequent precise positioning and emergency fire response, and improving positioning accuracy and robustness.
Smart Images

Figure CN123072138A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of photovoltaic power generation technology, and in particular to a fire point location method and system. Background Technology
[0002] Photovoltaic power plants typically occupy large areas with complex terrain, and are often built in remote areas such as mountains and deserts. To ensure the safety of these power plants, pan-tilt-zoom (PTZ) cameras are commonly used for smoke and fire detection and identification. However, existing technologies have the following main drawbacks: First, it can only detect but not locate. Existing smoke and fire detection algorithms can only output an alarm signal indicating "whether there is smoke or fire," but cannot provide the specific latitude and longitude coordinates of the fire point in three-dimensional space. This forces maintenance personnel to still conduct visual searches over a large area, delaying the opportunity to extinguish the fire. Secondly, the monocular positioning accuracy is low. Some systems attempt to use the angle and magnification parameters of a single gimbal for geometric calculations, but due to gimbal installation attitude errors, terrain undulations, and the lack of monocular visual depth information, the positioning error is often tens or even hundreds of meters, which cannot meet the requirements of drones for automatic firefighting or precise handling.
[0003] Therefore, there is an urgent need for a method to accurately locate fire points in complex terrain scenarios of photovoltaic power plants. Summary of the Invention
[0004] The purpose of this application is to propose a fire point location method and system to solve the problem that existing technologies can only detect smoke and fire but cannot accurately locate the fire point.
[0005] In a first aspect, embodiments of this application propose a fire point location method, comprising: responding to a fire point captured by a first gimbal camera deployed in a photovoltaic power station, performing fire point location based on a first image, first operating parameters, and a digital elevation model of the photovoltaic power station to obtain a first location; and using the first location as the fire point location result; wherein the first image is an image of the fire point captured by the first gimbal camera, and the first operating parameters are the operating parameters of the first gimbal camera when capturing the first image.
[0006] According to the fire location method of the present invention, by integrating digital elevation model, real-time operating parameters of gimbal camera and visual image, the location of fire point can be determined in complex terrain scenarios of photovoltaic power station by utilizing terrain constraints, providing coordinate reference for subsequent accurate positioning or fire emergency response.
[0007] In addition, the fire point location method according to the above embodiments of the present invention may further include the following additional technical features: According to an embodiment of the present invention, before using the first location as the fire point location result, the method further includes: determining whether there are multiple second gimbal cameras, the fields of view of the multiple second gimbal cameras covering the first location, and capturing the same fire point; in response to the existence of the multiple second gimbal cameras, for each second gimbal camera, performing fire point location based on a second image, second operating parameters, and the digital elevation model to obtain a second location; weighting the multiple second locations to obtain a third location; and using the third location as the fire point location result; wherein, the second image is an image of the fire point captured by the second gimbal camera, and the second operating parameters are the operating parameters of the second gimbal camera when capturing the second image.
[0008] According to an embodiment of the present invention, determining whether there are multiple second gimbal cameras includes: adding the first gimbal camera to a candidate camera set, and traversing all gimbal cameras deployed in the photovoltaic power station other than the first gimbal camera; for the i-th gimbal camera traversed, determining whether the first position is within the field of view coverage of the i-th gimbal camera based on the relative position of the first position and the i-th gimbal camera; in response to the first position being within the field of view coverage of the i-th gimbal camera, determining whether the i-th gimbal camera has captured a fire point within a target time window, wherein the target time window is determined based on the time when the first gimbal camera captures a fire point; in response to the i-th gimbal camera capturing a fire point within the target time window, adding the i-th gimbal camera to the candidate camera set; after traversal, recording the gimbal cameras in the candidate camera set as second gimbal cameras, and determining whether the number of second gimbal cameras is greater than 1.
[0009] According to an embodiment of the present invention, before determining whether the number of the second gimbal cameras is greater than 1, the method further includes: filtering the second gimbal cameras in the candidate camera set; wherein, the time difference between any two second gimbal cameras in the filtered candidate camera set capturing the fire point is less than or equal to a preset time threshold, and the distance difference between the fire point positions obtained based on the positioning of the two second gimbal cameras is less than or equal to a preset distance threshold.
[0010] According to one embodiment of the present invention, the step of weighting multiple second positions to obtain a third position includes: obtaining the optical center position and viewing direction of each second gimbal camera when capturing the second image; determining the weight corresponding to each second gimbal camera based on the angle between the viewing direction of each second gimbal camera and the normal direction of the surface where the fire point is located, wherein the weight is negatively correlated with the angle; and weighting and fusing multiple second positions based on the optical center position, viewing direction and corresponding weight of each second gimbal camera to obtain the third position.
[0011] According to one embodiment of the present invention, the method further includes: selecting the gimbal camera with the largest weight from each of the second gimbal cameras as the target gimbal camera; determining the pixel width of the photovoltaic module where the fire point is located based on the second image captured by the target gimbal camera; calculating a first distance between the target gimbal camera and the photovoltaic module according to the pixel width of the photovoltaic module, the physical width of the photovoltaic module, and the focal length of the target gimbal camera; taking a point at the first distance along the line of sight of the target gimbal camera and fusing it with the third position; projecting the fused position onto the layout diagram of the photovoltaic power station to obtain the photovoltaic array area number and photovoltaic module number where the fire point is located, as the fire point location result.
[0012] According to one embodiment of the present invention, the method further includes: in response to the absence of the plurality of second gimbal cameras and the first gimbal camera being stationary, calculating a second distance between the first gimbal camera and the photovoltaic module based on the pixel width of the photovoltaic module where the fire point is located in the first image, the physical width of the photovoltaic module, and the focal length of the first gimbal camera; taking a point at the second distance along the line of sight of the first gimbal camera and intersecting it with the digital elevation model to obtain a fourth position; in response to the fourth position matching the position of the component in the photovoltaic power station layout diagram, using the fourth position as the fire point location result; or, in response to the absence of the plurality of second gimbal cameras and the first gimbal camera being in motion, performing the step of using the first position as the fire point location result and issuing a verification prompt message.
[0013] According to one embodiment of the present invention, the method further includes: in response to a mismatch between the fourth position and the component position in the photovoltaic power plant layout diagram, estimating a third distance between the first gimbal camera and the fire point based on the pixel height and / or pixel area of the flame region in the first image; taking a point at the third distance along the line of sight of the first gimbal camera and intersecting it with the digital elevation model to obtain a fifth position; weighting the fifth position with the first position to obtain a sixth position; and using the sixth position as the fire point location result.
[0014] According to one embodiment of the present invention, the step of locating a fire point based on a first image, first operating parameters, and a digital elevation model of the photovoltaic power station to obtain a first position includes: obtaining the first pixel coordinates of the fire point in the first image; using a pre-calibrated rotation matrix of the first gimbal camera, and according to the first operating parameters, converting the first pixel coordinates into a line-of-sight vector in a geodetic coordinate system; constructing a ray originating from the optical center of the first gimbal camera based on the line-of-sight vector; and solving for the intersection of the ray with the digital elevation model to obtain the first position.
[0015] According to one embodiment of the present invention, the step of solving for the intersection of the ray and the digital elevation model to obtain the first position includes: taking the endpoint of the ray as the starting point, advancing along the ray according to a preset step size, calculating the difference between the elevation of the current point and the ground elevation of the corresponding position in the digital elevation model; in response to the difference changing from positive to negative, interpolating within a search interval, and taking the position where the difference is zero as the first position, wherein the search interval is the range of forward step size crossed by the difference changing from positive to negative.
[0016] According to one embodiment of the present invention, the calibration process of the rotation matrix of the gimbal camera includes: measuring the installation position of the gimbal camera; selecting at least three ground feature points with known geodetic coordinates within the photovoltaic power station, wherein the ground feature points are not collinear; controlling the gimbal camera to sequentially align the image center with each of the ground feature points, and recording the operating parameters of the gimbal camera corresponding to each of the ground feature points; and solving the rotation matrix from the gimbal camera coordinate system to the geodetic coordinate system based on the geodetic coordinates of each of the ground feature points and their corresponding operating parameters.
[0017] According to one embodiment of the present invention, the method further includes: sending the fire location result to a drone to control the drone to verify the fire point through an onboard camera, and / or controlling the drone to perform fire extinguishing operations.
[0018] Secondly, this application proposes a fire point location system, comprising: a plurality of PTZ cameras deployed in a photovoltaic power station, the PTZ cameras being used to capture images of the photovoltaic power station; and a terminal device for executing the method described in the above embodiments of the present invention, the terminal device being communicatively connected to the plurality of PTZ cameras.
[0019] According to the fire location system of the present invention, multiple pan-tilt cameras deployed in a photovoltaic power station acquire images in real time, and the terminal device executes the above-mentioned fire location method when any pan-tilt camera captures a fire point. By fusing digital elevation models, real-time operating parameters of pan-tilt cameras and visual images, the system can determine the location of the fire point in complex terrain scenarios of photovoltaic power stations by utilizing terrain constraints, and provide coordinate reference for subsequent accurate positioning or emergency fire response.
[0020] In addition, the fire point location method according to the above embodiments of the present invention may further include the following additional technical features: According to one embodiment of the present invention, it further includes: a drone, communicatively connected to the terminal device, for responding to receiving a fire location result sent by the terminal device, performing at least one of the following operations: verifying the fire location at the location corresponding to the fire location result using an airborne camera; and performing a fire extinguishing operation.
[0021] The advantages of this application in terms of its additional aspects will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0022] Figure 1 This is a flowchart illustrating a fire point location method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a fire point location method according to another embodiment of the present invention; Figure 3 This is a flowchart illustrating a fire point location method according to another embodiment of the present invention; Figure 4 This is a flowchart illustrating a fire point location method according to another embodiment of the present invention; Figure 5 This is a flowchart illustrating a fire point location method according to another embodiment of the present invention; Figure 6 This is a flowchart illustrating a fire point location method according to another embodiment of the present invention; Figure 7 This is a flowchart illustrating a fire point location method according to another embodiment of the present invention; Figure 8 This is a flowchart illustrating a fire point location method according to another embodiment of the present invention; Figure 9 This is a flowchart illustrating a fire point location method according to another embodiment of the present invention; Figure 10 This is a flowchart illustrating a fire point location method according to another embodiment of the present invention; Figure 11 This is a structural block diagram of the controller according to an embodiment of the present invention; Detailed Implementation The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0023] Photovoltaic power plants typically occupy large areas with complex terrain, and are often built in remote areas such as mountains and deserts. If the fire point is not located in time, it will lead to delays in firefighting, causing significant economic losses and safety hazards. Existing smoke and fire detection algorithms can only output alarm signals and cannot provide the specific latitude and longitude coordinates of the fire point; while geometric calculation methods based on the angle and magnification parameters of a single pan-tilt unit often have positioning errors of tens or even hundreds of meters due to pan-tilt unit installation attitude errors, terrain undulations, and lack of monocular visual depth information, which are difficult to meet the actual operation and maintenance needs.
[0024] To this end, this application proposes a fire point location method and system. By integrating digital elevation models, real-time operating parameters of gimbal cameras, and visual images, it can utilize terrain constraints to determine the location of fire points in complex terrain scenarios of photovoltaic power plants, providing coordinate references for subsequent precise location or emergency fire response.
[0025] The fire point location method and system of this application are described below with reference to the accompanying drawings.
[0026] Figure 1 This is a flowchart illustrating a fire point location method according to an embodiment of the present invention.
[0027] Specifically, in some embodiments of the present invention, such as Figure 1 As shown, the fire point location method includes: S101, in response to the first pan-tilt camera deployed in the photovoltaic power station capturing a fire point, the fire point is located based on the first image, the first operating parameters, and the digital elevation model of the photovoltaic power station to obtain the first position.
[0028] Specifically, in this embodiment, the first gimbal camera is any dual-light gimbal camera deployed in the photovoltaic power station, equipped with visible light image acquisition and infrared thermal imaging image acquisition functions, enabling all-weather inspection of the photovoltaic power station. When the smoke and fire detection algorithm built into the first gimbal camera (such as a deep learning-based target detection network, such as YOLO, Faster R-CNN, etc.) identifies a suspected smoke and fire target in its acquired images, the fire point localization process is triggered.
[0029] At this time, the system acquires the first image (i.e., the image frame that triggered the alarm) and the first operating parameters. The first operating parameters include, but are not limited to, the horizontal angle of the gimbal camera when capturing the first image. Pitch angle Variable multiples and alarm timestamp A digital elevation model (DEM) is a digital topographic model of a photovoltaic power station area, derived from UAV oblique photography or lidar mapping data.
[0030] Based on the above data, the system performs fire point location calculations to obtain the first location. This first location is the three-dimensional coordinates of the fire point in the geodetic coordinate system, including longitude, latitude, and altitude information.
[0031] S102, the first position is taken as the fire point location result; wherein, the first image is the image of the fire point captured by the first gimbal camera, and the first operating parameters are the operating parameters when the first gimbal camera captures the first image.
[0032] Specifically, in this embodiment, after obtaining the first location, the system outputs the first location as the preliminary positioning result of the current fire point. Since the first location is obtained based on the intersection of a single PTZ camera and the digital elevation model, its positioning accuracy is limited by the DEM grid resolution, PTZ angle measurement error, and calibration error, typically falling within the error range of 10 to 50 meters. This first location can serve as a benchmark for subsequent multi-view fusion positioning, and also as a backup positioning result under conditions where no other observations are available. Simultaneously, the system associates and stores the first location with information such as the alarm timestamp and the triggered PTZ number for subsequent spatiotemporal consistency verification and positioning accuracy evaluation.
[0033] Furthermore, in some embodiments of the present invention, such as Figure 2 As shown, before using the first position as the fire point location result, the method also includes: S201, determine whether there are multiple second gimbal cameras, the field of view of multiple second gimbal cameras covers the first position, and they capture the same fire point.
[0034] Specifically, in this embodiment, after obtaining the first location but before outputting it as the final fire point location result, the system actively determines whether there are multiple second gimbal cameras capable of forming multi-view collaborative observation with the first gimbal camera. The core of this determination process lies in two aspects: firstly, the field of view (FOV) of the second gimbal camera must cover the spatial area where the first location is located; secondly, the second gimbal camera must observe the same fire point synchronously or substantially synchronously with the first gimbal camera in time. Gimbal cameras that meet these two conditions will be identified as candidate second gimbal cameras for subsequent multi-source data fusion positioning. If two or more second gimbal cameras exist, it indicates that the current fire point is in a multi-view shared-view area, meeting the conditions for performing high-precision fusion positioning.
[0035] S202, in response to the existence of multiple second gimbal cameras, for each second gimbal camera, fire point localization is performed based on the second image, the second operating parameters and the digital elevation model to obtain the second position.
[0036] Specifically, in this embodiment, after determining that multiple second gimbal cameras exist, the system independently executes the same fire point localization process as step S101 for each second gimbal camera in the candidate set. For the first... The system acquires a second image and second operating parameters of the fire point captured by a second pan-tilt camera. These parameters are then combined with the digital elevation model (DEM) of the photovoltaic power station to perform ray-based DEM intersection, yielding the fire point location independently observed by that pan-tilt camera, denoted as the second location. Since different pan-tilt cameras observe the same fire point from different angles, the resulting second locations will differ due to variations in observation angle and measurement errors. These differences will be comprehensively considered in the subsequent weighted fusion process.
[0037] S203, weighted multiple second positions to obtain the third position.
[0038] Specifically, in this embodiment, after obtaining multiple second positions, the system performs weighted fusion on these second positions to obtain a more accurate third position. The core idea of weighted fusion is as follows: a gimbal camera observing directly towards the fire point has a smaller angle between its line of sight and the normal direction of the fire point surface, resulting in less perspective distortion in the fire point area of the image and higher positioning reliability, and should be given a larger weight; while a gimbal camera observing tangentially has a larger angle between its line of sight and the normal direction of the fire point surface, resulting in greater perspective distortion in the fire point area of the image and lower positioning reliability, and should be given a smaller weight. By constructing and solving a weighted least squares objective function, the spatial point that minimizes the sum of the squares of the weighted distances of all observed rays is obtained, which is the third position. Compared with the first position obtained by positioning with a single gimbal camera, the accuracy of the third position can be improved to sub-meter level (≤0.5 meters).
[0039] S204, take the third position as the fire point location result; wherein, the second image is the image of the fire point captured by the second gimbal camera, and the second operating parameters are the operating parameters when the second gimbal camera captures the second image.
[0040] Specifically, in this embodiment, the third position obtained by weighted fusion is used as the final fire point localization result under multi-view observation conditions. Compared with the first position, the third position effectively fuses the observation information of multiple gimbal cameras, and uses multi-view geometric constraints to eliminate the systematic bias caused by the lack of monocular depth information, significantly improving both positioning accuracy and robustness. This third position will be used as the target coordinates subsequently output to the UAV or operation and maintenance platform.
[0041] Furthermore, in some embodiments of the present invention, such as Figure 3 As shown, determine whether there are multiple second gimbal cameras, including: S301 adds the first gimbal camera to the candidate camera set and iterates through all gimbal cameras deployed in the photovoltaic power station except for the first gimbal camera.
[0042] Specifically, in this embodiment, the process of determining whether multiple second PTZ cameras exist begins with the construction of a candidate camera set. First, the first PTZ camera that triggered the alarm is added to the candidate camera set as a baseline. Then, all PTZ cameras deployed in the photovoltaic power station, excluding the first PTZ camera, are traversed. For the i-th PTZ camera encountered, the system calculates whether its field of view coverage includes the first location based on its known installation location (optical center geodetic coordinates), field of view parameters (horizontal and vertical field of view), and the current PTZ pointing angle. This calculation comprehensively considers factors such as the PTZ's maximum rotation range, effective detection distance, and terrain obstruction to filter out PTZ cameras that spatially meet the conditions for observing the first location.
[0043] S302, for the i-th gimbal camera that has been traversed, determine whether the first position is within the field of view coverage of the i-th gimbal camera based on the relative position of the first position and the i-th gimbal camera.
[0044] Specifically, in this embodiment, for the i-th pan-tilt camera traversed, the system first determines the position based on the first position. The optical center position of the gimbal camera Calculate from point to The direction vector is then transformed into the gimbal camera coordinate system to obtain the corresponding desired horizontal angle. and desired pitch angle The system determines whether both angles are within the current mechanical rotation range of the gimbal camera (i.e., the horizontal and pitch limits of the gimbal), and and The distance between them is less than the effective detection range of the gimbal camera. If all the above conditions are met, the first position is determined to be within the field of view coverage of the gimbal camera; otherwise, it is considered that the gimbal camera cannot observe the area and it is not added to the candidate camera set.
[0045] S303, in response to the first position being within the field of view of the i-th gimbal camera, determine whether the i-th gimbal camera has captured the fire point within the target time window, wherein the target time window is determined based on the time when the first gimbal camera captures the fire point.
[0046] Specifically, in this embodiment, for the i-th gimbal camera that has spatial observation capabilities, the system further performs verification in the time dimension. Based on the alarm timestamp t of the first gimbal camera, the system determines the target time window as [t-Δt, t+Δt], where Δt can be preset according to the system synchronization accuracy and the fire spread rate.
[0047] The system queries whether the i-th PTZ camera also has a smoke and fire detection alarm record within the time window. If it does, it means that the i-th PTZ camera observed the fire point synchronously with the first PTZ camera in time; if it does not, it means that although the PTZ camera's field of view covers the first position, it did not detect the fire point at the corresponding time, possibly due to reasons such as the fire point being blocked, the flame not being visible from that angle, or the detection algorithm missing the detection. In this case, it is not added to the candidate camera set.
[0048] S304, in response to the i-th gimbal camera capturing a fire point within the target time window, adds the i-th gimbal camera to the candidate camera set.
[0049] Specifically, in this embodiment, for the i-th PTZ camera that simultaneously meets the spatial field-of-view coverage condition and the time synchronization alarm condition, the system adds it to the candidate camera set. The PTZ cameras in this set are considered to possess the basic conditions for multi-view collaborative positioning with the first PTZ camera. The remaining PTZ cameras are then traversed until all PTZ cameras have been processed.
[0050] S305: After traversal, the gimbal camera in the candidate camera set is recorded as the second gimbal camera, and it is determined whether the number of the second gimbal camera is greater than 1.
[0051] Specifically, in this embodiment, after traversal, the system records all gimbal cameras in the candidate camera set as second gimbal cameras and counts their number M. If M > 1, it is determined that there are multiple second gimbal cameras, and the system enters the multi-source data fusion precise positioning process; if M ≤ 1, it is determined that there are no multiple second gimbal cameras, the system cannot perform multi-view fusion positioning, and instead enters single-view positioning mode.
[0052] Furthermore, in some embodiments of the present invention, before determining whether the number of second gimbal cameras is greater than 1, the method further includes: filtering the second gimbal cameras in the candidate camera set; wherein, the time difference between any two second gimbal cameras in the filtered candidate camera set capturing the fire point is less than or equal to a preset time threshold, and the distance difference between the fire point positions obtained based on the positioning of any two second gimbal cameras is less than or equal to a preset distance threshold.
[0053] Specifically, in this embodiment, before determining whether the number of second gimbal cameras is greater than 1, the system also performs consistency screening on the second gimbal cameras in the candidate camera set to eliminate abnormal observations that may be triggered by different fire sources or false alarms. The consistency screening includes two dimensions: temporal consistency and spatial consistency.
[0054] Regarding time consistency, the system checks the difference between the alarm timestamps of any two second PTZ cameras in the candidate set. If the difference is greater than a preset time threshold (e.g., 3 seconds), the two observations are determined to be inconsistent in time, and the observation with the larger deviation is removed.
[0055] Regarding spatial consistency, the system calculates the spatial distance between any two second positions based on the independent positioning of each second gimbal camera. If this distance is greater than a preset distance threshold (e.g., 50 meters), it is determined that the two observations may correspond to different fire points, and the observation with the larger deviation is discarded.
[0056] The above-mentioned dual consistency screening ensures that all observations participating in the fusion correspond to the same physical fire source, thus avoiding positioning errors caused by mismatches.
[0057] Furthermore, in some embodiments of the present invention, such as Figure 4 As shown, the third position is obtained by weighting multiple second positions, including: S401, acquire the optical center position and line of sight of each second gimbal camera when capturing the second image.
[0058] Specifically, in this embodiment, before performing weighted fusion, the system first obtains the optical center position and line-of-sight direction of each second gimbal camera when capturing the second image. Optical center position The three-dimensional coordinates of the second gimbal camera in the geodetic coordinate system are obtained based on the high-precision measurement and calibration in step S901. Line of sight direction. The unit direction vector pointing from the optical center to the fire point is calculated as follows: obtain the pixel coordinates of the fire point in the second image, and combine the intrinsic parameters (including focal length, principal point coordinates, distortion coefficients) and extrinsic parameters (rotation matrix) of the gimbal camera to convert the pixel coordinates into a three-dimensional direction vector in the geodetic coordinate system.
[0059] S402, determine the weight of each second gimbal camera based on the angle between the line of sight of each second gimbal camera and the normal direction of the surface where the fire point is located, where the weight is negatively correlated with the angle.
[0060] Specifically, in this embodiment, the system determines the weight of each second gimbal camera based on the angle between the line of sight of each second gimbal camera and the normal direction of the surface at the location of the fire point. The normal direction of the surface at the location of the fire point can be approximated as the zenith direction perpendicular to the ground, or it can be precisely set according to the actual orientation of the photovoltaic module at the first location. Angle The smaller the value, the more directly the gimbal camera is observing the fire point, resulting in less perspective distortion in the fire area of the image and higher positioning reliability; therefore, it is assigned a larger weight. For example, weight... ,when The weight is 1 when =0°, when The weight is 0 when the angle is 90°; other monotonically decreasing functions (such as Gaussian functions) can also be used to assign weights. Finally, the weights are normalized so that the sum of the weights of all second gimbal cameras is 1.
[0061] S403, based on the optical center position, line of sight direction and corresponding weight of each second gimbal camera, performs weighted fusion of multiple second positions to obtain the third position.
[0062] Specifically, in this embodiment, based on the optical center position of each second gimbal camera... Direction of sight and the corresponding weights The system performs weighted fusion on multiple second positions, for example, to solve... To obtain the third position ,in, This indicates the total number of gimbal cameras participating in the fusion. Indicates the first The weight of each gimbal camera, Indicates the first The three-dimensional position of the optical center of the gimbal camera in the geodetic coordinate system. Indicates the first The line-of-sight vector from the optical center of each gimbal camera to the fire point is represented by a unit vector. This fusion method fully utilizes multi-view observation information, effectively suppresses the influence of single-view measurement noise, and significantly improves positioning accuracy.
[0063] Furthermore, in some embodiments of the present invention, such as Figure 5 As shown, the fire point location method also includes: S501 selects the gimbal camera with the highest weight from each of the second gimbal cameras as the target gimbal camera.
[0064] Specifically, in this embodiment, the system selects the gimbal camera with the highest weight from among the second gimbal cameras as the target gimbal camera. This gimbal camera has the best viewing angle among all the common-view cameras, and its image shows the least perspective distortion of the fire point and surrounding photovoltaic modules, with the clearest edges, making it most suitable for depth verification based on the photovoltaic module scale.
[0065] S502, based on the second image captured by the target gimbal camera, determines the pixel width of the photovoltaic module where the fire point is located.
[0066] Specifically, in this embodiment, the system determines the pixel width of the photovoltaic module where the fire point is located based on the second image captured by the target gimbal camera. First, the Canny edge detection operator is used to extract image edges in the visible light image, and then Hough transform is applied to detect straight line segments. Since the photovoltaic modules are arranged regularly, their edge lines are usually parallel and evenly spaced. The system determines the boundary line of the module where the fire point is located based on its pixel position, and then calculates the pixel width of the module in the image. To improve robustness, the detected edge lines of multiple components can be statistically averaged to reduce the impact of errors in detecting a single edge.
[0067] S503 calculates the first distance between the target gimbal camera and the photovoltaic module based on the pixel width of the photovoltaic module, the physical width of the photovoltaic module, and the focal length of the target gimbal camera.
[0068] Specifically, in this embodiment, based on the pixel width of the photovoltaic module Physical width of photovoltaic modules (Known constants, derived from photovoltaic module specifications) and the focal length of the target gimbal camera. (Based on internal parameter calibration), the first distance between the target gimbal camera and the photovoltaic module is calculated using the following formula:
[0069] This distance estimation is based on a pinhole imaging model, and it has high accuracy when the component plane is approximately parallel to the imaging plane. The first distance here is used as an estimate of the object distance from the optical center of the target gimbal camera to the component plane where the fire point is located.
[0070] S504: Take the point at the first distance along the line of sight of the target gimbal camera and fuse it with the third position.
[0071] Specifically, in this embodiment, a spatial point at a first distance D is taken along the line of sight of the target gimbal camera. ,in The optical center position of the target gimbal camera. Its view direction vector. The depth point is recovered based on component scale. The third position obtained based on multi-view weighted triangulation Perform the fusion. The fusion method could be a weighted average of the two, for example, the final coordinates of the fused data. Or, different weights can be assigned based on their respective confidence levels, for example... ,in It can be dynamically determined based on factors such as the observation angle of the target gimbal camera and the quality of edge detection.
[0072] S505 projects the merged location onto the layout map of the photovoltaic power station to obtain the photovoltaic array area number and photovoltaic module number where the fire point is located, which serves as the fire point location result.
[0073] Specifically, in this embodiment, the fused positions The image is projected onto the CAD layout of the photovoltaic power plant. This CAD layout pre-stores the precise location, size, orientation, and corresponding array area and module number for each photovoltaic module. The system then uses spatial coordinate matching to project the image onto the CAD layout. Mapping to the nearest component location determines the photovoltaic array zone number and photovoltaic module number where the fire point is located. This step transforms abstract latitude and longitude coordinates into an intuitive format of "zone, row, and module number," facilitating rapid location by maintenance personnel and precise approach by drones.
[0074] Furthermore, in some embodiments of the present invention, such as Figure 6 As shown, the fire point location method also includes: S601, in response to the absence of multiple second gimbal cameras and the first gimbal camera being stationary, calculate the second distance between the first gimbal camera and the photovoltaic module based on the pixel width of the photovoltaic module where the fire point is located in the first image, the physical width of the photovoltaic module, and the focal length of the first gimbal camera.
[0075] Specifically, in this embodiment, when the system determines that there are no multiple second gimbal cameras (i.e., multi-view fusion positioning cannot be performed) and the first gimbal camera is currently stationary, the system attempts to perform single-view positioning using the photovoltaic module size constraints.
[0076] The phrase "the first gimbal camera is stationary" means that, within a predetermined time period before and after capturing the first image, the changes in both the horizontal and vertical angles of the gimbal are less than preset angle thresholds. This indicates that the gimbal has not rotated, and the conditions for depth recovery using component-scale data from a single frame image are met.
[0077] Based on the pixel width of the photovoltaic module where the fire point is located in the first image Physical width of photovoltaic modules (Given constants) and the focal length of the first gimbal camera (Based on internal parameter calibration), the system calculates the second distance between the first gimbal camera and the photovoltaic module. This distance serves as an estimate of the object distance from the optical center of the first gimbal camera to the plane of the component where the fire point is located.
[0078] S602, take the point at the second distance along the line of sight of the first gimbal camera, and intersect it with the digital elevation model to obtain the fourth position.
[0079] Specifically, in this embodiment, a second distance is taken along the line of sight of the first gimbal camera. spatial point ,in The optical center position of the first gimbal camera. Let it be the vector of its line of sight. (This refers to the spatial point.) Intersection verification with the digital elevation model, i.e., verification of spatial points Does the elevation at this point match the ground elevation of the DEM? If not, then... Fine-tune along the line of sight to the DEM surface to obtain the fourth position. Then determine the fourth position. Check if it matches the component position in the CAD layout drawing. If it matches, accept the positioning result. The positioning accuracy is between 2 and 5 meters.
[0080] S603, in response to the fourth position matching the component position in the photovoltaic power plant layout diagram, uses the fourth position as the fire point location result.
[0081] Specifically, in this embodiment, the system determines the fourth position. Whether it matches the component positions in the CAD layout drawing of the photovoltaic power plant, i.e., check Does it fall within the geometric range of a particular photovoltaic module? If a match is successful, it indicates that the depth information recovered based on the module scale is consistent with the prior CAD information, indicating high positioning reliability. The system will then determine the fourth location. This outputs the final fire location result. The positioning accuracy in this mode can reach 2 to 5 meters.
[0082] It should be noted that, in some embodiments of the present invention, in response to the absence of multiple second gimbal cameras and the first gimbal camera being in motion, the step of taking the first position as the fire point location result is executed, and a verification prompt message is issued.
[0083] Specifically, in this embodiment, when the system determines that there are no multiple second gimbal cameras and the first gimbal camera is in motion (i.e., the changes in the horizontal and pitch angles of the gimbal exceed a preset angle threshold within a predetermined time period before and after capturing the first image), the system cannot use the parallax between consecutive frames for depth recovery, nor can it use the component scale for positioning (the image may have motion blur in motion). At this time, the system directly outputs the ray-DEM coarse positioning result (i.e., the first position) obtained in step S101 as the fire point positioning result, and at the same time issues a verification prompt message, reminding the maintenance personnel that the positioning result has low accuracy (usually 10 to 50 meters), and suggesting manual verification or dispatching a drone to confirm.
[0084] Furthermore, in some embodiments of the present invention, such as Figure 7 As shown, the fire point location method also includes: S701, in response to the mismatch between the fourth position and the component position in the photovoltaic power plant layout diagram, estimates the third distance between the first gimbal camera and the fire point based on the pixel height and / or pixel area of the flame region in the first image.
[0085] Specifically, in this embodiment, when the fourth position When the component positions do not match the layout plan of the photovoltaic power plant (e.g.) If the component falls outside the gaps or range of the components, it indicates that the depth recovery based on the component scale may fail due to reasons such as image blurring, component occlusion, or edge recognition errors. The system is then downgraded to the second level of localization—flame feature empirical model localization.
[0086] The system first segments the flame region in the thermal image of the first image. Segmentation methods can include thresholding, region growing, or a semantic segmentation network based on deep learning. Then, the pixel height of the flame region is calculated. and / or pixel area Using an empirical model fitted in advance through calibration experiments within the power plant, the third distance between the first pan-tilt camera and the fire point is estimated. or ,in The fitting coefficients were obtained by calibrating a standard fire source at different distances. This third distance, used as an estimate of the object distance from the optical center of the first gimbal camera to the fire point, was then used for subsequent spatial point construction and weighted fusion.
[0087] S702, take the point at the third distance along the line of sight of the first gimbal camera, and intersect it with the digital elevation model to obtain the fifth position.
[0088] Specifically, in this embodiment, a third distance is taken along the line of sight of the first gimbal camera. Spatial nodes Then, it is intersected with the digital elevation model to verify it. Does the elevation at this point match the ground elevation of the DEM? If not, then... Fine-tune along the line of sight to the DEM surface to obtain the fifth position. The intersection method is the same as step S602.
[0089] S703, weight the fifth position with the first position to obtain the sixth position.
[0090] Specifically, in this embodiment, the fifth position is... With the first position (i.e., the coarse ray-DEM localization results obtained in step S101) are weighted and averaged to obtain the sixth position. The weight λ can be set according to factors such as flame segmentation quality and the confidence level of the empirical model, for example, λ=0.6. By fusing the location estimated based on flame scale with the terrain-constrained location based on DEM, the estimation error of a single orientation can be suppressed to a certain extent.
[0091] S704, the sixth position is used as the fire point location result.
[0092] Specifically, in this embodiment, the system will use the sixth position. The fire point location result is output and marked with a "low confidence" label. In this mode, the positioning accuracy is between 10 and 30 meters. While this is less accurate than component-scale constrained positioning and the aforementioned multi-view fusion positioning, it can still provide valuable reference locations for maintenance personnel under extreme conditions where only a single gimbal is available and component-scale information is unavailable, preventing situations where positioning is completely impossible. Maintenance personnel can manually verify and confirm the positioning result based on the "low confidence" label and the actual on-site conditions.
[0093] Furthermore, in some embodiments of the present invention, such as Figure 8 As shown, the fire point is located based on the first image, the first operating parameters, and the digital elevation model of the photovoltaic power station, resulting in the first location, including: S801, obtain the first pixel coordinates of the fire point in the first image.
[0094] Specifically, in this embodiment, firstly, the pixel coordinates (u, v) of the fire point are obtained from the first image. These pixel coordinates can be provided by the center coordinates of the detection frame output by the smoke and fire detection algorithm, or by the center coordinates of the bottom edge of the detection frame. For ground-based fire points, the center of the bottom edge of the detection frame is closer to the actual fire source location, resulting in higher accuracy; for airborne fire points (such as when the flame is in the initial rising stage of combustion) or when the fire point is detached from the ground, the center coordinates of the detection frame are more representative. The specific acquisition method can be flexibly selected according to the actual application scenario.
[0095] S802 uses the pre-calibrated rotation matrix of the first gimbal camera to convert the first pixel coordinates into a line-of-sight vector in the geodetic coordinate system according to the first operating parameters.
[0096] Specifically, in this embodiment, the rotation matrix R of the pre-calibrated first gimbal camera (this matrix is a 3×3 matrix used to transform vectors in the camera coordinate system to the geodetic coordinate system), combined with the gimbal horizontal angle and pitch angle in the first operating parameters, is used to convert the first pixel coordinates into a line-of-sight vector in the geodetic coordinate system. The specific conversion process is as follows: First, using the camera intrinsic parameters (including focal length) , and principal point coordinates ( , Convert pixel coordinates (u, v) to normalized coordinates in the camera coordinate system. :
[0097] After distortion correction Next, the direction vector in the camera coordinate system is constructed. , , because of this vector It doesn't necessarily have to be a unit vector; it needs to be normalized. Then, it's processed using the calibrated rotation matrix. Transform it to the geodetic coordinate system to obtain the three-dimensional line-of-sight direction vector. , used for subsequent ray construction. S803, based on the line-of-sight vector, constructs a ray originating from the optical center of the first gimbal camera.
[0098] Specifically, in this embodiment, based on the line-of-sight vector Constructing the optical core from the first gimbal camera The parametric equations of the starting ray:
[0099] Among them, This represents the distance traveled from the optical center along the line of sight; the goal is to find the minimum distance. This ray represents the set of all possible locations of the fire point in three-dimensional space. Since a single ray cannot determine the depth, terrain constraints from a digital elevation model (DEM) are needed to determine the spatial location of the fire point. In photovoltaic power plants, the fire point is usually located on the ground or on the photovoltaic modules; therefore, it can be assumed that the fire point is located on the ground surface. Coarse location coordinates are obtained by solving for the intersections of the ray and the DEM mesh. The goal is to find the minimum... This makes the elevation of Q(t) equal to the ground elevation of the DEM at that point.
[0100] S804, solve for the intersection of the ray and the digital elevation model to obtain the first position.
[0101] Specifically, in this embodiment, since the depth of a single ray cannot be determined, the spatial location of the fire point needs to be determined using terrain constraints. Considering that the fire point in a photovoltaic power station is usually located on the ground or on the photovoltaic modules, and the modules are close to the ground or have a certain tilt angle but their approximate height is known, it is assumed that the fire point is located on the ground surface. The coarse location coordinates can be obtained by solving for the intersection of the ray and the DEM grid. For the constructed ray parameter equation, the ray is solved... The first location is obtained by finding the intersection point with the DEM grid. For example, starting from the endpoint of the ray, the system moves forward along the ray at a preset step size, calculates the difference between the elevation of the current point and the ground elevation of the corresponding position in the digital elevation model, and performs interpolation within the search interval in response to the difference changing from positive to negative. The position where the difference is zero is taken as the first position, where the search interval is the range of forward step size that the difference changes from positive to negative. Furthermore, in some embodiments of the present invention, such as Figure 9 As shown, the intersection of the ray and the digital elevation model is solved to obtain the first position, including: S8041, starting from the endpoint of the ray, moves along the ray at a preset step size and calculates the difference between the elevation of the current point and the ground elevation of the corresponding position in the digital elevation model.
[0102] Specifically, in this embodiment, starting from the endpoint of the ray (d=0), the system advances step by step along the ray direction at a preset step size Δd (e.g., equivalent to the DEM grid resolution, preferably 1 meter). The step size is related to the grid resolution of the digital elevation model. At each step, the geodetic coordinates of the current point are calculated, and the ground elevation value of the corresponding location in the digital elevation model is queried. Then, the elevation of the current point is calculated. With DEM elevation The difference between the two values indicates that when the difference changes from positive to negative, it means that the ray passes from above the ground to below the ground within that step range, i.e., it passes through the terrain surface.
[0103] S8042, in response to the difference changing from positive to negative, performs interpolation within the search interval, and takes the position where the difference is zero as the first position, where the search interval is the range of forward steps crossed by the difference changing from positive to negative.
[0104] Specifically, in this embodiment, when the difference changes from positive to negative, it indicates that the ray has passed through the ground within the corresponding search interval. At this time, a binary interpolation is performed within this interval to find the precise location where the difference is zero, thus obtaining the precise intersection point. This is output as the first position. Compared to directly taking grid vertices, this iterative intersection method can obtain intersection positions with sub-grid accuracy, effectively improving the accuracy of coarse positioning. Due to the grid accuracy and ray direction errors of the digital elevation model, the positioning error of this result is usually between 10 meters and 50 meters. It can be used as a benchmark for subsequent multi-view fusion positioning, or as a backup positioning result under the condition of no other observations.
[0105] Furthermore, in some embodiments of the present invention, such as Figure 10 As shown, the calibration process for the rotation matrix of the gimbal camera includes: S901, measures the installation position of the gimbal camera.
[0106] Specifically, in this embodiment, the rotation matrix calibration of the gimbal camera is performed offline during system deployment. First, a high-precision real-time dynamic positioning (RTK) device is used to measure the three-dimensional coordinates of the gimbal camera mounting base. During measurement, the RTK rover is placed at the exact center of the gimbal base, and data from at least 30 epochs is collected and averaged to obtain the latitude, longitude, and altitude of that point. Then, the latitude and longitude are converted into planar coordinates using either the Gauss-Kruger projection or the Universal Transverse Mercator projection (UTM). The unit is meters. The actual elevation of the gimbal's optical center also needs to be added to the gimbal's own installation height. ,Right now Finally, the precise position of the gimbal's optical center in the geodetic coordinate system was obtained. .
[0107] S902, select at least 3 ground feature points with known geodetic coordinates within the photovoltaic power station, and the ground feature points shall not be collinear.
[0108] Specifically, in this embodiment, at least three ground feature points with known geodetic coordinates are selected within the photovoltaic power station. These feature points should be as dispersed and non-collinear as possible to ensure the numerical stability of the calibration solution. Feature points can be corners of the inverter building, fixed pile foundations, the center of permanent markers, etc., and their geodetic coordinates can be obtained through RTK measurement. Preferably, the number of feature points is four to six to provide redundant observations and improve calibration accuracy.
[0109] S903 controls the gimbal camera to sequentially align the image center with each local surface feature point, and records the operating parameters of the gimbal camera corresponding to each local surface feature point.
[0110] Specifically, in this embodiment, the gimbal camera is controlled to sequentially align the image center with each ground feature point. Whenever the image center is aligned with a feature point, the horizontal angle of the gimbal is recorded. and pitch angle And the pixel coordinates of the feature point (since it is aligned with the image center, the pixel coordinates are the principal point coordinates). For the first Each feature point, its geodetic coordinates The combination of the observation data and the corresponding gimbal angle is used to form an observation data pair.
[0111] S904, based on the geodetic coordinates of various ground feature points and their corresponding operating parameters, solves the rotation matrix from the gimbal camera coordinate system to the geodetic coordinate system.
[0112] Specifically, in this embodiment, based on the geodetic coordinates of all ground feature points and their corresponding gimbal operating parameters, the rotation matrix from the gimbal coordinate system to the geodetic coordinate system is solved using the PnP (Perspective-n-Point) algorithm. With translation vector Eliminate gimbal installation angle deviations and store them in the edge computing unit for later use in real-time positioning.
[0113] Furthermore, in some embodiments of the present invention, the fire location method further includes: sending the fire location result to a drone to control the drone to verify the fire point through an onboard camera, and / or controlling the drone to perform fire extinguishing operations.
[0114] Specifically, in this embodiment, after obtaining the final fire location result (including latitude, longitude, altitude, component number, etc.), the system encapsulates the result into a data packet of a preset format and pushes it to the UAV flight control platform via an API interface. Upon receiving the target coordinates, the UAV flight control platform plans a flight path, autonomously takes off, and flies to the location of the fire. The UAV verifies the target area using its onboard infrared camera and transmits real-time video back to the operations and maintenance center for confirmation by operations and maintenance personnel. If the UAV is equipped with fire extinguishing devices (such as dry powder fire extinguishers, fire extinguishing bombs, etc.), after confirming the fire, the flight control platform can further control the UAV to perform automatic fire extinguishing operations, achieving a complete closed loop from "detection-location-confirmation-response." This step significantly shortens the time from fire discovery to fire suppression, effectively reducing fire losses.
[0115] In summary, the fire location method according to the embodiments of the present invention, by integrating digital elevation models, real-time operating parameters of gimbal cameras and visual images, can utilize terrain constraints to determine the location of fire points in complex terrain scenarios of photovoltaic power plants, providing coordinate references for subsequent precise positioning or emergency fire response.
[0116] Based on the fire point location method proposed in the foregoing embodiments of the present invention, the present invention also proposes a terminal device, which includes a memory, a processor, and a computer program stored in the memory. When the computer program is executed by the processor, it implements the fire point location method as described in the foregoing embodiments of the present invention.
[0117] According to the terminal device of the present invention, by storing and executing the computer program of the above-described fire point location method, it can determine the location of the fire point in the complex terrain scenario of a photovoltaic power station by integrating digital elevation model, real-time operating parameters of gimbal camera and visual image, and utilizing terrain constraints, thus providing coordinate reference for subsequent accurate positioning or fire emergency response.
[0118] Based on the fire point location method proposed in the foregoing embodiments of the present invention, the present invention also proposes a fire point location system, which includes: multiple PTZ cameras deployed in a photovoltaic power station, the PTZ cameras being used to capture images of the photovoltaic power station; and a terminal device executing the method of the above embodiments of the present invention, the terminal device being communicatively connected to the multiple PTZ cameras.
[0119] Specifically, in response to any pan-tilt camera capturing a fire point, the terminal device locates the fire point based on the image from that pan-tilt camera, its operating parameters, and the digital elevation model of the photovoltaic power station. When multiple pan-tilt cameras simultaneously observe the same fire point, the terminal device also performs weighted fusion of the fire point locations obtained from the individual locations of the multiple pan-tilt cameras to obtain a more accurate positioning result.
[0120] Furthermore, in some embodiments of the present invention, the fire location system further includes: a drone, which is communicatively connected to a terminal device, and is used to perform at least one of the following operations in response to receiving a fire location result sent by the terminal device: one method is to verify the location corresponding to the fire location result by using an airborne camera to confirm the authenticity and current status of the fire, and to transmit the verification result back to the operation and maintenance center in real time; the other method is to perform a fire extinguishing operation after confirming the fire.
[0121] In summary, the fire location system according to the embodiments of the present invention acquires images in real time by deploying multiple PTZ cameras in a photovoltaic power station, and the terminal device executes the above-mentioned fire location method when any PTZ camera captures a fire point. Furthermore, by fusing digital elevation models, real-time operating parameters of PTZ cameras, and visual images, it can utilize terrain constraints to determine the location of the fire point in complex terrain scenarios of photovoltaic power stations, providing coordinate references for subsequent accurate positioning or emergency fire response.
[0122] Figure 11 This is a structural block diagram of the controller according to an embodiment of the present invention.
[0123] like Figure 11As shown, the controller 500 includes a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, for example, via a bus 502. Optionally, the controller 500 may also include a transceiver 504. It should be noted that in practical applications, the transceiver 504 is not limited to one type, and the structure of the controller 500 does not constitute a limitation on the embodiments of this application.
[0124] Processor 501 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 501 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.
[0125] Bus 502 may include a pathway for transmitting information between the aforementioned components. Bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 502 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 11 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0126] The memory 503 stores a computer program corresponding to the asynchronous serial communication method of the above embodiments of this application. This computer program is controlled and executed by the processor 501. The processor 501 executes the computer program stored in the memory 503 to implement the content shown in the aforementioned method embodiments.
[0127] The controller 500 includes, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 11The controller 500 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.
[0128] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.
[0129] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0130] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0131] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.
[0132] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0133] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.
[0134] In this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.
[0135] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.
Claims
1. A method for locating a fire point, characterized in that, include: In response to the first pan-tilt camera deployed in the photovoltaic power station capturing a fire point, the fire point is located based on the first image, the first operating parameters, and the digital elevation model of the photovoltaic power station to obtain the first position. The first location is taken as the fire point location result; Wherein, the first image is the image of the fire point captured by the first gimbal camera, and the first operating parameters are the operating parameters of the first gimbal camera when capturing the first image.
2. The fire point location method according to claim 1, characterized in that, Before using the first location as the fire point location result, the method further includes: Determine whether there are multiple second gimbal cameras, the field of view of which covers the first position and captures the same fire point; In response to the presence of the plurality of second gimbal cameras, for each second gimbal camera, fire point localization is performed based on the second image, the second operating parameters and the digital elevation model to obtain the second position; The third position is obtained by weighting multiple second positions; The third location is taken as the fire point location result; Wherein, the second image is the image of the fire point captured by the second gimbal camera, and the second operating parameter is the operating parameter when the second gimbal camera captures the second image.
3. The fire point location method according to claim 2, characterized in that, The determination of whether multiple second gimbal cameras exist includes: Add the first gimbal camera to the candidate camera set, and iterate through all gimbal cameras deployed in the photovoltaic power station except for the first gimbal camera. For the i-th gimbal camera that has been traversed, determine whether the first position is within the field of view coverage of the i-th gimbal camera based on the relative position of the first position and the i-th gimbal camera. In response to the first position being within the field of view of the i-th gimbal camera, it is determined whether the i-th gimbal camera has captured a fire point within a target time window, wherein the target time window is determined based on the time when the first gimbal camera captures the fire point; In response to the i-th gimbal camera capturing a fire point within the target time window, the i-th gimbal camera is added to the candidate camera set; After the traversal is completed, the gimbal camera in the candidate camera set is recorded as the second gimbal camera, and it is determined whether the number of the second gimbal camera is greater than 1.
4. The fire point location method according to claim 3, characterized in that, Before determining whether the number of the second gimbal cameras is greater than 1, the method further includes: The second gimbal camera in the candidate camera set is selected; Wherein, the time difference between any two second gimbal cameras in the selected candidate camera set that capture the fire point is less than or equal to a preset time threshold, and the distance difference between the fire point locations obtained by locating the fire point based on the two second gimbal cameras is less than or equal to a preset distance threshold.
5. The fire point location method according to claim 2, characterized in that, The step of weighting multiple second positions to obtain a third position includes: Obtain the optical center position and line-of-sight direction of each of the second gimbal cameras when capturing the second image; The weight of each second gimbal camera is determined based on the angle between the line of sight of each second gimbal camera and the normal direction of the surface where the fire point is located, wherein the weight is negatively correlated with the angle. Based on the optical center position, line of sight direction and corresponding weight of each second gimbal camera, multiple second positions are weighted and fused to obtain the third position.
6. The fire point location method according to claim 5, characterized in that, Also includes: Select the gimbal camera with the highest weight from each of the second gimbal cameras as the target gimbal camera; Based on the second image captured by the target gimbal camera, the pixel width of the photovoltaic module where the fire point is located is determined; The first distance between the target gimbal camera and the photovoltaic module is calculated based on the pixel width of the photovoltaic module, the physical width of the photovoltaic module, and the focal length of the target gimbal camera. Take a point at the first distance along the line of sight of the target gimbal camera and fuse it with the third position; The merged location is projected onto the layout diagram of the photovoltaic power station to obtain the photovoltaic array area number and photovoltaic module number where the fire point is located, which serves as the fire point location result.
7. The fire point location method according to claim 2, characterized in that, Also includes: In response to the absence of the plurality of second gimbal cameras and the first gimbal camera being stationary, a second distance between the first gimbal camera and the photovoltaic module is calculated based on the pixel width of the photovoltaic module where the fire point is located in the first image, the physical width of the photovoltaic module, and the focal length of the first gimbal camera. Take a point at the second distance along the line of sight of the first gimbal camera, and intersect it with the digital elevation model to obtain the fourth position; In response to the fourth position matching the component position in the photovoltaic power plant layout diagram, the fourth position is taken as the fire point location result; Alternatively, in response to the absence of the plurality of second gimbal cameras and the first gimbal camera being in motion, the step of taking the first position as the fire point location result is performed, and a verification prompt message is issued.
8. The fire point location method according to claim 7, characterized in that, Also includes: In response to the mismatch between the fourth position and the component position in the photovoltaic power station layout diagram, a third distance between the first gimbal camera and the fire point is estimated based on the pixel height and / or pixel area of the flame region in the first image. The fifth position is obtained by intersecting the point at the third distance along the line of sight of the first gimbal camera with the digital elevation model. The fifth position is weighted and summed with the first position to obtain the sixth position; The sixth position is taken as the fire point location result.
9. The fire point location method according to any one of claims 1-8, characterized in that, The process of locating the fire point based on the first image, the first operating parameters, and the digital elevation model of the photovoltaic power station to obtain the first location includes: Obtain the first pixel coordinates of the fire point in the first image; Using the pre-calibrated rotation matrix of the first gimbal camera, and based on the first operating parameters, the first pixel coordinates are converted into a line-of-sight vector in the geodetic coordinate system. Based on the line-of-sight vector, a ray is constructed originating from the optical center of the first gimbal camera; The first position is obtained by solving for the intersection point of the ray and the digital elevation model.
10. The fire point location method according to claim 9, characterized in that, The step of finding the intersection point of the ray and the digital elevation model to obtain the first position includes: Starting from the endpoint of the ray, advance along the ray at a preset step size, and calculate the difference between the elevation of the current point and the ground elevation of the corresponding position in the digital elevation model. In response to the difference changing from positive to negative, interpolation is performed within the search interval, and the position where the difference is zero is taken as the first position, wherein the search interval is the range of forward steps crossed by the difference changing from positive to negative.
11. The fire point location method according to claim 9, characterized in that, The calibration process for the rotation matrix of the gimbal camera includes: Measure the installation position of the gimbal camera; Within the photovoltaic power station, at least three ground feature points with known geodetic coordinates are selected, and these ground feature points are not collinear; The gimbal camera is controlled to align the image center with each of the ground feature points in sequence, and the operating parameters of the gimbal camera corresponding to each of the ground feature points are recorded respectively. Based on the geodetic coordinates of each ground feature point and its corresponding operating parameters, the rotation matrix from the gimbal camera coordinate system to the geodetic coordinate system is calculated.
12. A fire point location system, characterized in that, include: Multiple PTZ cameras are deployed at a photovoltaic power station, and the PTZ cameras are used to capture images of the photovoltaic power station. as well as A terminal device that performs the method according to any one of claims 1-11, wherein the terminal device is communicatively connected to the plurality of gimbal cameras.
13. The fire point location system according to claim 12, characterized in that, Also includes: The drone, communicatively connected to the terminal device, is configured to, in response to receiving a fire point location result sent by the terminal device, perform at least one of the following operations: The location corresponding to the fire point location result is verified by an airborne camera. Perform fire extinguishing operations.
