An abnormal positioning method and system for unmanned aerial vehicle thermal infrared remote sensing
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 齐鲁空天信息研究院
- Filing Date
- 2026-05-13
- Publication Date
- 2026-06-09
AI Technical Summary
Existing UAV thermal infrared remote sensing systems are prone to misjudgment due to visual feature similarity in anomaly identification, and the location reporting information is vague, leading to false alarms and inaccurate location.
By combining temperature measurement to verify visual detection results, an abnormal area confirmation method that integrates AI visual recognition and physical quantity verification is adopted. The geographical location of abnormal targets is estimated by combining UAV flight data. A modular, event-driven system architecture design is used to achieve an efficient and reliable early warning process.
It reduces visual misjudgments, improves the reliability of anomaly detection and the practicality of location information, provides accurate geographic coordinate references, and enhances the operability of early warning.
Smart Images
Figure CN122176429A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent early warning technology for unmanned aerial vehicles (UAVs), and in particular to an anomaly localization method and system using thermal infrared remote sensing from UAVs. Background Technology
[0002] With the development of drone and IoT technologies, remote monitoring using drones equipped with visible light and thermal infrared cameras has become a common technique. Current mainstream architectures are divided into two categories: centralized (cloud processing) and edge (terminal processing), employing artificial intelligence (AI) algorithms, particularly deep learning models, to automatically identify anomalies in images.
[0003] In terms of technical implementation, it typically includes two key steps: anomaly detection and location reporting.
[0004] In anomaly detection, some systems analyze both visible light (RGB) images and thermal infrared images simultaneously to improve accuracy. Visible light images provide rich texture and color information, while thermal infrared images perceive the temperature distribution on an object's surface. By fusing these two types of visual information, AI models can learn richer features, thereby improving detection accuracy to some extent.
[0005] However, these methods still rely on similarity analysis of visual features in images, lacking verification of the target's physical properties—that is, determining whether an object "looks" like an anomaly. In complex environments, many normal objects or phenomena may visually resemble real anomalies. For example, the reflection of sunlight on a metal roof may appear as a high-temperature area in a thermal infrared image, thus being misjudged as an abnormal heat source; evening clouds may also be misidentified as smoke or flames. Such false alarms can interfere with normal operation and reduce user trust in automated systems.
[0006] Regarding location reporting, the geographical location of the detected anomaly must be provided. A common method is to directly record and report the drone's own GPS coordinates when it captured the abnormal image. While this method is simple and easy to implement, the location information provided deviates significantly from the actual location of the abnormal target. Drones typically fly at altitudes of tens of meters or even higher, resulting in a considerable distance between their own position and the target's location. Furthermore, alerts that only include the drone's coordinates are vague and impractical for scenarios requiring ground personnel to respond. Summary of the Invention
[0007] To address the aforementioned issues, this invention proposes an anomaly localization method and system based on UAV thermal infrared remote sensing. It combines temperature measurement with visual detection results to improve the reliability of early warning, and combines UAV flight data to estimate the geographical location of abnormal targets to enhance the practicality of location information.
[0008] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides an anomaly localization method using thermal infrared remote sensing from an unmanned aerial vehicle (UAV), comprising: Candidate anomaly regions and their corresponding confidence scores are obtained by reasoning from the acquired thermal infrared target images. Based on the absolute temperature values at the pixel coordinates of the thermal infrared target image, a temperature matrix is obtained. Based on the position of the candidate anomaly region in the pixel coordinate system, the temperature sub-matrix at the corresponding position is extracted from the temperature matrix. The highest temperature of each candidate anomaly region is determined based on the temperature sub-matrix. The final anomaly region is determined based on the confidence score and the highest temperature of the candidate anomaly regions; The pixel coordinates of the final abnormal region are converted into a unit line-of-sight vector in the world coordinate system. Starting from the current position of the UAV, the geographical location of the abnormal target is obtained by extending the unit line-of-sight vector by a set distance. The set distance is the distance from the UAV to the target point along the direction of the unit line-of-sight vector.
[0009] As an alternative implementation method, the process of obtaining the temperature matrix includes: By analyzing the thermal infrared target image, the infrared radiation intensity corresponding to each pixel is extracted, and the one-dimensional infrared radiation intensity data stream is reconstructed into a two-dimensional matrix corresponding to the image pixels.
[0010] As an alternative implementation method, the two-dimensional matrix is represented as follows: ;in, Pixel coordinates The absolute temperature value at that location, It represents the infrared radiation intensity at the corresponding pixel. It is gain; It's the offset.
[0011] As an alternative implementation, the process of determining the final abnormal region includes: when the confidence score of the candidate abnormal region is greater than a set confidence threshold and the highest temperature is greater than a set absolute temperature threshold, the candidate abnormal region is determined as the final abnormal region.
[0012] As an alternative implementation, the process of converting to a unit line-of-sight vector in the world coordinate system includes: The homogeneous pixel coordinates of the final abnormal region Through camera intrinsic parameter matrix inverse matrix Transform to the camera coordinate system to obtain the unit line-of-sight vector in the camera coordinate system. : ; The unit line-of-sight vector in the camera coordinate system By rotation matrix Transform to the world coordinate system to obtain the unit line-of-sight vector in the world coordinate system. : Rotation matrix It is determined by the attitude angle of the drone and the fixed mounting angle of the camera on the drone body.
[0013] As an alternative implementation method, abnormal target geographic location Represented as: ; in, It is the distance from the drone to the target point along the unit line-of-sight vector direction. , It refers to the ground elevation. The altitude of the drone. It is the unit line-of-sight vector. In the vertical direction The components on; This shows the current location of the drone.
[0014] Secondly, the present invention provides an anomaly localization system based on thermal infrared remote sensing of an unmanned aerial vehicle (UAV), comprising: The visual reasoning module is configured to reason about the acquired thermal infrared target image to obtain candidate abnormal regions and their corresponding confidence scores. The temperature calculation module is configured to obtain a temperature matrix based on the absolute temperature value at the pixel coordinates of the thermal infrared target image, extract the temperature sub-matrix at the corresponding position from the temperature matrix based on the position of the candidate anomaly region in the pixel coordinate system, and determine the highest temperature of each candidate anomaly region based on the temperature sub-matrix. The two-factor validation module is configured to determine the final anomaly region based on the confidence score of the candidate anomaly region and the highest temperature. The positioning module is configured to convert the pixel coordinates of the final abnormal area into a unit line-of-sight vector in the world coordinate system, and then calculate the geographical location of the abnormal target by extending the unit line-of-sight vector along the current position of the UAV from the current position of the UAV to the target point along the direction of the unit line-of-sight vector.
[0015] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.
[0016] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.
[0017] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.
[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention adds a physical verification step, proposing an anomaly area confirmation method that integrates AI visual recognition and physical quantity verification. First, candidate anomaly areas are identified through visual initial screening. Then, by analyzing the raw sensor data from thermal infrared images, the actual physical temperature of the candidate anomaly areas is compared with a threshold, achieving physical verification of the visual initial screening results, reducing false alarms, and improving the reliability of anomaly detection. Simultaneously, a geolocation method integrating UAV telemetry data and target pixel coordinates is proposed. By combining the UAV's real-time position and flight attitude information with the target's pixel coordinates in the image, the target's ground coordinates are calculated, improving the operability of early warning information and enhancing the practicality of location information.
[0019] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0021] Figure 1 This is a flowchart of the anomaly localization method for UAV thermal infrared remote sensing provided in Embodiment 1 of the present invention; Figure 2 This is an overall architecture diagram of the UAV thermal infrared remote sensing anomaly localization method provided in Embodiment 1 of the present invention. Detailed Implementation
[0022] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0024] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0025] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.
[0026] Example 1 This embodiment provides an anomaly localization method using UAV thermal infrared remote sensing, such as... Figure 1 As shown, it includes: Candidate anomaly regions and their corresponding confidence scores are obtained by reasoning from the acquired thermal infrared target images. Based on the absolute temperature values at the pixel coordinates of the thermal infrared target image, a temperature matrix is obtained. Based on the position of the candidate anomaly region in the pixel coordinate system, the temperature sub-matrix at the corresponding position is extracted from the temperature matrix. The highest temperature of each candidate anomaly region is determined based on the temperature sub-matrix. The final anomaly region is determined based on the confidence score and the highest temperature of the candidate anomaly regions; The pixel coordinates of the final abnormal region are converted into a unit line-of-sight vector in the world coordinate system. Starting from the current position of the UAV, the geographical location of the abnormal target is obtained by extending the unit line-of-sight vector by a set distance. The set distance is the distance from the UAV to the target point along the direction of the unit line-of-sight vector.
[0027] In implementing the above solutions, many existing monitoring systems fail to fully leverage the advantages of distributed system architecture in their design. Some systems employ tightly coupled monolithic architectures with strong inter-module dependencies, making independent updates and expansion difficult. Others lack unified, event-driven communication mechanisms, resulting in high data processing latency and failing to meet real-time early warning requirements. In practical applications, the system needs to handle concurrent data streams from a large number of drones and ensure the reliability of large file transfers in unstable network environments. The lack of a loosely coupled, high-concurrency system architecture makes it difficult for the system to operate efficiently and stably.
[0028] Therefore, to address the issues of insufficient system architecture integration and scalability, this embodiment adopts a modular, event-driven architecture design. Through a unified communication protocol, conditional processing logic, collaborative storage mechanism, and closed-loop early warning process, it achieves fully automated processing from data access to accurate alarms.
[0029] like Figure 2 As shown, the overall architecture consists of an MQTT communication module, a data processing module, a MinIO storage module, and an anomaly warning module, forming a complete, efficient, and reliable technical closed loop of data access, processing, storage, analysis, and warning.
[0030] Specifically: (1) MQTT communication module: As a communication hub, it is built based on the MQTT broker in the publish / subscribe pattern and is responsible for asynchronous message communication between modules. Its core function is to decouple the data source and the data processing module to realize an event-driven workflow.
[0031] Its working mechanism is as follows: The front-end data collection platform, such as drones, acts as the message publisher. After completing a data collection, it publishes a JSON-formatted message containing task metadata (including request ID, task type, data type, and storage path) to a predefined topic. The data processing module acts as a subscriber, listening to these topics. Once a new message is published, the MQTT broker immediately pushes the message to the subscribers, triggering subsequent processing. Similarly, after processing is completed and an alert result is generated, the MQTT communication module receives the alert result, and this alert result is published to the corresponding result topic for downstream platforms to respond to.
[0032] (2) Data processing module: As the scheduling core, it listens to the messages published by the MQTT communication module, adopts a multi-process concurrent model, and starts an independent sub-process for each task to ensure real-time response capability.
[0033] Its working mechanism is as follows: The main process is responsible for listening to messages published by the MQTT communication module. Whenever a new task message is received, an independent child process is immediately started to handle the task. After receiving the task data, the child process first parses the message content and extracts the URL of the data file to be processed; then it calls the client of the MinIO storage module to download the corresponding data file. After the data is ready, according to the task type and preset business logic, multi-process scheduling is started, the corresponding processing function is called, and the AI visual anomaly detection inference process is started. After the inference is completed, the result file (including the annotation file and anomaly information) is stored in the MinIO storage module; the parsed message can also be filtered conditionally and stored in the data file of the MinIO storage module (the data file may include visible light / infrared images / videos).
[0034] (3) MinIO storage module: provides scalable object storage services for storing unstructured data such as images and videos, and has built-in data verification and transmission retry mechanisms.
[0035] Its working mechanism is as follows: the drone uploads the raw data files it collects to a designated storage bucket. The data processing module downloads the files through the corresponding API for preliminary detection and uploads the inference result files.
[0036] In the storage client implementation, robust mechanisms for unstable network environments are also designed, including data existence verification, automatic retry upon failure, and progress visualization, which ensure the reliability and integrity of data transmission and are the foundation for ensuring the stable operation of the entire system.
[0037] (4) Anomaly warning module: As the core of intelligent analysis, it is responsible for performing anomaly detection, judgment, verification and location, and generating the final structured warning information.
[0038] Its working mechanism is as follows: if the AI model detects an anomaly, it calls the corresponding pipeline, including the two-factor authentication method and the target geolocation algorithm, and then generates a detailed, structured JSON format warning report, which includes accurate geographic coordinates, anomaly type, timestamp, and URL pointing to the labeled evidence file; if there is no anomaly, it only constructs a default normal report and finally feeds the report back to the MQTT communication module.
[0039] This embodiment is based on an event-driven architecture, which automates the entire process from multimodal data access, high-reliability anomaly confirmation, target location estimation to structured early warning distribution.
[0040] The implementation process of the method in this embodiment will be described in detail below.
[0041] Step 1: Task triggering, receiving and data acquisition.
[0042] The drone uploads the acquired thermal infrared target images to the MinIO storage module and publishes a task message via the MQTT communication module. After receiving the task message, the data processing module starts a sub-process to download the corresponding data file to the local machine for processing.
[0043] Step 2: The identification of abnormal areas by integrating AI visual recognition and physical quantity verification improves the reliability of anomaly detection and helps reduce false alarms.
[0044] Step 2-1: Quick screening of candidate abnormal regions.
[0045] For the acquired thermal infrared target images, a pre-trained deep learning target detection model is used. After rapid inference, the detection results of candidate anomaly regions are obtained. Each detection result includes a bounding box that defines the position and size of the potential anomaly in the image pixel coordinate system. And category labels and corresponding confidence scores, These are the x-coordinate value, y-coordinate value, width, and height, respectively.
[0046] Step 2-2: Physical quantity verification.
[0047] For each candidate anomalous region selected, a second verification procedure is initiated. This stage no longer relies on visual patterns but directly explores the underlying physical properties of the candidate anomalous regions. To this end, a thermal infrared image radiometric calibration and temperature calculation algorithm is implemented to process R-JPEG (Radiometric JPEG, a special format of JPEG images that embeds temperature data of each pixel while retaining the visual information of traditional JPEG images, making the image usable for subsequent temperature analysis and quantization) format images used by drone manufacturers.
[0048] The specific implementation steps are as follows: (1) Structure analysis and data extraction.
[0049] First, the acquired thermal infrared target image is read in as a binary stream and parsed to identify and locate a private data block embedded in the file metadata, defined by the camera manufacturer. This private data block contains unprocessed thermal sensor readings, representing the intensity of infrared radiation received by each thermal sensor pixel, i.e., the Digital Number (DN).
[0050] In R-JPEG files, the raw infrared data is hidden within the Application Segment (APP) marker segment (usually APP1 or APP11) of a regular JPEG file. The parsing engine scans the TIFF / EXIF directory in the file header to find the manufacturer's MakerNote tag. Once the corresponding tag is identified, a data block containing the data offset and length is obtained. Based on the obtained offset and length, a segment of encapsulated raw data is extracted from the JPEG binary stream. This data is typically lossless compressed and then decompressed to restore it to its original 16-bit integer array.
[0051] (2) Temperature matrix reconstruction and temperature value conversion.
[0052] Based on the sensor resolution (640x512), the one-dimensional DN data stream is reconstructed into a two-dimensional DN value matrix corresponding to the image pixels.
[0053] To convert the dimensionless DN value into a physically meaningful temperature value, a radiation calibration formula is defined. This formula is typically a linear or polynomial model, and its general form is: ; in, Pixel coordinates The absolute temperature value at that location, It is the original reading value of the pixel, that is, the infrared radiation intensity corresponding to the pixel. (Gain) and The offset is a calibration parameter determined based on the sensor resolution.
[0054] (3) Temperature statistics and threshold comparison.
[0055] In obtaining the temperature matrix of the entire thermal infrared target image Afterwards, regarding the first bounding boxes of candidate anomaly regions From the complete temperature matrix Extract the corresponding temperature submatrix .
[0056] Then, for the temperature submatrix Statistical analysis of the temperature values was performed to obtain the highest temperature in the candidate anomaly area.
[0057] Finally, the highest temperature is compared with a configurable, physics-based temperature threshold (which can be customized).
[0058] Steps 2-3: Integrated decision-making.
[0059] An anomaly is only definitively confirmed if it simultaneously meets the criteria for both visual and thermodynamic characteristics. This embodiment uses a fusion confirmation function. To formalize this decision-making process, let For the first bounding boxes of candidate anomaly regions The confidence score, For the first The highest temperature in each candidate anomaly region.
[0060] The fusion confirmation function is defined as follows: ; in, Indicates the first One candidate anomaly region was confirmed as a real anomaly; This represents the confidence threshold, used to filter out detection results with low confidence. This represents the absolute temperature threshold, indicating the thermodynamic conditions sufficient to constitute an anomaly.
[0061] Step 3: Locating the geographical location of the abnormal target.
[0062] For the final anomaly areas confirmed by the two-factor authentication method, an estimation method combining UAV telemetry data and target pixel positions is used to calculate the geographic coordinates of the anomaly targets in order to improve the practicality of the location information.
[0063] The specific steps are as follows: (1) Data synchronization and extraction.
[0064] For video data, the SRT (SubRip text, an industry-standard format for video subtitle synchronization, with the file extension .srt, which can be directly edited with a plain text editor) file generated along with the video file is first parsed. This file records the telemetry data for each frame or every few frames in text format. Based on inference, the frame sequence to which the candidate anomaly region belongs is obtained, and the corresponding UAV GPS position and attitude angle are accurately located and extracted.
[0065] For static images, the UAV's GPS position and attitude angles are read from the image's EXIF metadata.
[0066] The target pixel coordinates are the pixel coordinates of the center point of the candidate anomaly region in the pixel coordinate system. .
[0067] (2) Coordinate system and camera intrinsic parameters.
[0068] Establish three key coordinate systems: the World Coordinate System (WCS, usually WGS-84), the UAV Body Coordinate System (BCS), and the Camera Coordinate System (CCS).
[0069] The mapping relationship between 3D points in the camera coordinate system and 2D pixels in the image plane is determined by the camera intrinsic parameter matrix. Define the camera intrinsic parameter matrix. Obtained in advance through a one-time camera calibration process: ; in, and It is the focal length of the camera, expressed in pixels along the x and y axes; and These are the pixel coordinates of the intersection point (image center) between the camera's optical axis and the image sensor plane; This describes the tilt of the image sensor's coordinate axes, typically 0.
[0070] (3) Calculation of line of sight vector and coordinate system transformation.
[0071] First, the homogeneous pixel coordinates of the final abnormal region are... Through camera intrinsic parameter matrix inverse matrix Transform to the camera coordinate system to obtain the direction vector pointing to the target; at the same time, for ease of calculation, normalize it to obtain the unit line-of-sight vector in the camera coordinate system. : .
[0072] Then, the unit line-of-sight vector in the camera coordinate system Through rotation matrix Transform to the world coordinate system, and obtain the unit line-of-sight vector in the world coordinate system. : .
[0073] Wherein, rotation matrix The rotation matrix is determined by both the drone's attitude angle and the camera's fixed mounting angle on the drone's body. It is a transformation matrix that transforms vectors from the camera coordinate system (CCS) to the world coordinate system (WCS). It consists of two consecutive rotations: .
[0074] Specifically: Let be the UAV attitude rotation matrix, representing the rotation from the UAV body coordinate system (BCS) to the world coordinate system (WCS), derived from the analyzed UAV attitude angles (yaw angle). Pitch angle Roll angle The rotation sequence is determined by the common ZYX (yaw-pitch-roll) order, which corresponds to the rotation matrices around the three axes: Rotation around the X-axis (roll angle) ): ; Rotation around the Y-axis (pitch angle) ): ; Rotation around the Z-axis (yaw angle) ): ; The attitude rotation matrix of the UAV is then: .
[0075] A rotation matrix is fixedly mounted for the camera, representing the rotation from the camera coordinate system (CCS) to the UAV body coordinate system (BCS). It is assumed that the camera has a fixed yaw offset relative to the UAV body. Pitch bias Roll bias (If equipped with a gimbal, then it is the real-time three-axis angle of the gimbal), and its calculation principle is the same as above, namely: .
[0076] (4) Solving for the geographical location of abnormal targets.
[0077] Abnormal target geographical location The location is determined by finding the intersection of a ray with the ground plane. This ray is based on the UAV's current GPS position. Starting from the unit line-of-sight vector in the world coordinate system directional extension.
[0078] Assuming the ground is a locally flat plane, then the geographical location of the anomalous target... Represented as: ; in, This is the distance from the UAV to the target point along the unit line-of-sight vector direction, obtained by solving for the intersection of the ray and the plane: ; It refers to the ground elevation. The altitude of the drone. It is the component of the unit line-of-sight vector in the vertical direction (Z-axis).
[0079] The final solution It is a three-dimensional coordinate vector, which can be converted into a standard latitude and longitude format.
[0080] Step 4: Structured early warning generation and feedback.
[0081] First, threshold filtering is performed on anomaly information that is identical in time and space, removing redundant and duplicate alerts. Then, a structured JSON object is created, which is a complete digital archive containing the request ID, two-factor authentication confirmation status, target latitude and longitude calculated by a geolocation algorithm, anomaly type code (e.g., 0 for no anomaly, 1 for open flame, 2 for overheating, etc.), a timestamp accurate to milliseconds, and a URL pointing to an labeled evidence image in MinIO. Finally, the alert information is published to the MQTT results topic, allowing clients to receive reliable and actionable alerts.
[0082] The above-described solution in this embodiment can be applied to application scenarios that reliably detect and locate events such as fires and thermal anomalies, and mainly achieves the following beneficial effects in terms of early warning reliability and information usability.
[0083] (1) Existing technologies mainly rely on AI to judge the visual features of images, which can easily misjudge normal phenomena with similar appearances as abnormalities. This embodiment adds a physical verification step and proposes an abnormal area confirmation method that integrates AI visual recognition and physical quantity verification. First, the AI model performs visual screening to obtain candidate abnormal areas. Then, by analyzing the original sensor data of the thermal infrared image, the real physical temperature of the candidate abnormal area is compared with the threshold to achieve physical verification of the AI detection results. That is, after the AI identifies a suspected target, it further verifies its real physical properties by analyzing the original sensor data. A target is only confirmed as abnormal if it "looks like" and "meets the physical property standards". This dual verification mechanism adds a layer of "physical fact" guarantee to the visual judgment of AI, which helps to reduce false alarms caused by visual confusion and improve the reliability of anomaly detection.
[0084] (2) The existing technology provides UAV coordinates directly that deviates significantly from the ground target, resulting in relatively vague information. This embodiment proposes a geolocation method that integrates UAV telemetry data and target pixel coordinates. By combining the real-time position and flight attitude information of the UAV obtained from the UAV video SRT file or photo EXIF, and the pixel coordinates of the target in the image detected by the AI model, the ground coordinates of the target are calculated through projection calculation. This provides a more direct and valuable location reference for personnel who need to handle the situation on-site, improves the operability of the early warning information, and enhances the practicality of the location information.
[0085] Example 2 This embodiment provides an anomaly localization system based on UAV thermal infrared remote sensing, including: The visual reasoning module is configured to reason about the acquired thermal infrared target image to obtain candidate abnormal regions and their corresponding confidence scores. The temperature calculation module is configured to obtain a temperature matrix based on the absolute temperature value at the pixel coordinates of the thermal infrared target image, extract the temperature sub-matrix at the corresponding position from the temperature matrix based on the position of the candidate anomaly region in the pixel coordinate system, and determine the highest temperature of each candidate anomaly region based on the temperature sub-matrix. The two-factor validation module is configured to determine the final anomaly region based on the confidence score of the candidate anomaly region and the highest temperature. The positioning module is configured to convert the pixel coordinates of the final abnormal area into a unit line-of-sight vector in the world coordinate system, and then calculate the geographical location of the abnormal target by extending the unit line-of-sight vector along the current position of the UAV from the current position of the UAV to the target point along the direction of the unit line-of-sight vector.
[0086] It should be noted that the above modules correspond to the steps described in Embodiment 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.
[0087] In further embodiments, the following is also provided: An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.
[0088] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.
[0089] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.
[0090] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.
[0091] The method in Example 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.
[0092] A computer program product includes a computer program that, when executed by a processor, implements the method described in Embodiment 1.
[0093] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.
[0094] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.
[0095] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.
[0096] Those skilled in the art will recognize that the units and algorithm steps described in connection with the various examples of this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.
[0097] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.
Claims
1. An anomaly localization method using UAV thermal infrared remote sensing, characterized in that, include: Candidate anomaly regions and their corresponding confidence scores are obtained by reasoning from the acquired thermal infrared target images. Based on the absolute temperature values at the pixel coordinates of the thermal infrared target image, a temperature matrix is obtained. Based on the position of the candidate anomaly region in the pixel coordinate system, the temperature sub-matrix at the corresponding position is extracted from the temperature matrix. The highest temperature of each candidate anomaly region is determined based on the temperature sub-matrix. The final anomaly region is determined based on the confidence score and the highest temperature of the candidate anomaly regions; The pixel coordinates of the final abnormal region are converted into a unit line-of-sight vector in the world coordinate system. Starting from the current position of the UAV, the geographical location of the abnormal target is obtained by extending the unit line-of-sight vector by a set distance. The set distance is the distance from the UAV to the target point along the direction of the unit line-of-sight vector.
2. The anomaly localization method using UAV thermal infrared remote sensing as described in claim 1, characterized in that, The process of obtaining the temperature matrix includes: analyzing the thermal infrared target image to extract the infrared radiation intensity corresponding to each pixel, and reconstructing the one-dimensional infrared radiation intensity data stream into a two-dimensional matrix corresponding to the image pixels.
3. The anomaly localization method using UAV thermal infrared remote sensing as described in claim 2, characterized in that, A two-dimensional matrix is represented as: ;in, Pixel coordinates The absolute temperature value at that location, It represents the infrared radiation intensity at the corresponding pixel. It is gain; It's the offset.
4. The anomaly localization method using UAV thermal infrared remote sensing as described in claim 1, characterized in that, The process of determining the final abnormal region includes: when the confidence score of the candidate abnormal region is greater than the set confidence threshold and the highest temperature is greater than the set absolute temperature threshold, the candidate abnormal region is determined as the final abnormal region.
5. The anomaly localization method using UAV thermal infrared remote sensing as described in claim 1, characterized in that, The process of converting to a unit line-of-sight vector in the world coordinate system includes: The homogeneous pixel coordinates of the final abnormal region Through camera intrinsic parameter matrix inverse matrix Transform to the camera coordinate system to obtain the unit line-of-sight vector in the camera coordinate system. : ; The unit line-of-sight vector in the camera coordinate system By rotation matrix Transform to the world coordinate system to obtain the unit line-of-sight vector in the world coordinate system. : .
6. The anomaly localization method using UAV thermal infrared remote sensing as described in claim 1, characterized in that, Abnormal target geographical location Represented as: ； in, It is the distance from the drone to the target point along the unit line-of-sight vector direction. , It refers to the ground elevation. The altitude of the drone. It is the unit line-of-sight vector. In the vertical direction The components on; This shows the current location of the drone.
7. An anomaly localization system based on thermal infrared remote sensing from an unmanned aerial vehicle (UAV), characterized in that, include: The visual reasoning module is configured to reason about the acquired thermal infrared target image to obtain candidate abnormal regions and their corresponding confidence scores. The temperature calculation module is configured to obtain a temperature matrix based on the absolute temperature value at the pixel coordinates of the thermal infrared target image, extract the temperature sub-matrix at the corresponding position from the temperature matrix based on the position of the candidate anomaly region in the pixel coordinate system, and determine the highest temperature of each candidate anomaly region based on the temperature sub-matrix. The two-factor validation module is configured to determine the final anomaly region based on the confidence score of the candidate anomaly region and the highest temperature. The positioning module is configured to convert the pixel coordinates of the final abnormal area into a unit line-of-sight vector in the world coordinate system, and then calculate the geographical location of the abnormal target by extending the unit line-of-sight vector along the current position of the UAV from the current position of the UAV to the target point along the direction of the unit line-of-sight vector.
8. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-6.
10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the method described in any one of claims 1-6.